From nobody Wed Dec 17 04:17:08 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9BD6D20A5E5 for ; Fri, 14 Nov 2025 15:02:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132572; cv=none; b=JHsVjsLK0RlWox0o0W19KrTcdLiq7PlauaVEBhNk/5DKBvvxg99Tudj4cjWEGHGJBHBM6p/qXJttb1Ojwp4mysleudDIXK9jbsOtRzCuYo8Q3lqNP+SLDjOKzB+wh0jbeSvtogtsagdawU2XHaBCPe0/LjdJxz/KWFAqGsHzKTQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132572; c=relaxed/simple; bh=Dsxrkf28GQuyqD4QjCXp2aGE+3iNdRSDLPbA+0swbpk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hbblaqotXvinYhFhenpLDEdSn6d6TVbHw5433gL9Nm+VGaFDuG2ictXTx/Yn12CQANRZp2+g51dIB2pcXh34rOTT7aDwYU4s5cUhD9lkBTgomeqIZAjMGNmrXjfYdyudP/16Oia3QslT9PgxO7xRqcmf9q64e1JMd7O79QtEfFc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=El2tDJYB; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="El2tDJYB" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1763132569; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=b4IT63puubgpuedNrSBejwiXJhFoMe1SM8s9x/q/mXU=; b=El2tDJYBnCVYoSdRQPjKeDzvdgqTVNxwzPLwzCdN3yldYUIL246S/p5yw6ekPbcgrj57m+ y1tTz+nBwczGNojYR3X97gpn3AbMacQBh7oWAl2uqe8/dfGCKEEpBKq/QAcY2NBhtS3ZBM vuag4msejhD/x/hBGdktlHToCmr4zy4= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-563-NweJLrEmMpKdNLpvmtUD0Q-1; Fri, 14 Nov 2025 10:02:45 -0500 X-MC-Unique: NweJLrEmMpKdNLpvmtUD0Q-1 X-Mimecast-MFC-AGG-ID: NweJLrEmMpKdNLpvmtUD0Q_1763132559 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 3FF231954230; Fri, 14 Nov 2025 15:02:36 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.45.226.10]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id C56B419560B9; Fri, 14 Nov 2025 15:02:19 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, rcu@vger.kernel.org, x86@kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Josh Poimboeuf , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Paolo Bonzini , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [PATCH v7 01/31] objtool: Make validate_call() recognize indirect calls to pv_ops[] Date: Fri, 14 Nov 2025 16:01:03 +0100 Message-ID: <20251114150133.1056710-2-vschneid@redhat.com> In-Reply-To: <20251114150133.1056710-1-vschneid@redhat.com> References: <20251114150133.1056710-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" call_dest_name() does not get passed the file pointer of validate_call(), which means its invocation of insn_reloc() will always return NULL. Make it take a file pointer. While at it, make sure call_dest_name() uses arch_dest_reloc_offset(), otherwise it gets the pv_ops[] offset wrong. Fabricating an intentional warning shows the change; previously: vmlinux.o: warning: objtool: __flush_tlb_all_noinstr+0x4: call to {dynami= c}() leaves .noinstr.text section now: vmlinux.o: warning: objtool: __flush_tlb_all_noinstr+0x4: call to pv_ops[= 1]() leaves .noinstr.text section Signed-off-by: Valentin Schneider Acked-by: Josh Poimboeuf --- tools/objtool/check.c | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/tools/objtool/check.c b/tools/objtool/check.c index 9004fbc067693..12b6967e5fd0d 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -3325,7 +3325,7 @@ static inline bool func_uaccess_safe(struct symbol *f= unc) return false; } =20 -static inline const char *call_dest_name(struct instruction *insn) +static inline const char *call_dest_name(struct objtool_file *file, struct= instruction *insn) { static char pvname[19]; struct reloc *reloc; @@ -3334,9 +3334,9 @@ static inline const char *call_dest_name(struct instr= uction *insn) if (insn_call_dest(insn)) return insn_call_dest(insn)->name; =20 - reloc =3D insn_reloc(NULL, insn); + reloc =3D insn_reloc(file, insn); if (reloc && !strcmp(reloc->sym->name, "pv_ops")) { - idx =3D (reloc_addend(reloc) / sizeof(void *)); + idx =3D (arch_dest_reloc_offset(reloc_addend(reloc)) / sizeof(void *)); snprintf(pvname, sizeof(pvname), "pv_ops[%d]", idx); return pvname; } @@ -3415,17 +3415,19 @@ static int validate_call(struct objtool_file *file, { if (state->noinstr && state->instr <=3D 0 && !noinstr_call_dest(file, insn, insn_call_dest(insn))) { - WARN_INSN(insn, "call to %s() leaves .noinstr.text section", call_dest_n= ame(insn)); + WARN_INSN(insn, "call to %s() leaves .noinstr.text section", call_dest_n= ame(file, insn)); return 1; } =20 if (state->uaccess && !func_uaccess_safe(insn_call_dest(insn))) { - WARN_INSN(insn, "call to %s() with UACCESS enabled", call_dest_name(insn= )); + WARN_INSN(insn, "call to %s() with UACCESS enabled", + call_dest_name(file, insn)); return 1; } =20 if (state->df) { - WARN_INSN(insn, "call to %s() with DF set", call_dest_name(insn)); + WARN_INSN(insn, "call to %s() with DF set", + call_dest_name(file, insn)); return 1; } =20 --=20 2.51.0 From nobody Wed Dec 17 04:17:08 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 47E98328260 for ; Fri, 14 Nov 2025 15:03:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132590; cv=none; b=Zq8QJbfZOeDltjvuFcfN6Ryb0suLuyCBzOfd+lHr50t2glCw3YSwKqvciH4iQKbDfifg7OmS7h3lY1W6yvHHnYGZXac4AbvXeUB+gQpN31bLySzg6D2t5+qQnT7Qd0eZ6EEU5GqJBZY9b1hqJbC069mUgyyPXfE5q+2eUkYZYHs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132590; c=relaxed/simple; bh=8Uv9Hr++vH4bMWjZP5EdI6a/+f8GIcJbi8wyI5DKJ5E=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=H/WRkV9pJBQvLqsln9Irmbqgvi7fezjWsCYEK19hZHMJCEv/SOy2GrtgAHyPm7RV8RP78r+RoXnt/t2cnBigW2gvzX4wLObpe+IIQj+gIdAg6zo8I+RpXw3mVU8XRJEe9dh8xNMeGw49JFs2Jt4R/vIpVjgkbIU09+HcebC/3VA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=a4CdBKtO; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="a4CdBKtO" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1763132588; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gOlu8hBDGdeSMsm+Xh5M8vOkl2dHtZBdnmdHu3tigmo=; b=a4CdBKtO2UlqDklDpk0j98dV3Zsgn0OZz/V9Rr2gpnlZotkJT2GOIMiKUC/fuwHxLp5bjl XaO4r7Mj5KBu0n5ygfPjWaBsHaK3qtV9B4rm/LRxY6EYjJlvotNzhewu4t4J4Wy0YQ9o1z xmf8H+VrUBR3HvG+7RC0xo6HmweVmsA= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-30-eCuXdCzbMACz_ODxDptErw-1; Fri, 14 Nov 2025 10:03:03 -0500 X-MC-Unique: eCuXdCzbMACz_ODxDptErw-1 X-Mimecast-MFC-AGG-ID: eCuXdCzbMACz_ODxDptErw_1763132572 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 976E819560BC; Fri, 14 Nov 2025 15:02:51 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.45.226.10]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id B71AB1956048; Fri, 14 Nov 2025 15:02:36 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, rcu@vger.kernel.org, x86@kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Josh Poimboeuf , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Paolo Bonzini , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [PATCH v7 02/31] objtool: Flesh out warning related to pv_ops[] calls Date: Fri, 14 Nov 2025 16:01:04 +0100 Message-ID: <20251114150133.1056710-3-vschneid@redhat.com> In-Reply-To: <20251114150133.1056710-1-vschneid@redhat.com> References: <20251114150133.1056710-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" I had to look into objtool itself to understand what this warning was about; make it more explicit. Signed-off-by: Valentin Schneider Acked-by: Josh Poimboeuf --- tools/objtool/check.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/objtool/check.c b/tools/objtool/check.c index 12b6967e5fd0d..1efa9f1bf16ba 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -3363,7 +3363,7 @@ static bool pv_call_dest(struct objtool_file *file, s= truct instruction *insn) =20 list_for_each_entry(target, &file->pv_ops[idx].targets, pv_target) { if (!target->sec->noinstr) { - WARN("pv_ops[%d]: %s", idx, target->name); + WARN("pv_ops[%d]: indirect call to %s() leaves .noinstr.text section", = idx, target->name); file->pv_ops[idx].clean =3D false; } } --=20 2.51.0 From nobody Wed Dec 17 04:17:08 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0AF07328260 for ; Fri, 14 Nov 2025 15:03:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132598; cv=none; b=cBOnIx5hYpVv8oKvfODz971qYHE0hsUjvGIyT1vXFGtSKFiHbQUGSdadRYZTV5TMqPLHiwIG8Nn8FDOTdAE33Hx/2SKShiZTmIMYCXe4jSLpCt/lunVDXEJCjP16tiVSOgTOLiJPfcHTNNxILZRQtRbON4eBxdK5uxG6h/XjMFA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132598; c=relaxed/simple; bh=MXaUkaUEuE4uWCCUTHZ1NY8rHU0uR53SWhSpjfmVb28=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dAxgvYWJO8OuXup4ckxlAvTK4Jv21BTKNw2PyeGGvKx62bOW7tt/7rvN1uNUlNyg4nC5zd/zv9WLG9DTr7qroVz4lzN/0Hixw3MxH+4N1LOto+dtAoFkJ6OByRazSh/FDiXn1lCARvEWZ505a+vMCxypetjzIkPs4ZRdLO14tpc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=gYydMF60; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="gYydMF60" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1763132596; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1HcZccm+EFWDfH2vO39s4AfZQgOMomacAD6OZP6p42M=; b=gYydMF600NWp0dUBTd45rRuemMYeap136Jnt5MvSIlRYFgGZmvSsyxRlmSh00VV+W1I+mt gejerxZ0tFyROgMup4Wo12OUbSTXPfmzIlMf5p8WciMKn7pZAC8tYu3wbvmDLFcHDNks3U RZUFXbtyzF0ejGoB2nn71PZneBNkpg4= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-599-caiLroS3O_OR7fs6-zrwAw-1; Fri, 14 Nov 2025 10:03:13 -0500 X-MC-Unique: caiLroS3O_OR7fs6-zrwAw-1 X-Mimecast-MFC-AGG-ID: caiLroS3O_OR7fs6-zrwAw_1763132586 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id DAD0D180034F; Fri, 14 Nov 2025 15:03:05 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.45.226.10]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 1CFD019560B9; Fri, 14 Nov 2025 15:02:51 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, rcu@vger.kernel.org, x86@kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: "Paul E. McKenney" , Frederic Weisbecker , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Josh Poimboeuf , Paolo Bonzini , Arnd Bergmann , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [PATCH v7 03/31] rcu: Add a small-width RCU watching counter debug option Date: Fri, 14 Nov 2025 16:01:05 +0100 Message-ID: <20251114150133.1056710-4-vschneid@redhat.com> In-Reply-To: <20251114150133.1056710-1-vschneid@redhat.com> References: <20251114150133.1056710-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" A later commit will reduce the size of the RCU watching counter to free up some bits for another purpose. Paul suggested adding a config option to test the extreme case where the counter is reduced to its minimum usable width for rcutorture to poke at, so do that. Make it only configurable under RCU_EXPERT. While at it, add a comment to explain the layout of context_tracking->state. Link: http://lore.kernel.org/r/4c2cb573-168f-4806-b1d9-164e8276e66a@paulmck= -laptop Suggested-by: Paul E. McKenney Signed-off-by: Valentin Schneider Reviewed-by: Paul E. McKenney Reviewed-by: Frederic Weisbecker --- include/linux/context_tracking_state.h | 44 ++++++++++++++++++++++---- kernel/rcu/Kconfig.debug | 15 +++++++++ 2 files changed, 52 insertions(+), 7 deletions(-) diff --git a/include/linux/context_tracking_state.h b/include/linux/context= _tracking_state.h index 7b8433d5a8efe..0b81248aa03e2 100644 --- a/include/linux/context_tracking_state.h +++ b/include/linux/context_tracking_state.h @@ -18,12 +18,6 @@ enum ctx_state { CT_STATE_MAX =3D 4, }; =20 -/* Odd value for watching, else even. */ -#define CT_RCU_WATCHING CT_STATE_MAX - -#define CT_STATE_MASK (CT_STATE_MAX - 1) -#define CT_RCU_WATCHING_MASK (~CT_STATE_MASK) - struct context_tracking { #ifdef CONFIG_CONTEXT_TRACKING_USER /* @@ -44,9 +38,45 @@ struct context_tracking { #endif }; =20 +/* + * We cram two different things within the same atomic variable: + * + * CT_RCU_WATCHING_START CT_STATE_START + * | | + * v v + * MSB [ RCU watching counter ][ context_state ] LSB + * ^ ^ + * | | + * CT_RCU_WATCHING_END CT_STATE_END + * + * Bits are used from the LSB upwards, so unused bits (if any) will always= be in + * upper bits of the variable. + */ #ifdef CONFIG_CONTEXT_TRACKING +#define CT_SIZE (sizeof(((struct context_tracking *)0)->state) * BITS_PER_= BYTE) + +#define CT_STATE_WIDTH bits_per(CT_STATE_MAX - 1) +#define CT_STATE_START 0 +#define CT_STATE_END (CT_STATE_START + CT_STATE_WIDTH - 1) + +#define CT_RCU_WATCHING_MAX_WIDTH (CT_SIZE - CT_STATE_WIDTH) +#define CT_RCU_WATCHING_WIDTH (IS_ENABLED(CONFIG_RCU_DYNTICKS_TORTURE)= ? 2 : CT_RCU_WATCHING_MAX_WIDTH) +#define CT_RCU_WATCHING_START (CT_STATE_END + 1) +#define CT_RCU_WATCHING_END (CT_RCU_WATCHING_START + CT_RCU_WATCHING= _WIDTH - 1) +#define CT_RCU_WATCHING BIT(CT_RCU_WATCHING_START) + +#define CT_STATE_MASK GENMASK(CT_STATE_END, CT_STATE_START) +#define CT_RCU_WATCHING_MASK GENMASK(CT_RCU_WATCHING_END, CT_RCU_WATCHING_= START) + +#define CT_UNUSED_WIDTH (CT_RCU_WATCHING_MAX_WIDTH - CT_RCU_WATCHING_WIDTH) + +static_assert(CT_STATE_WIDTH + + CT_RCU_WATCHING_WIDTH + + CT_UNUSED_WIDTH =3D=3D + CT_SIZE); + DECLARE_PER_CPU(struct context_tracking, context_tracking); -#endif +#endif /* CONFIG_CONTEXT_TRACKING */ =20 #ifdef CONFIG_CONTEXT_TRACKING_USER static __always_inline int __ct_state(void) diff --git a/kernel/rcu/Kconfig.debug b/kernel/rcu/Kconfig.debug index 12e4c64ebae15..625d75392647b 100644 --- a/kernel/rcu/Kconfig.debug +++ b/kernel/rcu/Kconfig.debug @@ -213,4 +213,19 @@ config RCU_STRICT_GRACE_PERIOD when looking for certain types of RCU usage bugs, for example, too-short RCU read-side critical sections. =20 + +config RCU_DYNTICKS_TORTURE + bool "Minimize RCU dynticks counter size" + depends on RCU_EXPERT && !COMPILE_TEST + default n + help + This option sets the width of the dynticks counter to its + minimum usable value. This minimum width greatly increases + the probability of flushing out bugs involving counter wrap, + but it also increases the probability of extending grace period + durations. This Kconfig option should therefore be avoided in + production due to the consequent increased probability of OOMs. + + This has no value for production and is only for testing. + endmenu # "RCU Debugging" --=20 2.51.0 From nobody Wed Dec 17 04:17:08 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3C0EF328628 for ; Fri, 14 Nov 2025 15:03:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132612; cv=none; b=k2uWLhSNe/AjPUm2HNIZZEdZpqaCuBS4U85fg+4jOeXRkd6JEoR88MrGNhJCEmlvLrejko0GPMNrXbPPrXXhIF7b4JtDHBdcePn/Fry6ZtqE+wgp9bFkDpvoVKBl5RDJ4WGaxpzu6YbMt5//r+wbyo45gTJq3ghH/fWnghqXZDs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132612; c=relaxed/simple; bh=snOpAJVDQ+hRntc+Abp5FCniuZuLGkuG33n3ekWfFvA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=FOciPSh/ljN0huPeHjDC8PUXdzNulm5HQjDmDj8r2L5BShYn3i9F5GcPBfDm9/t8x+n9FKayoUKPCcFIVVhKQaeItD897nn2Q/HJ9pYT+8KynZZ6B+h2jCJZ/a8QRXZ20ZGuwsW/5fWrYR6plmsOewdaEzVHbROgqAWkne2hJVQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=ELclghTz; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ELclghTz" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1763132610; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=W0yyCaz4IBzCuynoK7eJFq8qb8jjHV9aZwpbdt5Xpuk=; b=ELclghTzpwJWjR/UR+37efBuUNW3hHvHIUUhn9zdQ48DCauW/JKaepPcVxtipxLYdfPz1s TJlWlVYi3iWbOA+tzfvPwjIPhRqRVOdJu1ddQWwVZ12J5scAeXGp4aXLrUrSE5IQNdiYXs HS3Fnk+Wp5vdica94wXD+ZwjlDZ+yBI= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-486-neiQKoMHNwSSwPFkEubsAQ-1; Fri, 14 Nov 2025 10:03:27 -0500 X-MC-Unique: neiQKoMHNwSSwPFkEubsAQ-1 X-Mimecast-MFC-AGG-ID: neiQKoMHNwSSwPFkEubsAQ_1763132601 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 621861800107; Fri, 14 Nov 2025 15:03:20 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.45.226.10]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 848421956048; Fri, 14 Nov 2025 15:03:06 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, rcu@vger.kernel.org, x86@kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: "Paul E. McKenney" , Frederic Weisbecker , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Josh Poimboeuf , Paolo Bonzini , Arnd Bergmann , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [PATCH v7 04/31] rcutorture: Make TREE04 use CONFIG_RCU_DYNTICKS_TORTURE Date: Fri, 14 Nov 2025 16:01:06 +0100 Message-ID: <20251114150133.1056710-5-vschneid@redhat.com> In-Reply-To: <20251114150133.1056710-1-vschneid@redhat.com> References: <20251114150133.1056710-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" We now have an RCU_EXPERT config for testing small-sized RCU dynticks counter: CONFIG_RCU_DYNTICKS_TORTURE. Modify scenario TREE04 to exercise to use this config in order to test a ridiculously small counter (2 bits). Link: http://lore.kernel.org/r/4c2cb573-168f-4806-b1d9-164e8276e66a@paulmck= -laptop Suggested-by: Paul E. McKenney Signed-off-by: Valentin Schneider Reviewed-by: Paul E. McKenney Reviewed-by: Frederic Weisbecker --- tools/testing/selftests/rcutorture/configs/rcu/TREE04 | 1 + 1 file changed, 1 insertion(+) diff --git a/tools/testing/selftests/rcutorture/configs/rcu/TREE04 b/tools/= testing/selftests/rcutorture/configs/rcu/TREE04 index dc4985064b3ad..67caf4276bb01 100644 --- a/tools/testing/selftests/rcutorture/configs/rcu/TREE04 +++ b/tools/testing/selftests/rcutorture/configs/rcu/TREE04 @@ -16,3 +16,4 @@ CONFIG_DEBUG_OBJECTS_RCU_HEAD=3Dn CONFIG_RCU_EXPERT=3Dy CONFIG_RCU_EQS_DEBUG=3Dy CONFIG_RCU_LAZY=3Dy +CONFIG_RCU_DYNTICKS_TORTURE=3Dy --=20 2.51.0 From nobody Wed Dec 17 04:17:08 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1760B329C5F for ; Fri, 14 Nov 2025 15:03:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132626; cv=none; b=hrTZz9h4IBAFa2FGjqKUGZ0WsTBIrEVC9PZ+shLKRvufOfIrtuYNZvbFzbShadw/Uab/6egmC5qdKEUcopEHCoZg6XRdBQTFtsDyfNTXqZUpFcx/QrPFcIfOiOxKXwDHJlm3ZXAPm2kCnBzrCVDDbXrKjZiq3SS9y6IXJfnP7ok= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132626; c=relaxed/simple; bh=tUCxDYx2/0NN5QR1DEvZrfPrIv3zYWsV4oVjHSBsnzs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=mgV4dczKw4zkfHE9Eb/OHS612ACEEds8PahHtI8jKJm+Zia3VHNOPwukzed4NKtdGKDJ0JxTuAVI4wVOKx46HJMRrub2uakHOYBjRBLHqIOhN7COEUY4rSWuZDXw17uqftQy84HyVdsL7zb193LIQwiVUuM5+rwOqdBumtRdiUc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=BBE3rZCS; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="BBE3rZCS" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1763132624; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ypuA88VHIrtZCycTQvLRbp4fTjYOiPnVr/lgVBnkLI0=; b=BBE3rZCSc1PsvbM7expgtsu22bcRAFO5tkc2RAY1ZCDkMZHPFibt+xbHXcLw0iLLFOhyOq nDbec7XFNJScVZwtwsfJaTnkfKAqVfrQSlN0tW502o60Y4wPUf2cM4A9nOUCEtHTR1aH+3 kogwAaiTtAu/pWCpA2lwhuL+Hu+7sAo= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-504-SKUOVx5hNtSa3DHoze9ZYQ-1; Fri, 14 Nov 2025 10:03:40 -0500 X-MC-Unique: SKUOVx5hNtSa3DHoze9ZYQ-1 X-Mimecast-MFC-AGG-ID: SKUOVx5hNtSa3DHoze9ZYQ_1763132615 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id E500A18D95C7; Fri, 14 Nov 2025 15:03:34 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.45.226.10]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id DA7DF1956046; Fri, 14 Nov 2025 15:03:20 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, rcu@vger.kernel.org, x86@kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Josh Poimboeuf , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Paolo Bonzini , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [PATCH v7 05/31] jump_label: Add annotations for validating noinstr usage Date: Fri, 14 Nov 2025 16:01:07 +0100 Message-ID: <20251114150133.1056710-6-vschneid@redhat.com> In-Reply-To: <20251114150133.1056710-1-vschneid@redhat.com> References: <20251114150133.1056710-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" From: Josh Poimboeuf Deferring a code patching IPI is unsafe if the patched code is in a noinstr region. In that case the text poke code must trigger an immediate IPI to all CPUs, which can rudely interrupt an isolated NO_HZ CPU running in userspace. Some noinstr static branches may really need to be patched at runtime, despite the resulting disruption. Add DEFINE_STATIC_KEY_*_NOINSTR() variants for those. They don't do anything special yet; that will come later. Signed-off-by: Josh Poimboeuf --- include/linux/jump_label.h | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/include/linux/jump_label.h b/include/linux/jump_label.h index fdb79dd1ebd8c..c4f6240ff4d95 100644 --- a/include/linux/jump_label.h +++ b/include/linux/jump_label.h @@ -388,6 +388,23 @@ struct static_key_false { #define DEFINE_STATIC_KEY_FALSE_RO(name) \ struct static_key_false name __ro_after_init =3D STATIC_KEY_FALSE_INIT =20 +/* + * The _NOINSTR variants are used to tell objtool the static key is allowe= d to + * be used in noinstr code. + * + * They should almost never be used, as they prevent code patching IPIs fr= om + * being deferred, which can be problematic for isolated NOHZ_FULL CPUs ru= nning + * in pure userspace. + * + * If using one of these _NOINSTR variants, please add a comment above the + * definition with the rationale. + */ +#define DEFINE_STATIC_KEY_TRUE_NOINSTR(name) \ + DEFINE_STATIC_KEY_TRUE(name) + +#define DEFINE_STATIC_KEY_FALSE_NOINSTR(name) \ + DEFINE_STATIC_KEY_FALSE(name) + #define DECLARE_STATIC_KEY_FALSE(name) \ extern struct static_key_false name =20 --=20 2.51.0 From nobody Wed Dec 17 04:17:08 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A26BA329388 for ; Fri, 14 Nov 2025 15:03:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132641; cv=none; b=hoJIENokA1Z0beraaczDVXlBrWyn2Vs2YN0lD3Hq4q0J8AaKu6nmy+xDvUq5TknAMZKj8Eog4/WB3YGg/sjAOqmhNUmgm4B0CygLgwczsHFA6vwoQMNB6vwoAXIe4ALmjwAvN8tIoUqyptPxIoyD5HsiNTFkGEKD+yO2yb+/ywE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132641; c=relaxed/simple; bh=xCMOlKiIGKnSNkXBmGdxZOf4glAFhGzcKWNIOlI6Uso=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=VMKRi+1Ff69TDTx+ASxhti21TWdhhV90F0kBrT1OWs4VmCECKkrUfvXHtaljm3bcYPt4TCITc7GwGol6Vl25Hnw2hFwKscxeJr+NTGJrhBk5WYoMboqGF3l+fFdnwJxzK0leKY4o/gLuIvGMBaWatmbiil6YjoR2q9ud1AIF+TY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=A/hWveJr; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="A/hWveJr" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1763132638; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fSiRQxA5I7cbtbKutRpvTlGvB/h9JsDFAWOVN0dt9w8=; b=A/hWveJrAAWkHlXCvzMB08ItOsxCxOKzvT5q2ickctlboQmz0OaehJeBvdQM5BP+4HZ+0T mnqnSZnGL+SxYefhtHyvXUKGj6gfBYrav0tKTe/npKhlAnM7+hXg1GCT6+I4CVQfByBcFr orwgr4D83ubmcO1Zv5l6UbuN6BJVuxc= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-618-VO3NDkKcOP-dF92Nk4JHRQ-1; Fri, 14 Nov 2025 10:03:56 -0500 X-MC-Unique: VO3NDkKcOP-dF92Nk4JHRQ-1 X-Mimecast-MFC-AGG-ID: VO3NDkKcOP-dF92Nk4JHRQ_1763132631 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 54050180047F; Fri, 14 Nov 2025 15:03:50 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.45.226.10]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 694CE1955F43; Fri, 14 Nov 2025 15:03:35 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, rcu@vger.kernel.org, x86@kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Josh Poimboeuf , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Paolo Bonzini , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [PATCH v7 06/31] static_call: Add read-only-after-init static calls Date: Fri, 14 Nov 2025 16:01:08 +0100 Message-ID: <20251114150133.1056710-7-vschneid@redhat.com> In-Reply-To: <20251114150133.1056710-1-vschneid@redhat.com> References: <20251114150133.1056710-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" From: Josh Poimboeuf Deferring a code patching IPI is unsafe if the patched code is in a noinstr region. In that case the text poke code must trigger an immediate IPI to all CPUs, which can rudely interrupt an isolated NO_HZ CPU running in userspace. If a noinstr static call only needs to be patched during boot, its key can be made ro-after-init to ensure it will never be patched at runtime. Signed-off-by: Josh Poimboeuf --- include/linux/static_call.h | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/include/linux/static_call.h b/include/linux/static_call.h index 78a77a4ae0ea8..ea6ca57e2a829 100644 --- a/include/linux/static_call.h +++ b/include/linux/static_call.h @@ -192,6 +192,14 @@ extern long __static_call_return0(void); }; \ ARCH_DEFINE_STATIC_CALL_TRAMP(name, _func) =20 +#define DEFINE_STATIC_CALL_RO(name, _func) \ + DECLARE_STATIC_CALL(name, _func); \ + struct static_call_key __ro_after_init STATIC_CALL_KEY(name) =3D {\ + .func =3D _func, \ + .type =3D 1, \ + }; \ + ARCH_DEFINE_STATIC_CALL_TRAMP(name, _func) + #define DEFINE_STATIC_CALL_NULL(name, _func) \ DECLARE_STATIC_CALL(name, _func); \ struct static_call_key STATIC_CALL_KEY(name) =3D { \ @@ -200,6 +208,14 @@ extern long __static_call_return0(void); }; \ ARCH_DEFINE_STATIC_CALL_NULL_TRAMP(name) =20 +#define DEFINE_STATIC_CALL_NULL_RO(name, _func) \ + DECLARE_STATIC_CALL(name, _func); \ + struct static_call_key __ro_after_init STATIC_CALL_KEY(name) =3D {\ + .func =3D NULL, \ + .type =3D 1, \ + }; \ + ARCH_DEFINE_STATIC_CALL_NULL_TRAMP(name) + #define DEFINE_STATIC_CALL_RET0(name, _func) \ DECLARE_STATIC_CALL(name, _func); \ struct static_call_key STATIC_CALL_KEY(name) =3D { \ --=20 2.51.0 From nobody Wed Dec 17 04:17:08 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 730832F6195 for ; Fri, 14 Nov 2025 15:04:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132657; cv=none; b=l4iDOUNTY+5cmHjkXfUIx1jLGgowfmGWFyzpimUEcBgu7qZJ6p9ZFW3/HTxY6WjZN2DRWPx13hQvpXpEdPsKE9n0NB+4AeuquPCeT0nk35gjJm2y3l7Yc2NkEGmON48oAYaF+yi6uI9rOYXnm5PCHTJNNgTBVcWPkYBwNrOjJ5U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132657; c=relaxed/simple; bh=j1Uj5RiAQzza7RfiG80uIgtVjXdE2uVY1KJiDuyYz2w=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Gu8FfxW1iCtOMM0iVT+hEEcThyXmalxmGJxEtRofEU5lHc5TktNVaGFfX9/sbC/uSb2WbGE4LrADQ4vJcjTWsB2CmS8YyegkDOEkbH9/NzN4XiMQF7vUyNYzEpSbtf7zUXdu+mxM2yNMiolKUA2+Fwq12gS5m4HokzIy/p++dtU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=UZfWfRBx; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="UZfWfRBx" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1763132655; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=c85HG9M5FnMNKHSoA3zkMZkmQw264GRYsrTXaruk+rY=; b=UZfWfRBxWsn5XrovqUf6iyVN+tC+Cs8O5OEwrRdS/cUDOgwB77BogqOyycOmA4ivTPuztO iE0xL8e9RE66wUdfM+3pvnTmf4f4FVMTLF69hg9Nyz1Tw/iERCQJVgn/ZRR3AdHmmmKUlW 2Cc9EF/nJOw319K6kNeA1FlzqYzPFBU= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-686-IKnaOyZsP-KgTt9Crsfdyw-1; Fri, 14 Nov 2025 10:04:11 -0500 X-MC-Unique: IKnaOyZsP-KgTt9Crsfdyw-1 X-Mimecast-MFC-AGG-ID: IKnaOyZsP-KgTt9Crsfdyw_1763132645 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id BA6451800370; Fri, 14 Nov 2025 15:04:04 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.45.226.10]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 9839F19560B9; Fri, 14 Nov 2025 15:03:50 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, rcu@vger.kernel.org, x86@kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Josh Poimboeuf , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Paolo Bonzini , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [PATCH v7 07/31] x86/paravirt: Mark pv_sched_clock static call as __ro_after_init Date: Fri, 14 Nov 2025 16:01:09 +0100 Message-ID: <20251114150133.1056710-8-vschneid@redhat.com> In-Reply-To: <20251114150133.1056710-1-vschneid@redhat.com> References: <20251114150133.1056710-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" Later commits will cause objtool to warn about static calls being used in .noinstr sections in order to safely defer instruction patching IPIs targeted at NOHZ_FULL CPUs. pv_sched_clock is updated in: o __init vmware_paravirt_ops_setup() o __init xen_init_time_common() o kvm_sched_clock_init() <- __init kvmclock_init() o hv_setup_sched_clock() <- __init hv_init_tsc_clocksource() IOW purely init context, and can thus be marked as __ro_after_init. Reported-by: Josh Poimboeuf Signed-off-by: Valentin Schneider --- arch/x86/kernel/paravirt.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c index ab3e172dcc693..34b6fa3fcc045 100644 --- a/arch/x86/kernel/paravirt.c +++ b/arch/x86/kernel/paravirt.c @@ -69,7 +69,7 @@ static u64 native_steal_clock(int cpu) } =20 DEFINE_STATIC_CALL(pv_steal_clock, native_steal_clock); -DEFINE_STATIC_CALL(pv_sched_clock, native_sched_clock); +DEFINE_STATIC_CALL_RO(pv_sched_clock, native_sched_clock); =20 void paravirt_set_sched_clock(u64 (*func)(void)) { --=20 2.51.0 From nobody Wed Dec 17 04:17:08 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 237A932A3EC for ; Fri, 14 Nov 2025 15:04:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132667; cv=none; b=qG+tlPqsIr6kn7XXaniNeB/n7Id936RB3+Y76yvMIgb+EZHY4dhfEP2auvRcJC9i7bTT2geEPIIE94shX76D30YUyRyKNAT93hMIaDpMT2kvipGl6KEP9PpfPvqZFxmV/pmrY1TsJmKCd1ueSq5a/LRatxTDfLRBwD5M5KuWXIg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132667; c=relaxed/simple; bh=F/NVns0WXfkje7SrWm8dD/gRDhkjUjgk+tcFuXTTheg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=X4qA1p+ZfQRYxCCzGxRBUriBkK71vf5K9Dq6FAQAgZ8JaDIGzaVoGa5HrvMlDanv1KfIUeXWic2fnXOrNguW0uuzb9UwIvH2tPERuXeK1PdAmEaaL1PvgZykTp0uw+vX5sE9Iy8FVtgvMMwssi5smy1SIJ4zz5pp5VkQ3qZIkQA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Txug1u5+; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Txug1u5+" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1763132665; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2oVSattGET5cairz4pLrVIWMCsCMvZMhD6IO+zCvSIQ=; b=Txug1u5+Lgf+8p2+NVWbGNP+CCqePxEz9JpVu5Qmqbu+LMKAzE+Djn0VtvRhpLGx/tV/rT zMUf8VZ0IY+10Nv68Qmz8PPQbGS68sXC4NEr/hUhJTGtdOE8qUE2ZIak6v5vyQhCG5NwCF XRojeWQywkH0Ao/9567BNMngtfVNOSg= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-417-hJJuaX7AOP2PLODmDdPtew-1; Fri, 14 Nov 2025 10:04:23 -0500 X-MC-Unique: hJJuaX7AOP2PLODmDdPtew-1 X-Mimecast-MFC-AGG-ID: hJJuaX7AOP2PLODmDdPtew_1763132659 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id DAFDE1800342; Fri, 14 Nov 2025 15:04:18 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.45.226.10]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 3F2E71956046; Fri, 14 Nov 2025 15:04:05 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, rcu@vger.kernel.org, x86@kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Josh Poimboeuf , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Paolo Bonzini , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [PATCH v7 08/31] x86/idle: Mark x86_idle static call as __ro_after_init Date: Fri, 14 Nov 2025 16:01:10 +0100 Message-ID: <20251114150133.1056710-9-vschneid@redhat.com> In-Reply-To: <20251114150133.1056710-1-vschneid@redhat.com> References: <20251114150133.1056710-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" Later commits will cause objtool to warn about static calls being used in .noinstr sections in order to safely defer instruction patching IPIs targeted at NOHZ_FULL CPUs. x86_idle is updated in: o xen_set_default_idle() <- __init xen_arch_setup() o __init select_idle_routine() IOW purely init context, and can thus be marked as __ro_after_init. Reported-by: Josh Poimboeuf Signed-off-by: Valentin Schneider --- arch/x86/kernel/process.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c index 4c718f8adc592..4f0c1868d43d9 100644 --- a/arch/x86/kernel/process.c +++ b/arch/x86/kernel/process.c @@ -771,7 +771,7 @@ void __cpuidle default_idle(void) EXPORT_SYMBOL(default_idle); #endif =20 -DEFINE_STATIC_CALL_NULL(x86_idle, default_idle); +DEFINE_STATIC_CALL_NULL_RO(x86_idle, default_idle); =20 static bool x86_idle_set(void) { --=20 2.51.0 From nobody Wed Dec 17 04:17:08 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 06DB21E285A for ; Fri, 14 Nov 2025 15:04:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132698; cv=none; b=R4Ac1XnU5qbihYV8ibCaVACCExJffa78bnC/vyOOHp1xea3mATS0/SK5ynzZVOslIt8Mtjkwu0sR22NgtAOuQUhwKw7FfjNKXCrRI96RCymttbQXNhqpjsv8591MLRPXiVU89uvtMviYy0YI9epEHnN7wdI6HkXnJUOoTUgaJVw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132698; c=relaxed/simple; bh=0ftFYWMe4QcACe6McWU1zHJQejt2jdkPVMUAuGKy9xM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=V0lPpHTIjJO0E4ucRumPiA5WdxuUUFk4IsqasnTaaNjt4BouDhOR/UgRRK8SUpky100ZwpCZwCUWcR97vTHuuIJv8FwQaI5c+DdpvpsZF1+PGNz2Q88pp5lOoll5a0h6EFQQMhsR3tJ2dbSahx1g3v8nGmfIbRAlyEogngTOgcg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Skacui5S; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Skacui5S" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1763132696; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1F5Hxk9Mh6e5UiSiQQtZ6dU5HQkdaz+pE8DP+qaty64=; b=Skacui5SkGhXJuz+dW12UnTuf4D0YT52Ps+c2zoNjYxOBVzvaXfyjDMILaX/mXw1TmAY/2 hMeZf0qHbAULVdKxNSP0JsBBL1rosbctFmuVhBHZ60yOovMmKGDm2YYexdmSMcK1ize/JP tTvNsphswErrhCmu/3G21kKWNkxA9JE= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-509-e5rvPyJPNhuu33SWJLhDsw-1; Fri, 14 Nov 2025 10:04:38 -0500 X-MC-Unique: e5rvPyJPNhuu33SWJLhDsw-1 X-Mimecast-MFC-AGG-ID: e5rvPyJPNhuu33SWJLhDsw_1763132674 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 3599D180034F; Fri, 14 Nov 2025 15:04:34 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.45.226.10]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 6B58A19560B9; Fri, 14 Nov 2025 15:04:19 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, rcu@vger.kernel.org, x86@kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Josh Poimboeuf , Paolo Bonzini , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [PATCH v7 09/31] x86/paravirt: Mark pv_steal_clock static call as __ro_after_init Date: Fri, 14 Nov 2025 16:01:11 +0100 Message-ID: <20251114150133.1056710-10-vschneid@redhat.com> In-Reply-To: <20251114150133.1056710-1-vschneid@redhat.com> References: <20251114150133.1056710-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" The static call is only ever updated in __init pv_time_init() __init xen_init_time_common() __init vmware_paravirt_ops_setup() __init xen_time_setup_guest( so mark it appropriately as __ro_after_init. Signed-off-by: Valentin Schneider --- arch/x86/kernel/paravirt.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c index 34b6fa3fcc045..f320a9617b1d6 100644 --- a/arch/x86/kernel/paravirt.c +++ b/arch/x86/kernel/paravirt.c @@ -68,7 +68,7 @@ static u64 native_steal_clock(int cpu) return 0; } =20 -DEFINE_STATIC_CALL(pv_steal_clock, native_steal_clock); +DEFINE_STATIC_CALL_RO(pv_steal_clock, native_steal_clock); DEFINE_STATIC_CALL_RO(pv_sched_clock, native_sched_clock); =20 void paravirt_set_sched_clock(u64 (*func)(void)) --=20 2.51.0 From nobody Wed Dec 17 04:17:08 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C890132C922 for ; Fri, 14 Nov 2025 15:04:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132701; cv=none; b=T1dqAIlH9nFmVfHt4v1z+/eoOLDBNAa8iMV3AwULaJI/89NpICec804luqAtC6DMaKB51c2kfYF7tUXXqhnZiXeACFHZiZTlRJKfoQHVnI3IjiuJ8H1EFYIiJFQZ8T3UPwgzqFnbN2RR2y7T9fyc+aEPJPLi9YYBM4QcYamDdvM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132701; c=relaxed/simple; bh=iUlTIJ6eMR+WnNU+CzY8CVbAR+KrvdOMvkUKsD3UqPw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Gx7sqixDJmW5YCnoxiLv/gxxkNki5YcUn7O8KNSxQg9VOLj5ihM7GVxubBP/GIkR3cW6UoLYsoAFdHGmUNf9Z8g8AiKigsn/S0ZZ9FDpzrCt1oxUZlL81atfUP9MrbxamxG72LsdjWL5bzX7N3OJ21XqbghE10SSaeUmqYySXGQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=WdY5mwtg; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="WdY5mwtg" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1763132699; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=IMfkEjR7bo9/GPZWUZrcsZpIJiAh+KjKI2oJCMzgHeQ=; b=WdY5mwtgnT/mz3lHz1azcCjxpA6NLnfD60Yk6tH8wGhJZwrpZd6/CrkyMBqe0gljZwX7DR neDOiZA5mPIP11FNboh0NmMemOr68m9/sS0wtROvlQHKGQsAqe7+gOocPTi/8nCDG+BoLw Q2qAecQDsxL713U+5sTlKBKMSkSCEtU= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-63-pAP02oI1M8itOtKzo3ac4Q-1; Fri, 14 Nov 2025 10:04:56 -0500 X-MC-Unique: pAP02oI1M8itOtKzo3ac4Q-1 X-Mimecast-MFC-AGG-ID: pAP02oI1M8itOtKzo3ac4Q_1763132688 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 84A53180049F; Fri, 14 Nov 2025 15:04:48 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.45.226.10]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id ACB2619560B9; Fri, 14 Nov 2025 15:04:34 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, rcu@vger.kernel.org, x86@kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Andrew Jones , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Josh Poimboeuf , Paolo Bonzini , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [PATCH v7 10/31] riscv/paravirt: Mark pv_steal_clock static call as __ro_after_init Date: Fri, 14 Nov 2025 16:01:12 +0100 Message-ID: <20251114150133.1056710-11-vschneid@redhat.com> In-Reply-To: <20251114150133.1056710-1-vschneid@redhat.com> References: <20251114150133.1056710-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" The static call is only ever updated in: __init pv_time_init() __init xen_time_setup_guest() so mark it appropriately as __ro_after_init. Signed-off-by: Valentin Schneider Reviewed-by: Andrew Jones --- arch/riscv/kernel/paravirt.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/riscv/kernel/paravirt.c b/arch/riscv/kernel/paravirt.c index fa6b0339a65de..dfe8808016fd8 100644 --- a/arch/riscv/kernel/paravirt.c +++ b/arch/riscv/kernel/paravirt.c @@ -30,7 +30,7 @@ static u64 native_steal_clock(int cpu) return 0; } =20 -DEFINE_STATIC_CALL(pv_steal_clock, native_steal_clock); +DEFINE_STATIC_CALL_RO(pv_steal_clock, native_steal_clock); =20 static bool steal_acc =3D true; static int __init parse_no_stealacc(char *arg) --=20 2.51.0 From nobody Wed Dec 17 04:17:08 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5550E1E285A for ; Fri, 14 Nov 2025 15:05:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132715; cv=none; b=qO1usDuIBGwDvCox0r2BBjXLf6OfzP2Wxm+39oopbqJ3PJkW3NJwGIagTFkdaXNZS/1Hw3H4EKAJ7CqNEJ3JBB6ChcV3b/fhwjrbDLKBUFcBXKUVfaHh3gwSlA+Sa3EpB8QsERm2zbiAk+YD4mYz73BU+d4h9lY6xOOnmC6ElRA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132715; c=relaxed/simple; bh=Z/EAI9hFh4H6XJt0ufOWcsWMChKRCUqaHJ27tZfIUEw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=uPtFbjc5IZAuU3E8DTOQeJC6LuwyRcTJvz+KMjRsfnFPfkECsRAjX63NuM+Y97UFrHFoqlQYQllqAJg/3h7147ILAqWLsTmK60ZA75FbFXdmoqogFhpt6GTefBt5J+/UlznmB29rTP7BEsCvgY+4fT0EgqfYJgEQ3bQTAYnMnWc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=GHYSjcz7; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="GHYSjcz7" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1763132713; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=q5MmuDhyyCRvim2U70KrPJRnnsespTvM3TjOrkj7aIY=; b=GHYSjcz7EQO7/rEqHOsQhu1JG+X6+QFTDsf9M9ymLMM0Bsi4j5247q++Th/vR13ktYJft/ lG/KmxkuT3j0B+FhC6Wezueck4kBKHVrbMntCAknLF8+37xUxIg9f9BCNnrWKlmVmuIKdU bxgHm/n7J50zHiazapXYUI0RY8agCSk= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-407-bfH3JdWFP9aA7fMba-3JXg-1; Fri, 14 Nov 2025 10:05:07 -0500 X-MC-Unique: bfH3JdWFP9aA7fMba-3JXg-1 X-Mimecast-MFC-AGG-ID: bfH3JdWFP9aA7fMba-3JXg_1763132702 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 816D61800EF6; Fri, 14 Nov 2025 15:05:02 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.45.226.10]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 0974619560B9; Fri, 14 Nov 2025 15:04:48 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, rcu@vger.kernel.org, x86@kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Josh Poimboeuf , Paolo Bonzini , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [PATCH v7 11/31] loongarch/paravirt: Mark pv_steal_clock static call as __ro_after_init Date: Fri, 14 Nov 2025 16:01:13 +0100 Message-ID: <20251114150133.1056710-12-vschneid@redhat.com> In-Reply-To: <20251114150133.1056710-1-vschneid@redhat.com> References: <20251114150133.1056710-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" The static call is only ever updated in __init pv_time_init() __init xen_time_setup_guest() so mark it appropriately as __ro_after_init. Signed-off-by: Valentin Schneider --- arch/loongarch/kernel/paravirt.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/loongarch/kernel/paravirt.c b/arch/loongarch/kernel/parav= irt.c index b1b51f920b231..9ec3f5c31fdab 100644 --- a/arch/loongarch/kernel/paravirt.c +++ b/arch/loongarch/kernel/paravirt.c @@ -19,7 +19,7 @@ static u64 native_steal_clock(int cpu) return 0; } =20 -DEFINE_STATIC_CALL(pv_steal_clock, native_steal_clock); +DEFINE_STATIC_CALL_RO(pv_steal_clock, native_steal_clock); =20 static bool steal_acc =3D true; =20 --=20 2.51.0 From nobody Wed Dec 17 04:17:08 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 96F8C32D428 for ; Fri, 14 Nov 2025 15:05:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132725; cv=none; b=famNbNA7+SYphEuKyIeZMAKpi5fn2QrBPdC/bwQdzCse0/oJjZw+C2WwzuwOIYK7ydu60z0VWMPFOdV23BOgXaeCJ09ED174oOCoXlE5yzMer8eCOjwN2FLLZ+7jiaTt1aWT+ypGSASl6UQN+UgxNn83dFmxMFZQ+O0oJEq2Pyg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132725; c=relaxed/simple; bh=ZUjlxRLvH+5h1yuAL0L16LTOqROZN6tqZs92aFf/GoA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=nbaO4ts8xXwcRcMpp+iOW1bFPrVmOkDE/KrZx3+XMw+UsA5VBePRxdARDmLP+m1cIZYdZoGPLiIweY6iyUtg3EW1V4OWZetPWQTosUrirj6nWTW8FNS9/E8DBkXnG3R6iTjQ0R4g2S1/qyswU1Jz/pmcvTe7frYvs53So3vKVHg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Oz3IZmNY; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Oz3IZmNY" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1763132722; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mYsxd3auPHrfEyiSRiw1uwn1LfepSkxHHrao83gvKCM=; b=Oz3IZmNY0C8gwU6NG6pT/wEzYaiwkxKUmUIKJ0owrUC947xbXzN5tZ7L8BeTfOs6LrsvcU 2mGWva/WWhw57ph1X+zVVkzB4TXYp+sQJiL/PZkCufEn4BrsIa567A4zfHSCEnyT/Rps65 wPRttTq6j97Gl4Ubf/zmj4irG/swa/s= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-39-6oHxJBeGO0qqNP5iNM1MBA-1; Fri, 14 Nov 2025 10:05:21 -0500 X-MC-Unique: 6oHxJBeGO0qqNP5iNM1MBA-1 X-Mimecast-MFC-AGG-ID: 6oHxJBeGO0qqNP5iNM1MBA_1763132716 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id A25441801306; Fri, 14 Nov 2025 15:05:16 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.45.226.10]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 0658819560B9; Fri, 14 Nov 2025 15:05:02 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, rcu@vger.kernel.org, x86@kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Josh Poimboeuf , Paolo Bonzini , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [PATCH v7 12/31] arm64/paravirt: Mark pv_steal_clock static call as __ro_after_init Date: Fri, 14 Nov 2025 16:01:14 +0100 Message-ID: <20251114150133.1056710-13-vschneid@redhat.com> In-Reply-To: <20251114150133.1056710-1-vschneid@redhat.com> References: <20251114150133.1056710-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" The static call is only ever updated in __init pv_time_init() __init xen_time_setup_guest() so mark it appropriately as __ro_after_init. Signed-off-by: Valentin Schneider --- arch/arm64/kernel/paravirt.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/arm64/kernel/paravirt.c b/arch/arm64/kernel/paravirt.c index aa718d6a9274a..ad28fa23c9228 100644 --- a/arch/arm64/kernel/paravirt.c +++ b/arch/arm64/kernel/paravirt.c @@ -32,7 +32,7 @@ static u64 native_steal_clock(int cpu) return 0; } =20 -DEFINE_STATIC_CALL(pv_steal_clock, native_steal_clock); +DEFINE_STATIC_CALL_RO(pv_steal_clock, native_steal_clock); =20 struct pv_time_stolen_time_region { struct pvclock_vcpu_stolen_time __rcu *kaddr; --=20 2.51.0 From nobody Wed Dec 17 04:17:08 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7DBC132D7F4 for ; Fri, 14 Nov 2025 15:05:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132740; cv=none; b=u/JCUxupgrw9EC7GSfUKVsd/aIzXHpHmI/rSNJ3uhrGfrvHf7mAEmTe3kIq1s0EOmtU/zAEDOGJyJeFlsmyL6ybD52C65IP45FORSmKcAi+ffYgN6653lW/9ffUVK6HtqgaJ/zfh6YgN17lIyQXFGgYuCwNAm26MobU152zPHhI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132740; c=relaxed/simple; bh=+ucWhHOy3jrIJ6vztnRs0M6NezyGIEaogefci+2tifU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=nPxbvILodxXrJpR9NArkS0ESgrzaTTN5GmISerRPXhi5MfyzvXRHP2o8WG8J66W9nuQurdcTaE9sL4cGrq3qqy5crCk8s9rOC3xAsRD94ExU+AzJr/TC2TWn0EejS8ZN1bUnIh7kTR5tgamNdQYVkrL3h34fEureFtlVzg4Znh4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=fw8SbZbg; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="fw8SbZbg" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1763132738; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CCrVovza8CGhJ7LNJd85GGwejUDYWs5pqgpl3zVqzeE=; b=fw8SbZbgW98b3VpR1lPBH4sNNFF6HU1xY85vJpOyA5TY9qQNMeVCwONaE5d2PPJLN0H9WD iROPBXCzI/0kntq8UbcRbWqHxqfhaug95QPcLz/2d3bz0DAL8+O4pgaLL/DWduUfN0D1Gz 1VDjyct0gTXL8/yJpNuech8q3znacE4= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-257-rVqwtSy8N46ZztWuPhAZGw-1; Fri, 14 Nov 2025 10:05:36 -0500 X-MC-Unique: rVqwtSy8N46ZztWuPhAZGw-1 X-Mimecast-MFC-AGG-ID: rVqwtSy8N46ZztWuPhAZGw_1763132730 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 8874118002C7; Fri, 14 Nov 2025 15:05:30 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.45.226.10]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 0EF6D19560B9; Fri, 14 Nov 2025 15:05:16 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, rcu@vger.kernel.org, x86@kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Josh Poimboeuf , Paolo Bonzini , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [PATCH v7 13/31] arm/paravirt: Mark pv_steal_clock static call as __ro_after_init Date: Fri, 14 Nov 2025 16:01:15 +0100 Message-ID: <20251114150133.1056710-14-vschneid@redhat.com> In-Reply-To: <20251114150133.1056710-1-vschneid@redhat.com> References: <20251114150133.1056710-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" The static call is only ever updated in __init xen_time_setup_guest() so mark it appropriately as __ro_after_init. Signed-off-by: Valentin Schneider --- arch/arm/kernel/paravirt.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/arm/kernel/paravirt.c b/arch/arm/kernel/paravirt.c index 7dd9806369fb0..632d8d5e06db3 100644 --- a/arch/arm/kernel/paravirt.c +++ b/arch/arm/kernel/paravirt.c @@ -20,4 +20,4 @@ static u64 native_steal_clock(int cpu) return 0; } =20 -DEFINE_STATIC_CALL(pv_steal_clock, native_steal_clock); +DEFINE_STATIC_CALL_RO(pv_steal_clock, native_steal_clock); --=20 2.51.0 From nobody Wed Dec 17 04:17:08 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D080132E146 for ; Fri, 14 Nov 2025 15:05:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132756; cv=none; b=TzhfKDVNIkppE6ZJ5UT0kfVRFu6RfpVvOWe6F+GKtOM93ZGWbNBvk4S7z/+fg4IzvIl/V16DLhvK++bewopDbzOBQZkhTy6W3C1kxxSXM5/cziXNoSQuYD/Nkm17VfkwKp1xGKzn7B9+sOLj07JT9CRo/bDVQaHrIuSOdakl2lo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132756; c=relaxed/simple; bh=NOP9Xcf5nJeQy8FdKMCOjsfGYThU7jsiFQxoXI2XDxA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Z/SV2Q6nd6YcYQuS71T5THdPkInVIsMtARAESztttPPc9UO6Dz5nGcZvg0J/QRYMguLNV216oXiWTV+qY3YZIbvhSVDs7mJiZ6N6FbYeAK/VFdKb7RGrgniiXJRaUwNKzT3FZPVR/OIYhrZ1m3fmbjmViWpbhmdtZ8OETxo89Yg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Mf7zl8mh; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Mf7zl8mh" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1763132754; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rr6xJ3c+EJv9VO46QEF+ZoB4rWgfcJmzisR3J3XhXsY=; b=Mf7zl8mhB+2qYmnKKeOhEVkM0+VM0cHfGeyXxvVsHD0kOWLCXD5CYma+T1Kj3TIAc1DA69 c8OFJyvPa2VFzZqIh9lpvFyMYtN5kHotV3UruF5r6xiyMB0JfpYEJ8Y7Vh+I2kpg21r/4A xxAw0zPnWRFBAvNVVU51O8ujew3KXNM= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-85-1COJ1lZ0NXqsZtNAsefjiw-1; Fri, 14 Nov 2025 10:05:52 -0500 X-MC-Unique: 1COJ1lZ0NXqsZtNAsefjiw-1 X-Mimecast-MFC-AGG-ID: 1COJ1lZ0NXqsZtNAsefjiw_1763132744 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 6C1A71800370; Fri, 14 Nov 2025 15:05:44 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.45.226.10]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 1448B19560B9; Fri, 14 Nov 2025 15:05:30 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, rcu@vger.kernel.org, x86@kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Josh Poimboeuf , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Paolo Bonzini , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [PATCH v7 14/31] perf/x86/amd: Mark perf_lopwr_cb static call as __ro_after_init Date: Fri, 14 Nov 2025 16:01:16 +0100 Message-ID: <20251114150133.1056710-15-vschneid@redhat.com> In-Reply-To: <20251114150133.1056710-1-vschneid@redhat.com> References: <20251114150133.1056710-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" Later commits will cause objtool to warn about static calls being used in .noinstr sections in order to safely defer instruction patching IPIs targeted at NOHZ_FULL CPUs. perf_lopwr_cb is used in .noinstr code, but is only ever updated in __init amd_brs_lopwr_init(), and can thus be marked as __ro_after_init. Reported-by: Josh Poimboeuf Signed-off-by: Valentin Schneider --- arch/x86/events/amd/brs.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/events/amd/brs.c b/arch/x86/events/amd/brs.c index 06f35a6b58a5b..71d7ba774a063 100644 --- a/arch/x86/events/amd/brs.c +++ b/arch/x86/events/amd/brs.c @@ -423,7 +423,7 @@ void noinstr perf_amd_brs_lopwr_cb(bool lopwr_in) } } =20 -DEFINE_STATIC_CALL_NULL(perf_lopwr_cb, perf_amd_brs_lopwr_cb); +DEFINE_STATIC_CALL_NULL_RO(perf_lopwr_cb, perf_amd_brs_lopwr_cb); EXPORT_STATIC_CALL_TRAMP_GPL(perf_lopwr_cb); =20 void __init amd_brs_lopwr_init(void) --=20 2.51.0 From nobody Wed Dec 17 04:17:08 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B58FE326927 for ; Fri, 14 Nov 2025 15:06:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132767; cv=none; b=N8Ju4O++2yUmiMTybLc8VoInGTDGHgjouGW2YmLNqHsiizz4VZQF/lZXhPfYgxZU+vpJw2Zb+CKqbqghFrDDOnTXlwSfVvyhLiE8CgFTQG4fYApGbKrkC7BFpmlO16G5wAw5oxJf3twT9E8xjdDx+kYlrnn7jzhtAdVweN8K37E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132767; c=relaxed/simple; bh=pKh76Nm+gleSSaDhponuTSE6i9RO/F37+kzRltiJzdQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=MQjrFntinl+Q+DfLVUyIphnzwzaxBbkT6g0ktL+GRbmY7HdBpNxMTAfB8j2nWQx0Lk4z9YTAXQ/Z+JB5IIGMCk6zVFkQH2dNnj2rAwU7MfSbp0ha+Rjy6hCINRaOpRcnbCEzoH4O0vVGkqYbmtT1BNZRnU6JOggtG+ItW8Wqilk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=DqEqH/sv; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="DqEqH/sv" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1763132764; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=s5bTE2UFDn4FhRjqFPlcPPiaXyzSIXCKwLwP8uiIlgI=; b=DqEqH/svpKebqQGCZZSzZeyCZseVE3jHLJQ7la1JCfhla7bz0R2HkkDX5LT9HiSEWX91SC 2lZ2VLdIR3cx0H+UmEXhaDv129Y+CYbtNl6xze1iAon+dslMFn1VCtVQhoK2XxczxL9109 n4jAHnZq8ElTe43AW0I5OWgBMd8J41o= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-491-wjT2W7RgMiuVd7KDFrJugQ-1; Fri, 14 Nov 2025 10:06:03 -0500 X-MC-Unique: wjT2W7RgMiuVd7KDFrJugQ-1 X-Mimecast-MFC-AGG-ID: wjT2W7RgMiuVd7KDFrJugQ_1763132758 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 85E3818AB416; Fri, 14 Nov 2025 15:05:58 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.45.226.10]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id DF79C19560B9; Fri, 14 Nov 2025 15:05:44 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, rcu@vger.kernel.org, x86@kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Josh Poimboeuf , Paolo Bonzini , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [PATCH v7 15/31] sched/clock: Mark sched_clock_running key as __ro_after_init Date: Fri, 14 Nov 2025 16:01:17 +0100 Message-ID: <20251114150133.1056710-16-vschneid@redhat.com> In-Reply-To: <20251114150133.1056710-1-vschneid@redhat.com> References: <20251114150133.1056710-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" sched_clock_running is only ever enabled in the __init functions sched_clock_init() and sched_clock_init_late(), and is never disabled. Mark it __ro_after_init. Signed-off-by: Valentin Schneider --- kernel/sched/clock.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/sched/clock.c b/kernel/sched/clock.c index f5e6dd6a6b3af..c1a028e99d2cd 100644 --- a/kernel/sched/clock.c +++ b/kernel/sched/clock.c @@ -69,7 +69,7 @@ notrace unsigned long long __weak sched_clock(void) } EXPORT_SYMBOL_GPL(sched_clock); =20 -static DEFINE_STATIC_KEY_FALSE(sched_clock_running); +static DEFINE_STATIC_KEY_FALSE_RO(sched_clock_running); =20 #ifdef CONFIG_HAVE_UNSTABLE_SCHED_CLOCK /* --=20 2.51.0 From nobody Wed Dec 17 04:17:08 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6D6E832E147 for ; Fri, 14 Nov 2025 15:06:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132784; cv=none; b=CYSPVnjGHpxb+fMqVKt1qm8MTT0S+bdGIvlBocnsC6sekjHMDelYHV8WU6O2cL/B+/sOKJlz8ycfloJh4icamoV1d+odaSfHEFaYThGfRMpfTu0XLxmBVhI6ZS6222hKjn1BTixtojC7r627eNQhOMO11Nhenp8D4iBEvPGWtWw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132784; c=relaxed/simple; bh=LsiJO6bn0Il9eUWGMvseELuZx5+sZ+NO7YPbdeK+kaE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lR2OO6/atqH4s0WfW5+/9h4QkSyg7J2rZsvfoqi9uIfxGkysIYA36xjVSPXPKaZ6ip1ib5uuC54xcVtg3bis7rc2IXsVsCPln6/d5As8tp4zA/f76qUcnWiRRuBIXbGvjp8Kl+I5rr47ISP/ZZ45RmJktNFai1BvVBloj4AyJ3I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=N/En2ZM4; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="N/En2ZM4" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1763132782; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FsW1nZftMuTt/QMj2LpkXoZxukhojZ+9zXL0gHIBS/o=; b=N/En2ZM4poy4OJlXhM7rvI9r/S3rL+lvYVyb4paGuPeMNfzxeehw0bV1P9VNX0e2Lqofw9 ZALuRQPkY/Q9kYf1QV0ohWByMIQZx0Is2m2dNk6/MIASf+/Ye41Z1rQqXL/31RYz1Up0N7 xZyY3FObhuZ45Qr1Kvx+LnIbcDxYzKc= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-397-mKrJmDmGOguRh1Y052DobQ-1; Fri, 14 Nov 2025 10:06:19 -0500 X-MC-Unique: mKrJmDmGOguRh1Y052DobQ-1 X-Mimecast-MFC-AGG-ID: mKrJmDmGOguRh1Y052DobQ_1763132772 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 82E521800372; Fri, 14 Nov 2025 15:06:12 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.45.226.10]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 07E3B19560B9; Fri, 14 Nov 2025 15:05:58 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, rcu@vger.kernel.org, x86@kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Josh Poimboeuf , Paolo Bonzini , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [PATCH v7 16/31] KVM: VMX: Mark __kvm_is_using_evmcs static key as __ro_after_init Date: Fri, 14 Nov 2025 16:01:18 +0100 Message-ID: <20251114150133.1056710-17-vschneid@redhat.com> In-Reply-To: <20251114150133.1056710-1-vschneid@redhat.com> References: <20251114150133.1056710-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" The static key is only ever enabled in __init hv_init_evmcs() so mark it appropriately as __ro_after_init. Reported-by: Sean Christopherson Signed-off-by: Valentin Schneider Acked-by: Sean Christopherson --- arch/x86/kvm/vmx/vmx_onhyperv.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kvm/vmx/vmx_onhyperv.c b/arch/x86/kvm/vmx/vmx_onhyper= v.c index b9a8b91166d02..ff3d80c9565bb 100644 --- a/arch/x86/kvm/vmx/vmx_onhyperv.c +++ b/arch/x86/kvm/vmx/vmx_onhyperv.c @@ -3,7 +3,7 @@ #include "capabilities.h" #include "vmx_onhyperv.h" =20 -DEFINE_STATIC_KEY_FALSE(__kvm_is_using_evmcs); +DEFINE_STATIC_KEY_FALSE_RO(__kvm_is_using_evmcs); =20 /* * KVM on Hyper-V always uses the latest known eVMCSv1 revision, the assum= ption --=20 2.51.0 From nobody Wed Dec 17 04:17:08 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 28DB832E75F for ; Fri, 14 Nov 2025 15:06:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132799; cv=none; b=QQ+LaryA/EDxyJyBpLezUlfc1A6S6gEEl7h9sRV/RpKJAI/eYncacLCqz1aEvhrJX4ayRNRFz/XgEHJcjhhzRuMTW2m7BnZs+evj18kJ4oyvDx6NZKGLLpPpVgEKsgpYNN4DayU3i+XiT/0+3eIQIjF3pSd2X+a2Rc+wD0LBDqs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132799; c=relaxed/simple; bh=HrXid4rFcLKsbwJAAXBJ1FHcFf0N5jAhis38Mj/DeC8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Sbn4B00eoilzrBQpg21kk/7EA3gkxNlz6xsA1675RtK7l6gVH9LY761dTYM9Gev7emR1vTlU2en2Q7TOZ/URCj28pOg73NTApBpN7/AW4mDv2K2xKEuYS9HuJgam4vA3M8WxrcWEcItLLFVUub4IgFtVF2lNh1fWQNMT0+0GExk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=NImRPB5g; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="NImRPB5g" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1763132797; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0EVYMqufOoIHV82nAeE2u6YNjD+PUPB4KvH6NE2Jsio=; b=NImRPB5g5o7Ey2VzyGjgolg8A5ZeiVof5dSqvW1NeZ1+Zhj205qI2a7EwbnRML61Go+qVU r4aHlhdYiH+75v7ugn50EuBAq7JwAbDBS5CqMXh9W4HMvUQ47mwcidvY8qrOYkyuftXaMt gYp1epuHB0dvh8q+9Z5wLNkuKu9oLKA= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-404-cI01Fc2gNgiloAaXk9-ykA-1; Fri, 14 Nov 2025 10:06:33 -0500 X-MC-Unique: cI01Fc2gNgiloAaXk9-ykA-1 X-Mimecast-MFC-AGG-ID: cI01Fc2gNgiloAaXk9-ykA_1763132786 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 4213318AB432; Fri, 14 Nov 2025 15:06:26 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.45.226.10]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 0671519560B9; Fri, 14 Nov 2025 15:06:12 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, rcu@vger.kernel.org, x86@kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Josh Poimboeuf , Paolo Bonzini , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [PATCH v7 17/31] x86/bugs: Mark cpu_buf_vm_clear key as allowed in .noinstr Date: Fri, 14 Nov 2025 16:01:19 +0100 Message-ID: <20251114150133.1056710-18-vschneid@redhat.com> In-Reply-To: <20251114150133.1056710-1-vschneid@redhat.com> References: <20251114150133.1056710-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" The static key is only ever updated in __init mmio_apply_mitigation Mark it to let objtool know not to warn about it. Signed-off-by: Valentin Schneider --- arch/x86/kernel/cpu/bugs.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c index d7fa03bf51b45..a2d7dc4d2a4a3 100644 --- a/arch/x86/kernel/cpu/bugs.c +++ b/arch/x86/kernel/cpu/bugs.c @@ -196,8 +196,11 @@ DEFINE_STATIC_KEY_FALSE(switch_mm_cond_l1d_flush); * Controls CPU Fill buffer clear before VMenter. This is a subset of * X86_FEATURE_CLEAR_CPU_BUF, and should only be enabled when KVM-only * mitigation is required. + * + * NOINSTR: This static key is only updated during init, so it can't be + * a cause of post-init interference. */ -DEFINE_STATIC_KEY_FALSE(cpu_buf_vm_clear); +DEFINE_STATIC_KEY_FALSE_NOINSTR(cpu_buf_vm_clear); EXPORT_SYMBOL_GPL(cpu_buf_vm_clear); =20 #undef pr_fmt --=20 2.51.0 From nobody Wed Dec 17 04:17:08 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 958D832ED3B for ; Fri, 14 Nov 2025 15:06:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132815; cv=none; b=XBCvIPEjNrsE3Nsnrh5E7Mt3rCQ6lgibBaGQDIcXRP0sTGbSl3/lwWRe7H2lbeAseKecgSTzcoi2+mCLs1QxjVrE6qPkG6MHaD3193pV0sdrMXsXP3vUOUHd2dsbo2uL3ey/zia9b/KI/cIdBcwZogYllL7s185D5GQs/TxRjTs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763132815; c=relaxed/simple; bh=8tjNRyjdqG0mcGJ+o+Eq0+ocygs3MGlgFlVGI75rVTU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ZOOlNCTF6VKEKcAXd03Czyw+wetIIIY+jh4miPkq/GkHdGuJ1dIt3jG/CfdE/5iQzcpNiNs9avKxdjhNsQDWPNnWhAvSKzoJ4XaEyOivjCjRVDNhpVxiNwP0yoHGQq1RJz2Ns8kJTuDP/9h0gT64H0sRwxaA3mCrHNxGn3yDemw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=igKtCKLf; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="igKtCKLf" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1763132812; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jPe85wKAQS8nAPbz9O1eWBHIVRYgj0iBULMk7IyxpE0=; b=igKtCKLfL3onsi7a0GkwqXb1Zl6n9JamxhWFVN/oeh7JJts0S7ZxiPptvkv8CD5VxsxMwY ZAGSVPaiHM6Ilh0sAnM8jIh9egZc6Zxbcn+ZOyHJyWDMGW3nERTH00dlwIVry+wcXMT1Fe mD9mS6mHAk+6aebehRpjBZM2BMOklVg= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-101-LEviGDC8P1mJckMlf_MqCQ-1; Fri, 14 Nov 2025 10:06:48 -0500 X-MC-Unique: LEviGDC8P1mJckMlf_MqCQ-1 X-Mimecast-MFC-AGG-ID: LEviGDC8P1mJckMlf_MqCQ_1763132801 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id A9FC518A1AED; Fri, 14 Nov 2025 15:06:40 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.45.226.10]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id BF9A91956048; Fri, 14 Nov 2025 15:06:26 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, rcu@vger.kernel.org, x86@kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Josh Poimboeuf , Paolo Bonzini , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [PATCH v7 18/31] x86/speculation/mds: Mark cpu_buf_idle_clear key as allowed in .noinstr Date: Fri, 14 Nov 2025 16:01:20 +0100 Message-ID: <20251114150133.1056710-19-vschneid@redhat.com> In-Reply-To: <20251114150133.1056710-1-vschneid@redhat.com> References: <20251114150133.1056710-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Type: text/plain; charset="utf-8" Later commits will cause objtool to warn about static keys being used in .noinstr sections in order to safely defer instruction patching IPIs targeted at NOHZ_FULL CPUs. cpu_buf_idle_clear is used in .noinstr code, and can be modified at runtime (SMT hotplug). Suppressing the text_poke_sync() IPI has little benefits for this key, as hotplug implies eventually going through takedown_cpu() -> stop_machine_cpuslocked() which is going to cause interference on all online CPUs anyway. Mark it to let objtool know not to warn about it. Signed-off-by: Valentin Schneider --- arch/x86/kernel/cpu/bugs.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c index a2d7dc4d2a4a3..39f6a2f9593f8 100644 --- a/arch/x86/kernel/cpu/bugs.c +++ b/arch/x86/kernel/cpu/bugs.c @@ -181,8 +181,13 @@ DEFINE_STATIC_KEY_FALSE(switch_mm_always_ibpb); DEFINE_STATIC_KEY_FALSE(switch_vcpu_ibpb); EXPORT_SYMBOL_GPL(switch_vcpu_ibpb); =20 -/* Control CPU buffer clear before idling (halt, mwait) */ -DEFINE_STATIC_KEY_FALSE(cpu_buf_idle_clear); +/* + * Control CPU buffer clear before idling (halt, mwait) + * + * NOINSTR: This static key is updated during SMT hotplug which itself alr= eady + * causes some interference. + */ +DEFINE_STATIC_KEY_FALSE_NOINSTR(cpu_buf_idle_clear); EXPORT_SYMBOL_GPL(cpu_buf_idle_clear); =20 /* --=20 2.51.0 From nobody Wed Dec 17 04:17:08 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 09377176ADE for ; Fri, 14 Nov 2025 15:11:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763133085; cv=none; b=b2ZzBikzfiQI3bA7dYcg8eOOMhAgMUw8wKcsjavN4mLP2jcgShIOFkGtosWN//YKtM3G//IeRac5HIjwsw7er4Cl6gSb/5vr2WpPsYEK9PYhHd5ypakxMfhIru5A8WSAwq91+PR2sDQSa/dRv1IiM5yFTImPE7gIGL5fTXo0Mp8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763133085; c=relaxed/simple; bh=DjmflE6ccSmR8hXooCtO/38G4Xihytqv9kmTm6nuKB0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=aLfLM6qtUVE3IBw2TkJNjRsq+WPkwGdQZ5+lhUAS0Kpke6pg3sd9EXvXs471oGdK47kTgM0Dc11LKSuEIF0hEf+hRdX4SkzLjWUqlRS5ARnvRrDFk7haiNn0Kvy8ywBtYY2YAmxF1TdbrZGmYdBQJFZhFjOKtRN6sIopSJb4bl8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=VIHU9Zde; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="VIHU9Zde" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1763133083; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6Z88cIt4bkuwH7lP1X8YYrnTJPptFDIGXgV9h1p1EOM=; b=VIHU9ZdeHO9V+vVe1KRi4oRxi87q/2wOIEiI4rh74FSuEpg9zVmiaPzlO3ypaF63NoWD4+ MnjJDwLoC6ffZnVhUc56os2Ms00qshUTy5EA0N6GloftUc93dKDlwc+kP8BSjGpQ0KdhWX BLGEx0ZkIEloLhHr2xvMDtd8J5SGczo= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-141-rdbQPBGSMPqDX7mD8R_GZA-1; Fri, 14 Nov 2025 10:11:20 -0500 X-MC-Unique: rdbQPBGSMPqDX7mD8R_GZA-1 X-Mimecast-MFC-AGG-ID: rdbQPBGSMPqDX7mD8R_GZA_1763133073 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 15A5718AB40A; Fri, 14 Nov 2025 15:11:13 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.45.226.10]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id F2CBF300018D; Fri, 14 Nov 2025 15:10:57 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, rcu@vger.kernel.org, x86@kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Josh Poimboeuf , Paolo Bonzini , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [PATCH v7 19/31] sched/clock, x86: Mark __sched_clock_stable key as allowed in .noinstr Date: Fri, 14 Nov 2025 16:10:46 +0100 Message-ID: <20251114151048.1061644-1-vschneid@redhat.com> In-Reply-To: <20251114150133.1056710-1-vschneid@redhat.com> References: <20251114150133.1056710-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" Later commits will cause objtool to warn about static keys being used in .noinstr sections in order to safely defer instruction patching IPIs targeted at NOHZ_FULL CPUs. __sched_clock_stable is used in .noinstr code, and can be modified at runtime (e.g. time_cpufreq_notifier()). Suppressing the text_poke_sync() IPI has little benefits for this key, as NOHZ_FULL is incompatible with an unstable TSC anyway. Mark it to let objtool know not to warn about it. Signed-off-by: Valentin Schneider --- kernel/sched/clock.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/kernel/sched/clock.c b/kernel/sched/clock.c index c1a028e99d2cd..5ee27349a4811 100644 --- a/kernel/sched/clock.c +++ b/kernel/sched/clock.c @@ -78,8 +78,11 @@ static DEFINE_STATIC_KEY_FALSE_RO(sched_clock_running); * * Similarly we start with __sched_clock_stable_early, thereby assuming we * will become stable, such that there's only a single 1 -> 0 transition. + * + * NOINSTR: an unstable TLC is incompatible with NOHZ_FULL, thus the text + * patching IPI would be the least of our concerns. */ -static DEFINE_STATIC_KEY_FALSE(__sched_clock_stable); +static DEFINE_STATIC_KEY_FALSE_NOINSTR(__sched_clock_stable); static int __sched_clock_stable_early =3D 1; =20 /* --=20 2.51.0 From nobody Wed Dec 17 04:17:08 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 099FB328B49 for ; Fri, 14 Nov 2025 15:12:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763133156; cv=none; b=AMzdx3gZDHPeJraeBEghmZbWYfimAiBnJA7tEP7BS7h46Bhvv1c+EoJ4cpoDfZGODGdAMmx7X6w46/N1UBtgCynZ/GCxYDDMIDH8+QXi+RmwHODq4SbLpLIiCJGYAUV2QqzxX+evHhzo1+K7vH/Uo77++/UjW1Z03Ob7MKa29Zo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763133156; c=relaxed/simple; bh=f6D8yvxbg0lYFiy19xAVin58pXS33Mh16mjnQIK3rjo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=gp0Wf7pz9wDPe79Z3ZJLFHpFrFG1vilaOeM9YJV7RuV2cQPCVvOh57NCK5JgKMOSdma/2Hpmq4qAw80vcOPuXbd5wThSFb2tlQht8wa1gYKVizPt9aGhYat94LFdccnaOhWzUueugCoEvhNfY2UOOH2ghACUWJ0BF/8wt6YQ1yg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Lftjkop3; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Lftjkop3" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1763133154; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KZ5SyK4DoPKkV4Z0L+pFMHxCyW+tdtvFdr+A8RoV6VQ=; b=Lftjkop36pO3Cl5KnmHX3D0uDwIEK2juGnZfGfQxbMeVF7Vet5a2MP8igEQ3XLmLb2I+Tq 5sgoLUDHeMXWqezuJqJKbLnMej1aLHJCpAgG9x5RuB0s5jXOFCdfC+3FSmNtUXRezRAhZF JAm+YGoh8qVvMlUL1tXW0pPn/QNhW+k= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-411-hYML2ISBMs6U1QZt9CCgwQ-1; Fri, 14 Nov 2025 10:12:31 -0500 X-MC-Unique: hYML2ISBMs6U1QZt9CCgwQ-1 X-Mimecast-MFC-AGG-ID: hYML2ISBMs6U1QZt9CCgwQ_1763133147 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id BB3D319560AD; Fri, 14 Nov 2025 15:12:26 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.45.226.10]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id CB1803000198; Fri, 14 Nov 2025 15:12:11 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, rcu@vger.kernel.org, x86@kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Josh Poimboeuf , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Paolo Bonzini , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [PATCH v7 20/31] KVM: VMX: Mark vmx_l1d_should flush and vmx_l1d_flush_cond keys as allowed in .noinstr Date: Fri, 14 Nov 2025 16:10:47 +0100 Message-ID: <20251114151048.1061644-2-vschneid@redhat.com> In-Reply-To: <20251114150133.1056710-1-vschneid@redhat.com> References: <20251114150133.1056710-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" Later commits will cause objtool to warn about static keys being used in .noinstr sections in order to safely defer instruction patching IPIs targeted at NOHZ_FULL CPUs. The VMX keys are used in .noinstr code, and can be modified at runtime (/proc/kernel/vmx* write). However it is not expected that they will be flipped during latency-sensitive operations, and thus shouldn't be a source of interference for NOHZ_FULL CPUs wrt the text patching IPI. Note, smp_text_poke_batch_finish() never defers IPIs if noinstr code is being patched, i.e. this is purely to tell objtool we're okay with updates to that key causing IPIs and to silence the associated objtool warning. Reported-by: Josh Poimboeuf Signed-off-by: Valentin Schneider Acked-by: Sean Christopherson --- arch/x86/kvm/vmx/vmx.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 91b6f2f3edc2a..99936a2af6641 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -203,8 +203,15 @@ module_param(pt_mode, int, S_IRUGO); =20 struct x86_pmu_lbr __ro_after_init vmx_lbr_caps; =20 -static DEFINE_STATIC_KEY_FALSE(vmx_l1d_should_flush); -static DEFINE_STATIC_KEY_FALSE(vmx_l1d_flush_cond); +/* + * NOINSTR: Both of these static keys end up being used in .noinstr sectio= ns, + * however they are only modified: + * - at init + * - from a /proc/kernel/vmx* write + * thus during latency-sensitive operations they should remain stable. + */ +static DEFINE_STATIC_KEY_FALSE_NOINSTR(vmx_l1d_should_flush); +static DEFINE_STATIC_KEY_FALSE_NOINSTR(vmx_l1d_flush_cond); static DEFINE_MUTEX(vmx_l1d_flush_mutex); =20 /* Storage for pre module init parameter parsing */ --=20 2.51.0 From nobody Wed Dec 17 04:17:08 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2F7EC329E72 for ; Fri, 14 Nov 2025 15:15:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763133304; cv=none; b=jSyyX9JjsXJqd26/9jdETl9UAf3kpNqmghDAQPmNRQwbMU39XHlkyYddDkfApeZCcIJt3Pru2+D7qyWU0x09fCaZdonXlSbqi3hjIBd7ydMvhagCr3/GA50l/jsqO5YISyEDtiHkpq62KxTN4dBi0Qsk81NRyLi1SeGOdJs+Hs0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763133304; c=relaxed/simple; bh=ZGFNntgSflJVhOXYJ02XlQ2RYuItBnLdIXYmR6xh+/U=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=iNYtxVg7Xd7bSQ7eGPqM15b0GTCmVyChVIyy0ocJF9wN3tiNg/JbPiLtuORJCTTyESYnexqImWCzde5ef8njoPWLuc5fLu39Z/LKTZyLKQGNbr/jstvOPjfk4a9Ze6z/BQHj9bf7WI4yIgWcX7OuXqkCItkXJkoEX/4g7FahxYw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=YFt3TH4I; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="YFt3TH4I" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1763133302; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hKNAHPDAvmAC9QequSTQUruQa42RH3SMYJaJvkkmTRU=; b=YFt3TH4IgfuzqmSakFe9H7NXpqVAsqsRbk/78kN5JGTZu6c/EU0agOu7DVEExibTqHKxAs Z4Jm0Z6CrezG+z6ps748VvB4nvhSjT35q9dkJd0b3wZm1aGUx13NnoulrwR7v83qSmHLLg Q8Rgo4ObUtnrAnrNJWuN5Zx/ro3QJZw= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-644-7iDwu_fMODCphb54CSyEig-1; Fri, 14 Nov 2025 10:14:58 -0500 X-MC-Unique: 7iDwu_fMODCphb54CSyEig-1 X-Mimecast-MFC-AGG-ID: 7iDwu_fMODCphb54CSyEig_1763133293 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 796421956088; Fri, 14 Nov 2025 15:14:53 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.45.226.10]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 1DFC2180049F; Fri, 14 Nov 2025 15:14:37 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, rcu@vger.kernel.org, x86@kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Josh Poimboeuf , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Paolo Bonzini , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [PATCH v7 21/31] stackleack: Mark stack_erasing_bypass key as allowed in .noinstr Date: Fri, 14 Nov 2025 16:14:18 +0100 Message-ID: <20251114151428.1064524-1-vschneid@redhat.com> In-Reply-To: <20251114150133.1056710-1-vschneid@redhat.com> References: <20251114150133.1056710-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Content-Type: text/plain; charset="utf-8" Later commits will cause objtool to warn about static keys being used in .noinstr sections in order to safely defer instruction patching IPIs targeted at NOHZ_FULL CPUs. stack_erasing_bypass is used in .noinstr code, and can be modified at runti= me (proc/sys/kernel/stack_erasing write). However it is not expected that it will be flipped during latency-sensitive operations, and thus shouldn't be a source of interference wrt the text patching IPI. Mark it to let objtool know not to warn about it. Reported-by: Josh Poimboeuf Signed-off-by: Valentin Schneider --- kernel/kstack_erase.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/kernel/kstack_erase.c b/kernel/kstack_erase.c index e49bb88b4f0a3..99ba44cf939bf 100644 --- a/kernel/kstack_erase.c +++ b/kernel/kstack_erase.c @@ -19,7 +19,11 @@ #include #include =20 -static DEFINE_STATIC_KEY_FALSE(stack_erasing_bypass); +/* + * NOINSTR: This static key can only be modified via its sysctl interface.= It is + * expected it will remain stable during latency-senstive operations. + */ +static DEFINE_STATIC_KEY_FALSE_NOINSTR(stack_erasing_bypass); =20 #ifdef CONFIG_SYSCTL static int stack_erasing_sysctl(const struct ctl_table *table, int write, --=20 2.51.0 From nobody Wed Dec 17 04:17:08 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 62FEF32A3D7 for ; Fri, 14 Nov 2025 15:15:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763133328; cv=none; b=CJup+kDipozcwhWIpeWDg8ps7WO0jQcGPfS5hw8Q5TOSU6umsrSb5yRgezbI/DnRCR3wZMoWPwy6/zDwVT1sNQ+VOAoWPYsBPEK+fmTJmVzS82BP8q0DDVPubP+LgDZWdKGVqp5tQXrCvkkX+v/rTE+PORbTw9BiWsQiRQZYaGE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763133328; c=relaxed/simple; bh=wYkKm/jxRSR58LqyhcDvvVuen6TBm+qsj8h2RmpwnDM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lugW57ctl4W9/zhKJJJ9v6VzKs0ZYUBJBPFP7oNyBr628qBXjq4zkfYSwfRYgW+a7gEu6xNT80oqqFfEov5HURT2QMxHBRBdh68GMuVMythtVXP6jlv5v8k6hsj59IjwQNWaS/ovFjkL37zdoQEuZmEP2xtJpbZxPcA67QDg/TQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=KDlcYp+s; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="KDlcYp+s" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1763133325; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hvGV990aqgS/6XaVWScoJX92mzsp0oEflxykXgSnr9U=; b=KDlcYp+skLCcwqAFhBOzOTFNRbcvXAdKhHfaEae4CqV81JOG1eYXu8+7QVxp6zEjyZhv5l N2Oe5Ift3vjGgH94xxoqK4FF0+qgS0GInulJyjyK0JUYY81+dwl3WpdPakmzQu6Wn9yGgp PHxZx21ILWcbk45h3zJ2AJe0socnPRc= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-608-pc6qRfMRPAqW4KHYPIFD0g-1; Fri, 14 Nov 2025 10:15:20 -0500 X-MC-Unique: pc6qRfMRPAqW4KHYPIFD0g-1 X-Mimecast-MFC-AGG-ID: pc6qRfMRPAqW4KHYPIFD0g_1763133310 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id C3A171956080; Fri, 14 Nov 2025 15:15:09 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.45.226.10]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 20E701800451; Fri, 14 Nov 2025 15:14:53 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, rcu@vger.kernel.org, x86@kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Josh Poimboeuf , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Paolo Bonzini , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [PATCH v7 22/31] objtool: Add noinstr validation for static branches/calls Date: Fri, 14 Nov 2025 16:14:19 +0100 Message-ID: <20251114151428.1064524-2-vschneid@redhat.com> In-Reply-To: <20251114150133.1056710-1-vschneid@redhat.com> References: <20251114150133.1056710-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Content-Type: text/plain; charset="utf-8" From: Josh Poimboeuf Warn about static branches/calls in noinstr regions, unless the corresponding key is RO-after-init or has been manually whitelisted with DEFINE_STATIC_KEY_*_NOINSTR((). Signed-off-by: Josh Poimboeuf Signed-off-by: Valentin Schneider --- include/linux/jump_label.h | 17 +++-- include/linux/objtool.h | 14 ++++ include/linux/static_call.h | 3 + tools/objtool/Documentation/objtool.txt | 34 +++++++++ tools/objtool/check.c | 92 ++++++++++++++++++++++--- tools/objtool/include/objtool/check.h | 1 + tools/objtool/include/objtool/elf.h | 1 + tools/objtool/include/objtool/special.h | 1 + tools/objtool/special.c | 15 +++- 9 files changed, 162 insertions(+), 16 deletions(-) diff --git a/include/linux/jump_label.h b/include/linux/jump_label.h index c4f6240ff4d95..0ea203ebbc493 100644 --- a/include/linux/jump_label.h +++ b/include/linux/jump_label.h @@ -76,6 +76,7 @@ #include #include #include +#include =20 extern bool static_key_initialized; =20 @@ -376,8 +377,9 @@ struct static_key_false { #define DEFINE_STATIC_KEY_TRUE(name) \ struct static_key_true name =3D STATIC_KEY_TRUE_INIT =20 -#define DEFINE_STATIC_KEY_TRUE_RO(name) \ - struct static_key_true name __ro_after_init =3D STATIC_KEY_TRUE_INIT +#define DEFINE_STATIC_KEY_TRUE_RO(name) \ + struct static_key_true name __ro_after_init =3D STATIC_KEY_TRUE_INIT; \ + ANNOTATE_NOINSTR_ALLOWED(name) =20 #define DECLARE_STATIC_KEY_TRUE(name) \ extern struct static_key_true name @@ -385,8 +387,9 @@ struct static_key_false { #define DEFINE_STATIC_KEY_FALSE(name) \ struct static_key_false name =3D STATIC_KEY_FALSE_INIT =20 -#define DEFINE_STATIC_KEY_FALSE_RO(name) \ - struct static_key_false name __ro_after_init =3D STATIC_KEY_FALSE_INIT +#define DEFINE_STATIC_KEY_FALSE_RO(name) \ + struct static_key_false name __ro_after_init =3D STATIC_KEY_FALSE_INIT; \ + ANNOTATE_NOINSTR_ALLOWED(name) =20 /* * The _NOINSTR variants are used to tell objtool the static key is allowe= d to @@ -400,10 +403,12 @@ struct static_key_false { * definition with the rationale. */ #define DEFINE_STATIC_KEY_TRUE_NOINSTR(name) \ - DEFINE_STATIC_KEY_TRUE(name) + DEFINE_STATIC_KEY_TRUE(name); \ + ANNOTATE_NOINSTR_ALLOWED(name) =20 #define DEFINE_STATIC_KEY_FALSE_NOINSTR(name) \ - DEFINE_STATIC_KEY_FALSE(name) + DEFINE_STATIC_KEY_FALSE(name); \ + ANNOTATE_NOINSTR_ALLOWED(name) =20 #define DECLARE_STATIC_KEY_FALSE(name) \ extern struct static_key_false name diff --git a/include/linux/objtool.h b/include/linux/objtool.h index 46ebaa46e6c58..78b73bf65d9a2 100644 --- a/include/linux/objtool.h +++ b/include/linux/objtool.h @@ -34,6 +34,19 @@ static void __used __section(".discard.func_stack_frame_non_standard") \ *__func_stack_frame_non_standard_##func =3D func =20 +#define __ANNOTATE_NOINSTR_ALLOWED(key) \ + static void __used __section(".discard.noinstr_allowed") \ + *__annotate_noinstr_allowed_##key =3D &key + +/* + * This is used to tell objtool that a given static key is safe to be used + * within .noinstr code, and it doesn't need to generate a warning about i= t. + * + * For more information, see tools/objtool/Documentation/objtool.txt, + * "non-RO static key usage in noinstr code" + */ +#define ANNOTATE_NOINSTR_ALLOWED(key) __ANNOTATE_NOINSTR_ALLOWED(key) + /* * STACK_FRAME_NON_STANDARD_FP() is a frame-pointer-specific function igno= re * for the case where a function is intentionally missing frame pointer se= tup, @@ -130,6 +143,7 @@ #define STACK_FRAME_NON_STANDARD_FP(func) #define __ASM_ANNOTATE(label, type) "" #define ASM_ANNOTATE(type) +#define ANNOTATE_NOINSTR_ALLOWED(key) #else .macro UNWIND_HINT type:req sp_reg=3D0 sp_offset=3D0 signal=3D0 .endm diff --git a/include/linux/static_call.h b/include/linux/static_call.h index ea6ca57e2a829..0d4b16d348501 100644 --- a/include/linux/static_call.h +++ b/include/linux/static_call.h @@ -133,6 +133,7 @@ =20 #include #include +#include #include =20 #ifdef CONFIG_HAVE_STATIC_CALL @@ -198,6 +199,7 @@ extern long __static_call_return0(void); .func =3D _func, \ .type =3D 1, \ }; \ + ANNOTATE_NOINSTR_ALLOWED(STATIC_CALL_TRAMP(name)); \ ARCH_DEFINE_STATIC_CALL_TRAMP(name, _func) =20 #define DEFINE_STATIC_CALL_NULL(name, _func) \ @@ -214,6 +216,7 @@ extern long __static_call_return0(void); .func =3D NULL, \ .type =3D 1, \ }; \ + ANNOTATE_NOINSTR_ALLOWED(STATIC_CALL_TRAMP(name)); \ ARCH_DEFINE_STATIC_CALL_NULL_TRAMP(name) =20 #define DEFINE_STATIC_CALL_RET0(name, _func) \ diff --git a/tools/objtool/Documentation/objtool.txt b/tools/objtool/Docume= ntation/objtool.txt index 9e97fc25b2d8a..991e085e10d95 100644 --- a/tools/objtool/Documentation/objtool.txt +++ b/tools/objtool/Documentation/objtool.txt @@ -456,6 +456,40 @@ the objtool maintainers. these special names and does not use module_init() / module_exit() macros to create them. =20 +13. file.o: warning: func()+0x2a: key: non-RO static key usage in noinstr = code + file.o: warning: func()+0x2a: key: non-RO static call usage in noinstr= code + + This means that noinstr function func() uses a static key or + static call named 'key' which can be modified at runtime. This is + discouraged because it prevents code patching IPIs from being + deferred. + + You have the following options: + + 1) Check whether the static key/call in question is only modified + during init. If so, define it as read-only-after-init with + DEFINE_STATIC_KEY_*_RO() or DEFINE_STATIC_CALL_RO(). + + 2) Avoid the runtime patching. For static keys this can be done by + using static_key_enabled() or by getting rid of the static key + altogether if performance is not a concern. + + For static calls, something like the following could be done: + + target =3D static_call_query(foo); + if (target =3D=3D func1) + func1(); + else if (target =3D=3D func2) + func2(); + ... + + 3) Silence the warning by defining the static key/call with + DEFINE_STATIC_*_NOINSTR(). This decision should not + be taken lightly as it may result in code patching IPIs getting + sent to isolated NOHZ_FULL CPUs running in pure userspace. A + comment should be added above the definition explaining the + rationale for the decision. + =20 If the error doesn't seem to make sense, it could be a bug in objtool. Feel free to ask objtool maintainers for help. diff --git a/tools/objtool/check.c b/tools/objtool/check.c index 1efa9f1bf16ba..474ba2fc87ac6 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -982,6 +982,45 @@ static int create_direct_call_sections(struct objtool_= file *file) return 0; } =20 +static int read_noinstr_allowed(struct objtool_file *file) +{ + struct section *rsec; + struct symbol *sym; + struct reloc *reloc; + + rsec =3D find_section_by_name(file->elf, ".rela.discard.noinstr_allowed"); + if (!rsec) + return 0; + + for_each_reloc(rsec, reloc) { + switch (reloc->sym->type) { + case STT_OBJECT: + case STT_FUNC: + sym =3D reloc->sym; + break; + + case STT_SECTION: + sym =3D find_symbol_by_offset(reloc->sym->sec, + reloc_addend(reloc)); + if (!sym) { + WARN_FUNC(reloc->sym->sec, reloc_addend(reloc), + "can't find static key/call symbol"); + return -1; + } + break; + + default: + WARN("unexpected relocation symbol type in %s: %d", + rsec->name, reloc->sym->type); + return -1; + } + + sym->noinstr_allowed =3D 1; + } + + return 0; +} + /* * Warnings shouldn't be reported for ignored functions. */ @@ -1868,6 +1907,8 @@ static int handle_jump_alt(struct objtool_file *file, return -1; } =20 + orig_insn->key =3D special_alt->key; + if (opts.hack_jump_label && special_alt->key_addend & 2) { struct reloc *reloc =3D insn_reloc(file, orig_insn); =20 @@ -2602,6 +2643,10 @@ static int decode_sections(struct objtool_file *file) if (ret) return ret; =20 + ret =3D read_noinstr_allowed(file); + if (ret) + return ret; + return 0; } =20 @@ -3371,9 +3416,9 @@ static bool pv_call_dest(struct objtool_file *file, s= truct instruction *insn) return file->pv_ops[idx].clean; } =20 -static inline bool noinstr_call_dest(struct objtool_file *file, - struct instruction *insn, - struct symbol *func) +static inline bool noinstr_call_allowed(struct objtool_file *file, + struct instruction *insn, + struct symbol *func) { /* * We can't deal with indirect function calls at present; @@ -3393,10 +3438,10 @@ static inline bool noinstr_call_dest(struct objtool= _file *file, return true; =20 /* - * If the symbol is a static_call trampoline, we can't tell. + * Only DEFINE_STATIC_CALL_*_RO allowed. */ if (func->static_call_tramp) - return true; + return func->noinstr_allowed; =20 /* * The __ubsan_handle_*() calls are like WARN(), they only happen when @@ -3409,14 +3454,29 @@ static inline bool noinstr_call_dest(struct objtool= _file *file, return false; } =20 +static char *static_call_name(struct symbol *func) +{ + return func->name + strlen("__SCT__"); +} + static int validate_call(struct objtool_file *file, struct instruction *insn, struct insn_state *state) { - if (state->noinstr && state->instr <=3D 0 && - !noinstr_call_dest(file, insn, insn_call_dest(insn))) { - WARN_INSN(insn, "call to %s() leaves .noinstr.text section", call_dest_n= ame(file, insn)); - return 1; + if (state->noinstr && state->instr <=3D 0) { + struct symbol *dest =3D insn_call_dest(insn); + + if (dest && dest->static_call_tramp) { + if (!dest->noinstr_allowed) { + WARN_INSN(insn, "%s: non-RO static call usage in noinstr", + static_call_name(dest)); + } + + } else if (dest && !noinstr_call_allowed(file, insn, dest)) { + WARN_INSN(insn, "call to %s() leaves .noinstr.text section", + call_dest_name(file, insn)); + return 1; + } } =20 if (state->uaccess && !func_uaccess_safe(insn_call_dest(insn))) { @@ -3481,6 +3541,17 @@ static int validate_return(struct symbol *func, stru= ct instruction *insn, struct return 0; } =20 +static int validate_static_key(struct instruction *insn, struct insn_state= *state) +{ + if (state->noinstr && state->instr <=3D 0 && !insn->key->noinstr_allowed)= { + WARN_INSN(insn, "%s: non-RO static key usage in noinstr", + insn->key->name); + return 1; + } + + return 0; +} + static struct instruction *next_insn_to_validate(struct objtool_file *file, struct instruction *insn) { @@ -3673,6 +3744,9 @@ static int validate_branch(struct objtool_file *file,= struct symbol *func, if (handle_insn_ops(insn, next_insn, &state)) return 1; =20 + if (insn->key) + validate_static_key(insn, &state); + switch (insn->type) { =20 case INSN_RETURN: diff --git a/tools/objtool/include/objtool/check.h b/tools/objtool/include/= objtool/check.h index 00fb745e72339..d79b08f55bcbc 100644 --- a/tools/objtool/include/objtool/check.h +++ b/tools/objtool/include/objtool/check.h @@ -81,6 +81,7 @@ struct instruction { struct symbol *sym; struct stack_op *stack_ops; struct cfi_state *cfi; + struct symbol *key; }; =20 static inline struct symbol *insn_func(struct instruction *insn) diff --git a/tools/objtool/include/objtool/elf.h b/tools/objtool/include/ob= jtool/elf.h index df8434d3b7440..545deba57266a 100644 --- a/tools/objtool/include/objtool/elf.h +++ b/tools/objtool/include/objtool/elf.h @@ -71,6 +71,7 @@ struct symbol { u8 frame_pointer : 1; u8 ignore : 1; u8 nocfi : 1; + u8 noinstr_allowed : 1; struct list_head pv_target; struct reloc *relocs; struct section *group_sec; diff --git a/tools/objtool/include/objtool/special.h b/tools/objtool/includ= e/objtool/special.h index 72d09c0adf1a1..e84d704f3f20e 100644 --- a/tools/objtool/include/objtool/special.h +++ b/tools/objtool/include/objtool/special.h @@ -18,6 +18,7 @@ struct special_alt { bool group; bool jump_or_nop; u8 key_addend; + struct symbol *key; =20 struct section *orig_sec; unsigned long orig_off; diff --git a/tools/objtool/special.c b/tools/objtool/special.c index c80fed8a840ee..d77f3fa4bbbc9 100644 --- a/tools/objtool/special.c +++ b/tools/objtool/special.c @@ -110,13 +110,26 @@ static int get_alt_entry(struct elf *elf, const struc= t special_entry *entry, =20 if (entry->key) { struct reloc *key_reloc; + struct symbol *key; + s64 key_addend; =20 key_reloc =3D find_reloc_by_dest(elf, sec, offset + entry->key); if (!key_reloc) { ERROR_FUNC(sec, offset + entry->key, "can't find key reloc"); return -1; } - alt->key_addend =3D reloc_addend(key_reloc); + + key =3D key_reloc->sym; + key_addend =3D reloc_addend(key_reloc); + + if (key->type =3D=3D STT_SECTION) + key =3D find_symbol_by_offset(key->sec, key_addend & ~3); + + /* embedded keys not supported */ + if (key) { + alt->key =3D key; + alt->key_addend =3D key_addend; + } } =20 return 0; --=20 2.51.0 From nobody Wed Dec 17 04:17:08 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CF0073321BD for ; Fri, 14 Nov 2025 15:15:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763133335; cv=none; b=LrEbTdZOZoVYJb7JhYWKurohAAlzMJKMDL6whYNu8/+orl4E6/gGwF7BhDhVk7csF4Z7LtXsfZ0SeZ8qANSqjUyhnU58UrCtmcSiDIGELsyu4clYMiuGqC3AOCWdp9AQdvTr0Zv1CMJYoq74tfRuWkwBmadSl9Bvma8CIvQfU2A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763133335; c=relaxed/simple; bh=1IlDDCmLB2DMyjMzK/Tdl0cNptbFgBRDV+MskYp8Re4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ty1yqk4r+Q37NqcP2cSzGDDB2KGo1LAhtkWprbdZf6iiVJ6JGmt/sHLrN8OwH7NqeK9yIEr/HmBRjICDqY5QRdDRDocc30a0IkzuSRbaHOl5Gamy9KCiUOqvMUG73ALaKRvuDXe3MGLmV9pxsmsrRRdbZKS0SG3Q56Pq/PdriKY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=CbCe+QLc; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="CbCe+QLc" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1763133333; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4ORd+Fn6+m/VKC6BHOBwuPVWzflJh2ADAar+c08WU+E=; b=CbCe+QLcqNZkQv9Ydm7u15a06Uxq4oTUj2rVb6EcNr08HQ20d/9vlCPlT/G/tMYEoBCrq/ tJeE8qiWjACQ+GzioGJk5yeHya4xDWNpJUW+dr7756Oc3o25StRfBbZiXqNadB5YTSVZhZ jyjECHFTyLs5MD6Wh5pNPRFUrZwCoMo= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-13-tXg0QNLVNguk-o74W8HyWg-1; Fri, 14 Nov 2025 10:15:29 -0500 X-MC-Unique: tXg0QNLVNguk-o74W8HyWg-1 X-Mimecast-MFC-AGG-ID: tXg0QNLVNguk-o74W8HyWg_1763133325 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 077A11955F4A; Fri, 14 Nov 2025 15:15:25 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.45.226.10]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 4BACB1800346; Fri, 14 Nov 2025 15:15:10 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, rcu@vger.kernel.org, x86@kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Josh Poimboeuf , Paolo Bonzini , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [PATCH v7 23/31] module: Add MOD_NOINSTR_TEXT mem_type Date: Fri, 14 Nov 2025 16:14:20 +0100 Message-ID: <20251114151428.1064524-3-vschneid@redhat.com> In-Reply-To: <20251114150133.1056710-1-vschneid@redhat.com> References: <20251114150133.1056710-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Content-Type: text/plain; charset="utf-8" As pointed out by Sean [1], is_kernel_noinstr_text() will return false for an address contained within a module's .noinstr.text section. A later patch will require checking whether a text address is noinstr, and this can unfortunately be the case of modules - KVM is one such case. A module's .noinstr.text section is already tracked as of commit 66e9b0717102 ("kprobes: Prevent probes in .noinstr.text section") for kprobe blacklisting purposes, but via an ad-hoc mechanism. Add a MOD_NOINSTR_TEXT mem_type, and reorganize __layout_sections() so that it maps all the sections in a single invocation. [1]: http://lore.kernel.org/r/Z4qQL89GZ_gk0vpu@google.com Signed-off-by: Valentin Schneider --- include/linux/module.h | 6 ++-- kernel/kprobes.c | 8 ++--- kernel/module/main.c | 76 ++++++++++++++++++++++++++++++++---------- 3 files changed, 66 insertions(+), 24 deletions(-) diff --git a/include/linux/module.h b/include/linux/module.h index e135cc79aceea..c0911973337c6 100644 --- a/include/linux/module.h +++ b/include/linux/module.h @@ -322,6 +322,7 @@ struct mod_tree_node { =20 enum mod_mem_type { MOD_TEXT =3D 0, + MOD_NOINSTR_TEXT, MOD_DATA, MOD_RODATA, MOD_RO_AFTER_INIT, @@ -492,8 +493,6 @@ struct module { void __percpu *percpu; unsigned int percpu_size; #endif - void *noinstr_text_start; - unsigned int noinstr_text_size; =20 #ifdef CONFIG_TRACEPOINTS unsigned int num_tracepoints; @@ -622,12 +621,13 @@ static inline bool module_is_coming(struct module *mo= d) return mod->state =3D=3D MODULE_STATE_COMING; } =20 -struct module *__module_text_address(unsigned long addr); struct module *__module_address(unsigned long addr); +struct module *__module_text_address(unsigned long addr); bool is_module_address(unsigned long addr); bool __is_module_percpu_address(unsigned long addr, unsigned long *can_add= r); bool is_module_percpu_address(unsigned long addr); bool is_module_text_address(unsigned long addr); +bool is_module_noinstr_text_address(unsigned long addr); =20 static inline bool within_module_mem_type(unsigned long addr, const struct module *mod, diff --git a/kernel/kprobes.c b/kernel/kprobes.c index ab8f9fc1f0d17..d60560dddec56 100644 --- a/kernel/kprobes.c +++ b/kernel/kprobes.c @@ -2551,9 +2551,9 @@ static void add_module_kprobe_blacklist(struct module= *mod) kprobe_add_area_blacklist(start, end); } =20 - start =3D (unsigned long)mod->noinstr_text_start; + start =3D (unsigned long)mod->mem[MOD_NOINSTR_TEXT].base; if (start) { - end =3D start + mod->noinstr_text_size; + end =3D start + mod->mem[MOD_NOINSTR_TEXT].size; kprobe_add_area_blacklist(start, end); } } @@ -2574,9 +2574,9 @@ static void remove_module_kprobe_blacklist(struct mod= ule *mod) kprobe_remove_area_blacklist(start, end); } =20 - start =3D (unsigned long)mod->noinstr_text_start; + start =3D (unsigned long)mod->mem[MOD_NOINSTR_TEXT].base; if (start) { - end =3D start + mod->noinstr_text_size; + end =3D start + mod->mem[MOD_NOINSTR_TEXT].size; kprobe_remove_area_blacklist(start, end); } } diff --git a/kernel/module/main.c b/kernel/module/main.c index c66b261849362..1f5bfdbb956a7 100644 --- a/kernel/module/main.c +++ b/kernel/module/main.c @@ -1653,7 +1653,17 @@ bool module_init_layout_section(const char *sname) return module_init_section(sname); } =20 -static void __layout_sections(struct module *mod, struct load_info *info, = bool is_init) +static bool module_noinstr_layout_section(const char *sname) +{ + return strstarts(sname, ".noinstr"); +} + +static bool module_default_layout_section(const char *sname) +{ + return !module_init_layout_section(sname) && !module_noinstr_layout_secti= on(sname); +} + +static void __layout_sections(struct module *mod, struct load_info *info) { unsigned int m, i; =20 @@ -1662,20 +1672,44 @@ static void __layout_sections(struct module *mod, s= truct load_info *info, bool i * Mask of excluded section header flags } */ static const unsigned long masks[][2] =3D { + /* Core */ + { SHF_EXECINSTR | SHF_ALLOC, ARCH_SHF_SMALL }, + { SHF_EXECINSTR | SHF_ALLOC, ARCH_SHF_SMALL }, + { SHF_ALLOC, SHF_WRITE | ARCH_SHF_SMALL }, + { SHF_RO_AFTER_INIT | SHF_ALLOC, ARCH_SHF_SMALL }, + { SHF_WRITE | SHF_ALLOC, ARCH_SHF_SMALL }, + { ARCH_SHF_SMALL | SHF_ALLOC, 0 }, + /* Init */ { SHF_EXECINSTR | SHF_ALLOC, ARCH_SHF_SMALL }, { SHF_ALLOC, SHF_WRITE | ARCH_SHF_SMALL }, { SHF_RO_AFTER_INIT | SHF_ALLOC, ARCH_SHF_SMALL }, { SHF_WRITE | SHF_ALLOC, ARCH_SHF_SMALL }, - { ARCH_SHF_SMALL | SHF_ALLOC, 0 } + { ARCH_SHF_SMALL | SHF_ALLOC, 0 }, }; - static const int core_m_to_mem_type[] =3D { + static bool (*const section_filter[])(const char *) =3D { + /* Core */ + module_default_layout_section, + module_noinstr_layout_section, + module_default_layout_section, + module_default_layout_section, + module_default_layout_section, + module_default_layout_section, + /* Init */ + module_init_layout_section, + module_init_layout_section, + module_init_layout_section, + module_init_layout_section, + module_init_layout_section, + }; + static const int mem_type_map[] =3D { + /* Core */ MOD_TEXT, + MOD_NOINSTR_TEXT, MOD_RODATA, MOD_RO_AFTER_INIT, MOD_DATA, MOD_DATA, - }; - static const int init_m_to_mem_type[] =3D { + /* Init */ MOD_INIT_TEXT, MOD_INIT_RODATA, MOD_INVALID, @@ -1684,16 +1718,16 @@ static void __layout_sections(struct module *mod, s= truct load_info *info, bool i }; =20 for (m =3D 0; m < ARRAY_SIZE(masks); ++m) { - enum mod_mem_type type =3D is_init ? init_m_to_mem_type[m] : core_m_to_m= em_type[m]; + enum mod_mem_type type =3D mem_type_map[m]; =20 for (i =3D 0; i < info->hdr->e_shnum; ++i) { Elf_Shdr *s =3D &info->sechdrs[i]; const char *sname =3D info->secstrings + s->sh_name; =20 - if ((s->sh_flags & masks[m][0]) !=3D masks[m][0] - || (s->sh_flags & masks[m][1]) - || s->sh_entsize !=3D ~0UL - || is_init !=3D module_init_layout_section(sname)) + if ((s->sh_flags & masks[m][0]) !=3D masks[m][0] || + (s->sh_flags & masks[m][1]) || + s->sh_entsize !=3D ~0UL || + !section_filter[m](sname)) continue; =20 if (WARN_ON_ONCE(type =3D=3D MOD_INVALID)) @@ -1733,10 +1767,7 @@ static void layout_sections(struct module *mod, stru= ct load_info *info) info->sechdrs[i].sh_entsize =3D ~0UL; =20 pr_debug("Core section allocation order for %s:\n", mod->name); - __layout_sections(mod, info, false); - - pr_debug("Init section allocation order for %s:\n", mod->name); - __layout_sections(mod, info, true); + __layout_sections(mod, info); } =20 static void module_license_taint_check(struct module *mod, const char *lic= ense) @@ -2625,9 +2656,6 @@ static int find_module_sections(struct module *mod, s= truct load_info *info) } #endif =20 - mod->noinstr_text_start =3D section_objs(info, ".noinstr.text", 1, - &mod->noinstr_text_size); - #ifdef CONFIG_TRACEPOINTS mod->tracepoints_ptrs =3D section_objs(info, "__tracepoints_ptrs", sizeof(*mod->tracepoints_ptrs), @@ -3872,12 +3900,26 @@ struct module *__module_text_address(unsigned long = addr) if (mod) { /* Make sure it's within the text section. */ if (!within_module_mem_type(addr, mod, MOD_TEXT) && + !within_module_mem_type(addr, mod, MOD_NOINSTR_TEXT) && !within_module_mem_type(addr, mod, MOD_INIT_TEXT)) mod =3D NULL; } return mod; } =20 +bool is_module_noinstr_text_address(unsigned long addr) +{ + scoped_guard(preempt) { + struct module *mod =3D __module_address(addr); + + /* Make sure it's within the .noinstr.text section. */ + if (mod) + return within_module_mem_type(addr, mod, MOD_NOINSTR_TEXT); + } + + return false; +} + /* Don't grab lock, we're oopsing. */ void print_modules(void) { --=20 2.51.0 From nobody Wed Dec 17 04:17:08 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 96E4532D0F3 for ; Fri, 14 Nov 2025 15:15:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763133356; cv=none; b=ONAARelgJKhmEgi5qk8eZ7bEG4q6+/tO3jN77/lzCUsZS5dzt4oEdC/oJtPF18N4JejZqFy8gkLzQ4B0ZRZczSEko1eiuvEePSyDHjpzDDF1BE4VHB6GvPoS033xRIlA5RvJBi3CQHlu5Z+aZlxcSI8XFHZ6zwHrcN3AFF1vGKQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763133356; c=relaxed/simple; bh=3vFmGwJ4O5d18d/9pF4r2lPh/GybRx9YL4exPV4e8rw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=or/KWcld0ATKOYk5bKjjr6ADKqYKQF22mSioUBZg9DXKP0bHR3IU5cjO5yi3P0cgieoP6VLC+mD0hzgjctkMDG8NtE0fQmrmZkg5ybdkfaGsm5+8Rhx4sDLEuU9Wk/6oEpZNLFQBVlFabe9z1awRvktcrGjnp2rFqP+TW1L+FCk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Ed7lwCxL; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Ed7lwCxL" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1763133353; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=y9qexTOcRzn1ACshFlnrwwbGyyms8r6ia30n/LGo1pE=; b=Ed7lwCxL/w8AlA5dLI8eeZfj8Kbarp1bV3DSPW43o6queXT3RzPNiSxZvv+ku9zSCpo+9C RWsCYZ0b6XZwqNv7KpCTYmlMzvUys8AtPcse18tjhquqeIUYTRquZV6kSqrDH34uptNfbj qhLFsCwx/PMIK5ECXdp6DD6CTHRLFEQ= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-621-udY8E4xTMwOi_pI-KC5J3g-1; Fri, 14 Nov 2025 10:15:51 -0500 X-MC-Unique: udY8E4xTMwOi_pI-KC5J3g-1 X-Mimecast-MFC-AGG-ID: udY8E4xTMwOi_pI-KC5J3g_1763133341 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 6B966195608D; Fri, 14 Nov 2025 15:15:41 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.45.226.10]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 9C10D1800451; Fri, 14 Nov 2025 15:15:25 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, rcu@vger.kernel.org, x86@kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Nicolas Saenz Julienne , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Josh Poimboeuf , Paolo Bonzini , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [PATCH v7 24/31] context-tracking: Introduce work deferral infrastructure Date: Fri, 14 Nov 2025 16:14:21 +0100 Message-ID: <20251114151428.1064524-4-vschneid@redhat.com> In-Reply-To: <20251114150133.1056710-1-vschneid@redhat.com> References: <20251114150133.1056710-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Content-Type: text/plain; charset="utf-8" smp_call_function() & friends have the unfortunate habit of sending IPIs to isolated, NOHZ_FULL, in-userspace CPUs, as they blindly target all online CPUs. Some callsites can be bent into doing the right, such as done by commit: cc9e303c91f5 ("x86/cpu: Disable frequency requests via aperfmperf IPI for= nohz_full CPUs") Unfortunately, not all SMP callbacks can be omitted in this fashion. However, some of them only affect execution in kernelspace, which means they don't have to be executed *immediately* if the target CPU is in userspace: stashing the callback and executing it upon the next kernel entry would suffice. x86 kernel instruction patching or kernel TLB invalidation are prime examples of it. Reduce the RCU dynticks counter width to free up some bits to be used as a deferred callback bitmask. Add some build-time checks to validate that setup. Presence of CT_RCU_WATCHING in the ct_state prevents queuing deferred work. Later commits introduce the bit:callback mappings. Link: https://lore.kernel.org/all/20210929151723.162004989@infradead.org/ Signed-off-by: Nicolas Saenz Julienne Signed-off-by: Valentin Schneider --- arch/Kconfig | 9 +++ arch/x86/Kconfig | 1 + arch/x86/include/asm/context_tracking_work.h | 16 +++++ include/linux/context_tracking.h | 21 ++++++ include/linux/context_tracking_state.h | 30 +++++--- include/linux/context_tracking_work.h | 24 +++++++ kernel/context_tracking.c | 72 +++++++++++++++++++- kernel/time/Kconfig | 5 ++ 8 files changed, 166 insertions(+), 12 deletions(-) create mode 100644 arch/x86/include/asm/context_tracking_work.h create mode 100644 include/linux/context_tracking_work.h diff --git a/arch/Kconfig b/arch/Kconfig index 61130b88964b9..6cc3965b8c9eb 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -1024,6 +1024,15 @@ config HAVE_CONTEXT_TRACKING_USER_OFFSTACK - No use of instrumentation, unless instrumentation_begin() got called. =20 +config HAVE_CONTEXT_TRACKING_WORK + bool + help + Architecture supports deferring work while not in kernel context. + This is especially useful on setups with isolated CPUs that might + want to avoid being interrupted to perform housekeeping tasks (for + ex. TLB invalidation or icache invalidation). The housekeeping + operations are performed upon re-entering the kernel. + config HAVE_TIF_NOHZ bool help diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index fa3b616af03a2..fa9229c0e0939 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -219,6 +219,7 @@ config X86 select HAVE_CMPXCHG_LOCAL select HAVE_CONTEXT_TRACKING_USER if X86_64 select HAVE_CONTEXT_TRACKING_USER_OFFSTACK if HAVE_CONTEXT_TRACKING_USER + select HAVE_CONTEXT_TRACKING_WORK if X86_64 select HAVE_C_RECORDMCOUNT select HAVE_OBJTOOL_MCOUNT if HAVE_OBJTOOL select HAVE_OBJTOOL_NOP_MCOUNT if HAVE_OBJTOOL_MCOUNT diff --git a/arch/x86/include/asm/context_tracking_work.h b/arch/x86/includ= e/asm/context_tracking_work.h new file mode 100644 index 0000000000000..5f3b2d0977235 --- /dev/null +++ b/arch/x86/include/asm/context_tracking_work.h @@ -0,0 +1,16 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_X86_CONTEXT_TRACKING_WORK_H +#define _ASM_X86_CONTEXT_TRACKING_WORK_H + +static __always_inline void arch_context_tracking_work(enum ct_work work) +{ + switch (work) { + case CT_WORK_n: + // Do work... + break; + case CT_WORK_MAX: + WARN_ON_ONCE(true); + } +} + +#endif diff --git a/include/linux/context_tracking.h b/include/linux/context_track= ing.h index af9fe87a09225..0b0faa040e9b5 100644 --- a/include/linux/context_tracking.h +++ b/include/linux/context_tracking.h @@ -5,6 +5,7 @@ #include #include #include +#include #include =20 #include @@ -137,6 +138,26 @@ static __always_inline unsigned long ct_state_inc(int = incby) return raw_atomic_add_return(incby, this_cpu_ptr(&context_tracking.state)= ); } =20 +#ifdef CONFIG_CONTEXT_TRACKING_WORK +static __always_inline unsigned long ct_state_inc_clear_work(int incby) +{ + struct context_tracking *ct =3D this_cpu_ptr(&context_tracking); + unsigned long new, old, state; + + state =3D arch_atomic_read(&ct->state); + do { + old =3D state; + new =3D old & ~CT_WORK_MASK; + new +=3D incby; + state =3D arch_atomic_cmpxchg(&ct->state, old, new); + } while (old !=3D state); + + return new; +} +#else +#define ct_state_inc_clear_work(x) ct_state_inc(x) +#endif + static __always_inline bool warn_rcu_enter(void) { bool ret =3D false; diff --git a/include/linux/context_tracking_state.h b/include/linux/context= _tracking_state.h index 0b81248aa03e2..d2c302133672f 100644 --- a/include/linux/context_tracking_state.h +++ b/include/linux/context_tracking_state.h @@ -5,6 +5,7 @@ #include #include #include +#include =20 /* Offset to allow distinguishing irq vs. task-based idle entry/exit. */ #define CT_NESTING_IRQ_NONIDLE ((LONG_MAX / 2) + 1) @@ -39,16 +40,19 @@ struct context_tracking { }; =20 /* - * We cram two different things within the same atomic variable: + * We cram up to three different things within the same atomic variable: * - * CT_RCU_WATCHING_START CT_STATE_START - * | | - * v v - * MSB [ RCU watching counter ][ context_state ] LSB - * ^ ^ - * | | - * CT_RCU_WATCHING_END CT_STATE_END + * CT_RCU_WATCHING_START CT_STATE_STA= RT + * | CT_WORK_START | + * | | | + * v v v + * MSB [ RCU watching counter ][ context work ][ context_state ] LSB + * ^ ^ ^ + * | | | + * | CT_WORK_END | + * CT_RCU_WATCHING_END CT_STATE_END * + * The [ context work ] region spans 0 bits if CONFIG_CONTEXT_WORK=3Dn * Bits are used from the LSB upwards, so unused bits (if any) will always= be in * upper bits of the variable. */ @@ -59,18 +63,24 @@ struct context_tracking { #define CT_STATE_START 0 #define CT_STATE_END (CT_STATE_START + CT_STATE_WIDTH - 1) =20 -#define CT_RCU_WATCHING_MAX_WIDTH (CT_SIZE - CT_STATE_WIDTH) +#define CT_WORK_WIDTH (IS_ENABLED(CONFIG_CONTEXT_TRACKING_WORK) ? CT_WORK_= MAX_OFFSET : 0) +#define CT_WORK_START (CT_STATE_END + 1) +#define CT_WORK_END (CT_WORK_START + CT_WORK_WIDTH - 1) + +#define CT_RCU_WATCHING_MAX_WIDTH (CT_SIZE - CT_WORK_WIDTH - CT_STATE_WIDT= H) #define CT_RCU_WATCHING_WIDTH (IS_ENABLED(CONFIG_RCU_DYNTICKS_TORTURE)= ? 2 : CT_RCU_WATCHING_MAX_WIDTH) -#define CT_RCU_WATCHING_START (CT_STATE_END + 1) +#define CT_RCU_WATCHING_START (CT_WORK_END + 1) #define CT_RCU_WATCHING_END (CT_RCU_WATCHING_START + CT_RCU_WATCHING= _WIDTH - 1) #define CT_RCU_WATCHING BIT(CT_RCU_WATCHING_START) =20 #define CT_STATE_MASK GENMASK(CT_STATE_END, CT_STATE_START) +#define CT_WORK_MASK GENMASK(CT_WORK_END, CT_WORK_START) #define CT_RCU_WATCHING_MASK GENMASK(CT_RCU_WATCHING_END, CT_RCU_WATCHING_= START) =20 #define CT_UNUSED_WIDTH (CT_RCU_WATCHING_MAX_WIDTH - CT_RCU_WATCHING_WIDTH) =20 static_assert(CT_STATE_WIDTH + + CT_WORK_WIDTH + CT_RCU_WATCHING_WIDTH + CT_UNUSED_WIDTH =3D=3D CT_SIZE); diff --git a/include/linux/context_tracking_work.h b/include/linux/context_= tracking_work.h new file mode 100644 index 0000000000000..3742f461183ac --- /dev/null +++ b/include/linux/context_tracking_work.h @@ -0,0 +1,24 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _LINUX_CONTEXT_TRACKING_WORK_H +#define _LINUX_CONTEXT_TRACKING_WORK_H + +#include + +enum { + CT_WORK_n_OFFSET, + CT_WORK_MAX_OFFSET +}; + +enum ct_work { + CT_WORK_n =3D BIT(CT_WORK_n_OFFSET), + CT_WORK_MAX =3D BIT(CT_WORK_MAX_OFFSET) +}; + +#ifdef CONFIG_CONTEXT_TRACKING_WORK +extern bool ct_set_cpu_work(unsigned int cpu, enum ct_work work); +#else +static inline bool +ct_set_cpu_work(unsigned int cpu, unsigned int work) { return false; } +#endif + +#endif diff --git a/kernel/context_tracking.c b/kernel/context_tracking.c index fb5be6e9b423f..5a3cc4ce4d8a6 100644 --- a/kernel/context_tracking.c +++ b/kernel/context_tracking.c @@ -25,6 +25,9 @@ #include #include =20 +#ifdef CONFIG_CONTEXT_TRACKING_WORK +#include +#endif =20 DEFINE_PER_CPU(struct context_tracking, context_tracking) =3D { #ifdef CONFIG_CONTEXT_TRACKING_IDLE @@ -72,6 +75,70 @@ static __always_inline void rcu_task_trace_heavyweight_e= xit(void) #endif /* #ifdef CONFIG_TASKS_TRACE_RCU */ } =20 +#ifdef CONFIG_CONTEXT_TRACKING_WORK +static noinstr void ct_work_flush(unsigned long seq) +{ + int bit; + + seq =3D (seq & CT_WORK_MASK) >> CT_WORK_START; + + /* + * arch_context_tracking_work() must be noinstr, non-blocking, + * and NMI safe. + */ + for_each_set_bit(bit, &seq, CT_WORK_MAX) + arch_context_tracking_work(BIT(bit)); +} + +/** + * ct_set_cpu_work - set work to be run at next kernel context entry + * + * If @cpu is not currently executing in kernelspace, it will execute the + * callback mapped to @work (see arch_context_tracking_work()) at its next + * entry into ct_kernel_enter_state(). + * + * If it is already executing in kernelspace, this will be a no-op. + */ +bool ct_set_cpu_work(unsigned int cpu, enum ct_work work) +{ + struct context_tracking *ct =3D per_cpu_ptr(&context_tracking, cpu); + unsigned int old; + bool ret =3D false; + + if (!ct->active) + return false; + + preempt_disable(); + + old =3D atomic_read(&ct->state); + + /* + * The work bit must only be set if the target CPU is not executing + * in kernelspace. + * CT_RCU_WATCHING is used as a proxy for that - if the bit is set, we + * know for sure the CPU is executing in the kernel whether that be in + * NMI, IRQ or process context. + * Clear CT_RCU_WATCHING here and let the cmpxchg do the check for us; + * the state could change between the atomic_read() and the cmpxchg(). + */ + old &=3D ~CT_RCU_WATCHING; + /* + * Try setting the work until either + * - the target CPU has entered kernelspace + * - the work has been set + */ + do { + ret =3D atomic_try_cmpxchg(&ct->state, &old, old | (work << CT_WORK_STAR= T)); + } while (!ret && !(old & CT_RCU_WATCHING)); + + preempt_enable(); + return ret; +} +#else +static __always_inline void ct_work_flush(unsigned long work) { } +static __always_inline void ct_work_clear(struct context_tracking *ct) { } +#endif + /* * Record entry into an extended quiescent state. This is only to be * called when not already in an extended quiescent state, that is, @@ -88,7 +155,7 @@ static noinstr void ct_kernel_exit_state(int offset) rcu_task_trace_heavyweight_enter(); // Before CT state update! // RCU is still watching. Better not be in extended quiescent state! WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && !rcu_is_watching_curr_cp= u()); - (void)ct_state_inc(offset); + (void)ct_state_inc_clear_work(offset); // RCU is no longer watching. } =20 @@ -99,7 +166,7 @@ static noinstr void ct_kernel_exit_state(int offset) */ static noinstr void ct_kernel_enter_state(int offset) { - int seq; + unsigned long seq; =20 /* * CPUs seeing atomic_add_return() must see prior idle sojourns, @@ -107,6 +174,7 @@ static noinstr void ct_kernel_enter_state(int offset) * critical section. */ seq =3D ct_state_inc(offset); + ct_work_flush(seq); // RCU is now watching. Better not be in an extended quiescent state! rcu_task_trace_heavyweight_exit(); // After CT state update! WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && !(seq & CT_RCU_WATCHING)= ); diff --git a/kernel/time/Kconfig b/kernel/time/Kconfig index 7c6a52f7836ce..1a0c027aad141 100644 --- a/kernel/time/Kconfig +++ b/kernel/time/Kconfig @@ -181,6 +181,11 @@ config CONTEXT_TRACKING_USER_FORCE Say N otherwise, this option brings an overhead that you don't want in production. =20 +config CONTEXT_TRACKING_WORK + bool + depends on HAVE_CONTEXT_TRACKING_WORK && CONTEXT_TRACKING_USER + default y + config NO_HZ bool "Old Idle dynticks config" help --=20 2.51.0 From nobody Wed Dec 17 04:17:08 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8A51B333442 for ; Fri, 14 Nov 2025 15:16:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763133370; cv=none; b=Y9R9aB8dsOwAbsBfxulS+KoeRsnpynEagnataKRhnkTz2ROdhNDfUdRzXDc57EzM+kaRs/ldoAIZyFW7rQNhyV//pXV5y8aHSjvABKUUEJt788UOywo6c6zAMh7MkYh4w0rzQMIP+AOiRtafn00xsP4GlUUNQHNUbQD13d0k1lE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763133370; c=relaxed/simple; bh=4Dzx5BUzitnePQdhXpaEwja2nERaXt4e9Hm/codkaL0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=YP6h0cghjMarhY1B7q4r9d+bUbiboDlxeIhGnj0Fd84pwSImHTLa0Vms9QRrjv0wAMZyQ3m56tODeJI19wJdvDT9nV/9B5rHpkdS0+KY4RjHozyS1MKLHcC7ObQOcSuaMg3eyM4ngsbP9EIP1APFOHNQso019ndYNMXfEcN5/j0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Pza14FkG; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Pza14FkG" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1763133367; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WZcT1Q7WbmOBLgKadBtI1A3F7m35fxkqgToUirFNbpE=; b=Pza14FkGuNmgz4tKAB2ZX62wC+Zo7sgdds6W9YMQmlOMokSerhCE7eXLLHvvWEsFzaKyFp cO8MkE3td5a5BbPor/FXE991tC+al7gS0on8G2yGL4UterzKosR6oDgUUrgRwOYPKVZIjh TeHuGmzSoS3PJ/DWrY7zI5fMz3UWiqg= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-296-0ZR_8xnzNWiuFcmFrrPZ5Q-1; Fri, 14 Nov 2025 10:16:05 -0500 X-MC-Unique: 0ZR_8xnzNWiuFcmFrrPZ5Q-1 X-Mimecast-MFC-AGG-ID: 0ZR_8xnzNWiuFcmFrrPZ5Q_1763133359 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 4E23C1805ECE; Fri, 14 Nov 2025 15:15:59 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.45.226.10]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id E84BF18004D8; Fri, 14 Nov 2025 15:15:41 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, rcu@vger.kernel.org, x86@kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: "Peter Zijlstra (Intel)" , Nicolas Saenz Julienne , Frederic Weisbecker , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Arnaldo Carvalho de Melo , Josh Poimboeuf , Paolo Bonzini , Arnd Bergmann , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [PATCH v7 25/31] context_tracking,x86: Defer kernel text patching IPIs Date: Fri, 14 Nov 2025 16:14:22 +0100 Message-ID: <20251114151428.1064524-5-vschneid@redhat.com> In-Reply-To: <20251114150133.1056710-1-vschneid@redhat.com> References: <20251114150133.1056710-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Content-Type: text/plain; charset="utf-8" text_poke_bp_batch() sends IPIs to all online CPUs to synchronize them vs the newly patched instruction. CPUs that are executing in userspace do not need this synchronization to happen immediately, and this is actually harmful interference for NOHZ_FULL CPUs. As the synchronization IPIs are sent using a blocking call, returning from text_poke_bp_batch() implies all CPUs will observe the patched instruction(s), and this should be preserved even if the IPI is deferred. In other words, to safely defer this synchronization, any kernel instruction leading to the execution of the deferred instruction sync (ct_work_flush()) must *not* be mutable (patchable) at runtime. This means we must pay attention to mutable instructions in the early entry code: - alternatives - static keys - static calls - all sorts of probes (kprobes/ftrace/bpf/???) The early entry code leading to ct_work_flush() is noinstr, which gets rid of the probes. Alternatives are safe, because it's boot-time patching (before SMP is even brought up) which is before any IPI deferral can happen. This leaves us with static keys and static calls. Any static key used in early entry code should be only forever-enabled at boot time, IOW __ro_after_init (pretty much like alternatives). Exceptions are explicitly marked as allowed in .noinstr and will always generate an IPI when flipped. The same applies to static calls - they should be only updated at boot time, or manually marked as an exception. Objtool is now able to point at static keys/calls that don't respect this, and all static keys/calls used in early entry code have now been verified as behaving appropriately. Leverage the new context_tracking infrastructure to defer sync_core() IPIs to a target CPU's next kernel entry. Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Nicolas Saenz Julienne Signed-off-by: Valentin Schneider Acked-by: Frederic Weisbecker --- arch/x86/include/asm/context_tracking_work.h | 6 ++- arch/x86/include/asm/text-patching.h | 1 + arch/x86/kernel/alternative.c | 39 +++++++++++++++++--- arch/x86/kernel/kprobes/core.c | 4 +- arch/x86/kernel/kprobes/opt.c | 4 +- arch/x86/kernel/module.c | 2 +- include/asm-generic/sections.h | 15 ++++++++ include/linux/context_tracking_work.h | 4 +- 8 files changed, 60 insertions(+), 15 deletions(-) diff --git a/arch/x86/include/asm/context_tracking_work.h b/arch/x86/includ= e/asm/context_tracking_work.h index 5f3b2d0977235..485b32881fde5 100644 --- a/arch/x86/include/asm/context_tracking_work.h +++ b/arch/x86/include/asm/context_tracking_work.h @@ -2,11 +2,13 @@ #ifndef _ASM_X86_CONTEXT_TRACKING_WORK_H #define _ASM_X86_CONTEXT_TRACKING_WORK_H =20 +#include + static __always_inline void arch_context_tracking_work(enum ct_work work) { switch (work) { - case CT_WORK_n: - // Do work... + case CT_WORK_SYNC: + sync_core(); break; case CT_WORK_MAX: WARN_ON_ONCE(true); diff --git a/arch/x86/include/asm/text-patching.h b/arch/x86/include/asm/te= xt-patching.h index f2d142a0a862e..ca989b0b6c0ae 100644 --- a/arch/x86/include/asm/text-patching.h +++ b/arch/x86/include/asm/text-patching.h @@ -33,6 +33,7 @@ extern void text_poke_apply_relocation(u8 *buf, const u8 = * const instr, size_t i */ extern void *text_poke(void *addr, const void *opcode, size_t len); extern void smp_text_poke_sync_each_cpu(void); +extern void smp_text_poke_sync_each_cpu_deferrable(void); extern void *text_poke_kgdb(void *addr, const void *opcode, size_t len); extern void *text_poke_copy(void *addr, const void *opcode, size_t len); #define text_poke_copy text_poke_copy diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index 8ee5ff547357a..ce8989e02ae77 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -6,6 +6,7 @@ #include #include #include +#include =20 #include #include @@ -2708,9 +2709,24 @@ static void do_sync_core(void *info) sync_core(); } =20 +static bool do_sync_core_defer_cond(int cpu, void *info) +{ + return !ct_set_cpu_work(cpu, CT_WORK_SYNC); +} + +static void __smp_text_poke_sync_each_cpu(smp_cond_func_t cond_func) +{ + on_each_cpu_cond(cond_func, do_sync_core, NULL, 1); +} + void smp_text_poke_sync_each_cpu(void) { - on_each_cpu(do_sync_core, NULL, 1); + __smp_text_poke_sync_each_cpu(NULL); +} + +void smp_text_poke_sync_each_cpu_deferrable(void) +{ + __smp_text_poke_sync_each_cpu(do_sync_core_defer_cond); } =20 /* @@ -2880,6 +2896,7 @@ noinstr int smp_text_poke_int3_handler(struct pt_regs= *regs) */ void smp_text_poke_batch_finish(void) { + smp_cond_func_t cond =3D do_sync_core_defer_cond; unsigned char int3 =3D INT3_INSN_OPCODE; unsigned int i; int do_sync; @@ -2916,11 +2933,21 @@ void smp_text_poke_batch_finish(void) * First step: add a INT3 trap to the address that will be patched. */ for (i =3D 0; i < text_poke_array.nr_entries; i++) { - text_poke_array.vec[i].old =3D *(u8 *)text_poke_addr(&text_poke_array.ve= c[i]); - text_poke(text_poke_addr(&text_poke_array.vec[i]), &int3, INT3_INSN_SIZE= ); + void *addr =3D text_poke_addr(&text_poke_array.vec[i]); + + /* + * There's no safe way to defer IPIs for patching text in + * .noinstr, record whether there is at least one such poke. + */ + if (is_kernel_noinstr_text((unsigned long)addr) || + is_module_noinstr_text_address((unsigned long)addr)) + cond =3D NULL; + + text_poke_array.vec[i].old =3D *((u8 *)addr); + text_poke(addr, &int3, INT3_INSN_SIZE); } =20 - smp_text_poke_sync_each_cpu(); + __smp_text_poke_sync_each_cpu(cond); =20 /* * Second step: update all but the first byte of the patched range. @@ -2982,7 +3009,7 @@ void smp_text_poke_batch_finish(void) * not necessary and we'd be safe even without it. But * better safe than sorry (plus there's not only Intel). */ - smp_text_poke_sync_each_cpu(); + __smp_text_poke_sync_each_cpu(cond); } =20 /* @@ -3003,7 +3030,7 @@ void smp_text_poke_batch_finish(void) } =20 if (do_sync) - smp_text_poke_sync_each_cpu(); + __smp_text_poke_sync_each_cpu(cond); =20 /* * Remove and wait for refs to be zero. diff --git a/arch/x86/kernel/kprobes/core.c b/arch/x86/kernel/kprobes/core.c index 3863d7709386f..51957cd737f52 100644 --- a/arch/x86/kernel/kprobes/core.c +++ b/arch/x86/kernel/kprobes/core.c @@ -790,7 +790,7 @@ void arch_arm_kprobe(struct kprobe *p) u8 int3 =3D INT3_INSN_OPCODE; =20 text_poke(p->addr, &int3, 1); - smp_text_poke_sync_each_cpu(); + smp_text_poke_sync_each_cpu_deferrable(); perf_event_text_poke(p->addr, &p->opcode, 1, &int3, 1); } =20 @@ -800,7 +800,7 @@ void arch_disarm_kprobe(struct kprobe *p) =20 perf_event_text_poke(p->addr, &int3, 1, &p->opcode, 1); text_poke(p->addr, &p->opcode, 1); - smp_text_poke_sync_each_cpu(); + smp_text_poke_sync_each_cpu_deferrable(); } =20 void arch_remove_kprobe(struct kprobe *p) diff --git a/arch/x86/kernel/kprobes/opt.c b/arch/x86/kernel/kprobes/opt.c index 0aabd4c4e2c4f..eada8dca1c2e8 100644 --- a/arch/x86/kernel/kprobes/opt.c +++ b/arch/x86/kernel/kprobes/opt.c @@ -513,11 +513,11 @@ void arch_unoptimize_kprobe(struct optimized_kprobe *= op) JMP32_INSN_SIZE - INT3_INSN_SIZE); =20 text_poke(addr, new, INT3_INSN_SIZE); - smp_text_poke_sync_each_cpu(); + smp_text_poke_sync_each_cpu_deferrable(); text_poke(addr + INT3_INSN_SIZE, new + INT3_INSN_SIZE, JMP32_INSN_SIZE - INT3_INSN_SIZE); - smp_text_poke_sync_each_cpu(); + smp_text_poke_sync_each_cpu_deferrable(); =20 perf_event_text_poke(op->kp.addr, old, JMP32_INSN_SIZE, new, JMP32_INSN_S= IZE); } diff --git a/arch/x86/kernel/module.c b/arch/x86/kernel/module.c index 0ffbae902e2fe..c6c4f391eb465 100644 --- a/arch/x86/kernel/module.c +++ b/arch/x86/kernel/module.c @@ -206,7 +206,7 @@ static int write_relocate_add(Elf64_Shdr *sechdrs, write, apply); =20 if (!early) { - smp_text_poke_sync_each_cpu(); + smp_text_poke_sync_each_cpu_deferrable(); mutex_unlock(&text_mutex); } =20 diff --git a/include/asm-generic/sections.h b/include/asm-generic/sections.h index 0755bc39b0d80..7d2403014010e 100644 --- a/include/asm-generic/sections.h +++ b/include/asm-generic/sections.h @@ -199,6 +199,21 @@ static inline bool is_kernel_inittext(unsigned long ad= dr) addr < (unsigned long)_einittext; } =20 + +/** + * is_kernel_noinstr_text - checks if the pointer address is located in the + * .noinstr section + * + * @addr: address to check + * + * Returns: true if the address is located in .noinstr, false otherwise. + */ +static inline bool is_kernel_noinstr_text(unsigned long addr) +{ + return addr >=3D (unsigned long)__noinstr_text_start && + addr < (unsigned long)__noinstr_text_end; +} + /** * __is_kernel_text - checks if the pointer address is located in the * .text section diff --git a/include/linux/context_tracking_work.h b/include/linux/context_= tracking_work.h index 3742f461183ac..abd3c196855f4 100644 --- a/include/linux/context_tracking_work.h +++ b/include/linux/context_tracking_work.h @@ -5,12 +5,12 @@ #include =20 enum { - CT_WORK_n_OFFSET, + CT_WORK_SYNC_OFFSET, CT_WORK_MAX_OFFSET }; =20 enum ct_work { - CT_WORK_n =3D BIT(CT_WORK_n_OFFSET), + CT_WORK_SYNC =3D BIT(CT_WORK_SYNC_OFFSET), CT_WORK_MAX =3D BIT(CT_WORK_MAX_OFFSET) }; =20 --=20 2.51.0 From nobody Wed Dec 17 04:17:08 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C18E632BF35 for ; Fri, 14 Nov 2025 15:16:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763133391; cv=none; b=XukMq8W2y4vQVoY3uu9rFAsrTC07kdsq3lYc3Fe5aNcRXYA+vRrlCyRTK8cg+n0/GtskI0yyIGEmGLBzI990I9rH3EKcfLITbegZOI9WvS2NiGh/3+oNnoEqtTrGE8/FANu/90zutRI+Uw8cVPtJbtRoaoYDeD5M7MOlPhPZDtg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763133391; c=relaxed/simple; bh=bRhz7pzmAvyxE+4fka8gvMXEnIf9tOF0eDiG/K4GcEQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=uvDrhFPGy5WhY1l/nBA+MxDkcjJYMnxW3tinNjy2x58kb5fNb+vuxMYMEgcPiBOQEthQ36bJa1ykPgFtzzcFSwRpVdzNQZtEdRcymXpsfMOhD5eaYQv4aZySLNJqWTQAl8fcQai0L+vHLuM1JgnBFWpcw568T9TB2+nr2yVM9NU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Lf0Nu9Pp; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Lf0Nu9Pp" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1763133388; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3wtNiwO7mQgs61ieJ83HPyGKadANIMB4TUgH9LUJTDI=; b=Lf0Nu9Ppgr1G7QosoyNgchuK6LbIAAmIDLapMhWG2VjoQYSrl2q4tMT0SeuI+biek5sH9S OqwQ35li0cYWh7c3F5hVcBN4fXUYwnP3USLdSnBVCoQjhCzWcKs1Dxax0y9SrpvSMngD9x uGIxLnT5EiZmx6ffir5CvPaznDqrlDI= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-530-KUYqLZu4MQq0ZGtzVQ5_Tw-1; Fri, 14 Nov 2025 10:16:26 -0500 X-MC-Unique: KUYqLZu4MQq0ZGtzVQ5_Tw-1 X-Mimecast-MFC-AGG-ID: KUYqLZu4MQq0ZGtzVQ5_Tw_1763133376 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 286521956094; Fri, 14 Nov 2025 15:16:16 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.45.226.10]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id EF3DD1800451; Fri, 14 Nov 2025 15:15:59 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, rcu@vger.kernel.org, x86@kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Frederic Weisbecker , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Josh Poimboeuf , Paolo Bonzini , Arnd Bergmann , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [PATCH v7 26/31] x86/jump_label: Add ASM support for static_branch_likely() Date: Fri, 14 Nov 2025 16:14:23 +0100 Message-ID: <20251114151428.1064524-6-vschneid@redhat.com> In-Reply-To: <20251114150133.1056710-1-vschneid@redhat.com> References: <20251114150133.1056710-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Content-Type: text/plain; charset="utf-8" A later commit will add some early entry code that only needs to be executed if nohz_full is present on the cmdline, not just if CONFIG_NO_HZ_FULL is compiled in. Add an ASM-callable static branch macro. Note that I haven't found a way to express unlikely (i.e. out-of-line) static branches in ASM macros without using extra jumps, which kind of defeats the purpose. Consider: .macro FOOBAR // Key enabled: JMP .Ldostuff_\@ // Key disabled: NOP STATIC_BRANCH_UNLIKELY key, .Ldostuff_\@ // Patched to JMP if enabled jmp .Lend_\@ .Ldostuff_\@: .Lend_\@: .endm Instead, this should be expressed as a likely (i.e. in-line) static key: .macro FOOBAR // Key enabled: NOP // Key disabled: JMP .Lend_\@ STATIC_BRANCH_LIKELY key, .Lend\@ // Patched to NOP if enabled .Lend_\@: .endm Suggested-by: Frederic Weisbecker Signed-off-by: Valentin Schneider --- arch/x86/include/asm/jump_label.h | 33 ++++++++++++++++++++++++++++++- 1 file changed, 32 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/jump_label.h b/arch/x86/include/asm/jump_= label.h index 61dd1dee7812e..3c9ba3948e225 100644 --- a/arch/x86/include/asm/jump_label.h +++ b/arch/x86/include/asm/jump_label.h @@ -7,7 +7,38 @@ #include #include =20 -#ifndef __ASSEMBLER__ +#ifdef __ASSEMBLER__ + +/* + * There isn't a neat way to craft unlikely static branches in ASM, so they + * all have to be expressed as likely (inline) static branches. This macro + * thus assumes a "likely" usage. + */ +.macro ARCH_STATIC_BRANCH_LIKELY_ASM key, label, jump, hack +1: +.if \jump || \hack + jmp \label +.else + .byte BYTES_NOP5 +.endif + .pushsection __jump_table, "aw" + _ASM_ALIGN + .long 1b - . + .long \label - . + /* LIKELY so bit0=3D1, bit1=3Dhack */ + _ASM_PTR \key + 1 + (\hack << 1) - . + .popsection +.endm + +.macro STATIC_BRANCH_TRUE_LIKELY key, label + ARCH_STATIC_BRANCH_LIKELY_ASM \key, \label, 0, IS_ENABLED(CONFIG_HAVE_JUM= P_LABEL_HACK) +.endm + +.macro STATIC_BRANCH_FALSE_LIKELY key, label + ARCH_STATIC_BRANCH_LIKELY_ASM \key, \label, 1, 0 +.endm + +#else /* !__ASSEMBLER__ */ =20 #include #include --=20 2.51.0 From nobody Wed Dec 17 04:17:08 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BBBF2334C12 for ; Fri, 14 Nov 2025 15:16:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763133405; cv=none; b=edoE+zAUza/bt8GdBJryy4LQN4lMEhD9Q6j+OSW4S61MOdzklcacXuqGHoOtgguaBp3MjCQ1wL8FRyFW/Qec6mUfXhqXRji5gdn/48eUXN2mSvub8oldqd7UhAdvUHauYxCCMWZYyc4jYilSbIW4alb1R4vIlUtWLop1KN2bJaY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763133405; c=relaxed/simple; bh=z9azdQ3tAjj8W6CUyGNM0oLwsd5ab0jCguNokgHO0/4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Ensw5br9PSMxJhVudGejl1VbdYUe9cvE2bzkpV69Vaph0llMvnn+YYdX6IY/HoW/diYK04vlS4WRWL4nN55YoPaNGXyX8W5T9xvdFiCMSlXOsH3W+OQEEBrKtyLYQAslE2v1XX3ZdEyrXNf4d31JLnhMfpbpxcvn/tPsbU2lcMs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=YhnSJbqQ; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="YhnSJbqQ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1763133402; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uUsWRA0PhCMnZLZEkKPa9puqLJ49gF+6GXOwWTaFVY4=; b=YhnSJbqQgNEyrB+UuTyAr1mSn9dHR6AZvBy0DzCi6wdSOaklhm7Je5xbQqoIkG6NQ2Xms0 FrRXS+gHkRMShiCEdnAIL54GalK033cBY0wNhttKLQnK8H+ovsI7I5PVYwTW+pe3YxS9KB hLNAWPWWcrolGxh7dBcrDlm1HOcv3BY= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-562-ckqYkCrGPcysWf_5rr0BIw-1; Fri, 14 Nov 2025 10:16:38 -0500 X-MC-Unique: ckqYkCrGPcysWf_5rr0BIw-1 X-Mimecast-MFC-AGG-ID: ckqYkCrGPcysWf_5rr0BIw_1763133392 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 3CBED180049F; Fri, 14 Nov 2025 15:16:32 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.45.226.10]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id BD14A18004D8; Fri, 14 Nov 2025 15:16:16 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, rcu@vger.kernel.org, x86@kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Josh Poimboeuf , Paolo Bonzini , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [PATCH v7 27/31] x86/mm: Make INVPCID type macros available to assembly Date: Fri, 14 Nov 2025 16:14:24 +0100 Message-ID: <20251114151428.1064524-7-vschneid@redhat.com> In-Reply-To: <20251114150133.1056710-1-vschneid@redhat.com> References: <20251114150133.1056710-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Content-Type: text/plain; charset="utf-8" A later commit will introduce a pure-assembly INVPCID invocation, allow assembly files to get the type definitions. Signed-off-by: Valentin Schneider --- arch/x86/include/asm/invpcid.h | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/arch/x86/include/asm/invpcid.h b/arch/x86/include/asm/invpcid.h index 734482afbf81d..27ae75c2d7fed 100644 --- a/arch/x86/include/asm/invpcid.h +++ b/arch/x86/include/asm/invpcid.h @@ -2,6 +2,13 @@ #ifndef _ASM_X86_INVPCID #define _ASM_X86_INVPCID =20 +#define INVPCID_TYPE_INDIV_ADDR 0 +#define INVPCID_TYPE_SINGLE_CTXT 1 +#define INVPCID_TYPE_ALL_INCL_GLOBAL 2 +#define INVPCID_TYPE_ALL_NON_GLOBAL 3 + +#ifndef __ASSEMBLER__ + static inline void __invpcid(unsigned long pcid, unsigned long addr, unsigned long type) { @@ -17,11 +24,6 @@ static inline void __invpcid(unsigned long pcid, unsigne= d long addr, :: [desc] "m" (desc), [type] "r" (type) : "memory"); } =20 -#define INVPCID_TYPE_INDIV_ADDR 0 -#define INVPCID_TYPE_SINGLE_CTXT 1 -#define INVPCID_TYPE_ALL_INCL_GLOBAL 2 -#define INVPCID_TYPE_ALL_NON_GLOBAL 3 - /* Flush all mappings for a given pcid and addr, not including globals. */ static inline void invpcid_flush_one(unsigned long pcid, unsigned long addr) @@ -47,4 +49,6 @@ static inline void invpcid_flush_all_nonglobals(void) __invpcid(0, 0, INVPCID_TYPE_ALL_NON_GLOBAL); } =20 +#endif /* __ASSEMBLER__ */ + #endif /* _ASM_X86_INVPCID */ --=20 2.51.0 From nobody Wed Dec 17 04:17:08 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A8F3A334C12 for ; Fri, 14 Nov 2025 15:16:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763133419; cv=none; b=lvr7vCdaREBrCE9tzGvsKkQ4cuUF5kAOW157vn7gAAw900Qx0B0rxKMFzD9TG4+37dHFjtzDM622F0QPm8pMxCiPkzkD4L8FaouQFESt1RgVk1vNiMZJSzHUVV58X9fTS9IKbUoreXIIfgtjO6WCJAH7wNYmW0O5i/bsBCgYICs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763133419; c=relaxed/simple; bh=FJOWpjikV7wP2WH8iGraq1RECH7fRFuBdooFdgvyDCQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tcVd49xW2IZyjKS3gjQ3JC+8xUO2ken2auMzY7qPo7Sn+OUtlfoqb+iaSvxgThWVfVp0DKqdvP2f8QmqRfc6Z7bHV2VTuXk0TkELG1yet5VnpqHPKG9vs/J+ERjdZZYv/6Ho8rlvD5iadJFgbR8HQYb7NCISbkVeRAqn7P0zBXQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=a0cvbhZk; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="a0cvbhZk" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1763133415; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KrHUzPGBZEMSREH/RvanIYE5RDDkqwnke9Df1bzomfM=; b=a0cvbhZkt1PuF7LXQGTNDbW8VeO4PGvetZhVBJVu/f1yl6lnU6+s4uWQACsdtLgLTwEvPE m7E8NVblrl2KFKrZ2WX+bw7lDQxfsN20zGyEzGdie6mskY6o+PsABbH0VZmDg/uZmGVdI7 sRFCtIX6DCaZcMiumb4GRp4Mv8UCPuw= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-683-nkLlETgCP3ukpkMvj_2ARA-1; Fri, 14 Nov 2025 10:16:52 -0500 X-MC-Unique: nkLlETgCP3ukpkMvj_2ARA-1 X-Mimecast-MFC-AGG-ID: nkLlETgCP3ukpkMvj_2ARA_1763133407 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 66EDF18D95D1; Fri, 14 Nov 2025 15:16:47 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.45.226.10]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id BF03E180049F; Fri, 14 Nov 2025 15:16:32 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, rcu@vger.kernel.org, x86@kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Josh Poimboeuf , Paolo Bonzini , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [RFC PATCH v7 28/31] x86/mm/pti: Introduce a kernel/user CR3 software signal Date: Fri, 14 Nov 2025 16:14:25 +0100 Message-ID: <20251114151428.1064524-8-vschneid@redhat.com> In-Reply-To: <20251114150133.1056710-1-vschneid@redhat.com> References: <20251114150133.1056710-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Content-Type: text/plain; charset="utf-8" Later commits will rely on this information to defer kernel TLB flush IPIs. Update it when switching to and from the kernel CR3. This will only be really useful for NOHZ_FULL CPUs, but it should be cheaper to unconditionally update a never-used per-CPU variable living in its own cacheline than to check a shared cpumask such as housekeeping_cpumask(HK_TYPE_KERNEL_NOISE) at every entry. Note that the COALESCE_TLBI config option is introduced in a later commit, when the whole feature is implemented. Signed-off-by: Valentin Schneider --- Per the cover letter, I really hate this, but couldn't come up with anything better. --- arch/x86/entry/calling.h | 21 +++++++++++++++++++++ arch/x86/entry/syscall_64.c | 4 ++++ arch/x86/include/asm/tlbflush.h | 3 +++ 3 files changed, 28 insertions(+) diff --git a/arch/x86/entry/calling.h b/arch/x86/entry/calling.h index 77e2d920a6407..0187c0ea2fddb 100644 --- a/arch/x86/entry/calling.h +++ b/arch/x86/entry/calling.h @@ -9,6 +9,7 @@ #include #include #include +#include /* @@ -170,11 +171,28 @@ For 32-bit we have the following conventions - kernel= is built with andq $(~PTI_USER_PGTABLE_AND_PCID_MASK), \reg .endm +.macro COALESCE_TLBI +#ifdef CONFIG_COALESCE_TLBI + STATIC_BRANCH_FALSE_LIKELY housekeeping_overridden, .Lend_\@ + movl $1, PER_CPU_VAR(kernel_cr3_loaded) +.Lend_\@: +#endif // CONFIG_COALESCE_TLBI +.endm + +.macro NOTE_SWITCH_TO_USER_CR3 +#ifdef CONFIG_COALESCE_TLBI + STATIC_BRANCH_FALSE_LIKELY housekeeping_overridden, .Lend_\@ + movl $0, PER_CPU_VAR(kernel_cr3_loaded) +.Lend_\@: +#endif // CONFIG_COALESCE_TLBI +.endm + .macro SWITCH_TO_KERNEL_CR3 scratch_reg:req ALTERNATIVE "jmp .Lend_\@", "", X86_FEATURE_PTI mov %cr3, \scratch_reg ADJUST_KERNEL_CR3 \scratch_reg mov \scratch_reg, %cr3 + COALESCE_TLBI .Lend_\@: .endm @@ -182,6 +200,7 @@ For 32-bit we have the following conventions - kernel i= s built with PER_CPU_VAR(cpu_tlbstate + TLB_STATE_user_pcid_flush_mask) .macro SWITCH_TO_USER_CR3 scratch_reg:req scratch_reg2:req + NOTE_SWITCH_TO_USER_CR3 mov %cr3, \scratch_reg ALTERNATIVE "jmp .Lwrcr3_\@", "", X86_FEATURE_PCID @@ -241,6 +260,7 @@ For 32-bit we have the following conventions - kernel i= s built with ADJUST_KERNEL_CR3 \scratch_reg movq \scratch_reg, %cr3 + COALESCE_TLBI .Ldone_\@: .endm @@ -257,6 +277,7 @@ For 32-bit we have the following conventions - kernel i= s built with bt $PTI_USER_PGTABLE_BIT, \save_reg jnc .Lend_\@ + NOTE_SWITCH_TO_USER_CR3 ALTERNATIVE "jmp .Lwrcr3_\@", "", X86_FEATURE_PCID /* diff --git a/arch/x86/entry/syscall_64.c b/arch/x86/entry/syscall_64.c index b6e68ea98b839..2589d232e0ba1 100644 --- a/arch/x86/entry/syscall_64.c +++ b/arch/x86/entry/syscall_64.c @@ -83,6 +83,10 @@ static __always_inline bool do_syscall_x32(struct pt_reg= s *regs, int nr) return false; } +#ifdef CONFIG_COALESCE_TLBI +DEFINE_PER_CPU(bool, kernel_cr3_loaded) =3D true; +#endif + /* Returns true to return using SYSRET, or false to use IRET */ __visible noinstr bool do_syscall_64(struct pt_regs *regs, int nr) { diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflus= h.h index 00daedfefc1b0..e39ae95b85072 100644 --- a/arch/x86/include/asm/tlbflush.h +++ b/arch/x86/include/asm/tlbflush.h @@ -17,6 +17,9 @@ #include DECLARE_PER_CPU(u64, tlbstate_untag_mask); +#ifdef CONFIG_COALESCE_TLBI +DECLARE_PER_CPU(bool, kernel_cr3_loaded); +#endif void __flush_tlb_all(void); -- 2.51.0 From nobody Wed Dec 17 04:17:08 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3BAC532E159 for ; Fri, 14 Nov 2025 15:17:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763133435; cv=none; b=eQqmF6mn78GX1LI2OBcdY9LivXdwDX0PUW2jS41woA3kqMSEOcIUmQdWgseVzivkJzgdaHfaBG5Am3tK0nxD8acjXM0207R3RXnwjUgmdwe1wo+GwI99eljLmT91/iRvgy0vzuJcgXd3LdwTf9BBqAV56Zw5GNi8Nv212Gxwp/M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763133435; c=relaxed/simple; bh=dq+9vdpT4F4YDpOuYqYMrNvq6Z0Eu8BbCmKisrHSM5A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Lvzd/nbugd75A0KL0w97gEaS95Buvxqcl65gW0CSb/y/JScUmLvU8AmS1I3qcou2OfaoNY/0FbWExxP2Ck3eYHf0HxZQ5+0TS0W4M5zK/NHXfLthnu4Ls9sK1xamJVfyHtWn0CClGAEPS6FwOUP8PZkQZXT+1I2Qvxj4QITfGqo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=az/THQw/; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="az/THQw/" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1763133433; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3X87uiojng2a6RCMXxkyxjZHQ7qUm6Xyc1h3s0fKM3s=; b=az/THQw/Nef9DVwrFDjDZtykiTD5qR2NqW2UM235/x7W4P8dCyJCBUIj1SOLmrXoMRy6aM h5ZhOXXOwqZYZhSJYC31a7UwRPhC/UttDyMpYgXbw2ag/0K1w/9YO33TOZ7lo3HY8dEsr1 L87X0VCMOHYY2vhJU0BZaUqaqEIyFis= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-128-2-uC21UoNNia6LI_UqojzA-1; Fri, 14 Nov 2025 10:17:08 -0500 X-MC-Unique: 2-uC21UoNNia6LI_UqojzA-1 X-Mimecast-MFC-AGG-ID: 2-uC21UoNNia6LI_UqojzA_1763133423 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id C8653180009D; Fri, 14 Nov 2025 15:17:02 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.45.226.10]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 7E4EA180049F; Fri, 14 Nov 2025 15:16:48 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, rcu@vger.kernel.org, x86@kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Josh Poimboeuf , Paolo Bonzini , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [RFC PATCH v7 29/31] x86/mm/pti: Implement a TLB flush immediately after a switch to kernel CR3 Date: Fri, 14 Nov 2025 16:14:26 +0100 Message-ID: <20251114151428.1064524-9-vschneid@redhat.com> In-Reply-To: <20251114150133.1056710-1-vschneid@redhat.com> References: <20251114150133.1056710-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Content-Type: text/plain; charset="utf-8" Deferring kernel range TLB flushes requires the guarantee that upon entering the kernel, no stale entry may be accessed. The simplest way to provide such a guarantee is to issue an unconditional flush upon switching to the kernel CR3, as this is the pivoting point where such stale entries may be accessed. As this is only relevant to NOHZ_FULL, restrict the mechanism to NOHZ_FULL CPUs. Note that the COALESCE_TLBI config option is introduced in a later commit, when the whole feature is implemented. Signed-off-by: Valentin Schneider --- arch/x86/entry/calling.h | 25 ++++++++++++++++++++++--- arch/x86/kernel/asm-offsets.c | 1 + 2 files changed, 23 insertions(+), 3 deletions(-) diff --git a/arch/x86/entry/calling.h b/arch/x86/entry/calling.h index 0187c0ea2fddb..620203ef04e9f 100644 --- a/arch/x86/entry/calling.h +++ b/arch/x86/entry/calling.h @@ -10,6 +10,7 @@ #include #include #include +#include /* @@ -171,9 +172,27 @@ For 32-bit we have the following conventions - kernel = is built with andq $(~PTI_USER_PGTABLE_AND_PCID_MASK), \reg .endm -.macro COALESCE_TLBI +.macro COALESCE_TLBI scratch_reg:req #ifdef CONFIG_COALESCE_TLBI STATIC_BRANCH_FALSE_LIKELY housekeeping_overridden, .Lend_\@ + /* No point in doing this for housekeeping CPUs */ + movslq PER_CPU_VAR(cpu_number), \scratch_reg + bt \scratch_reg, tick_nohz_full_mask(%rip) + jnc .Lend_tlbi_\@ + + ALTERNATIVE "jmp .Lcr4_\@", "", X86_FEATURE_INVPCID + movq $(INVPCID_TYPE_ALL_INCL_GLOBAL), \scratch_reg + /* descriptor is all zeroes, point at the zero page */ + invpcid empty_zero_page(%rip), \scratch_reg + jmp .Lend_tlbi_\@ +.Lcr4_\@: + /* Note: this gives CR4 pinning the finger */ + movq PER_CPU_VAR(cpu_tlbstate + TLB_STATE_cr4), \scratch_reg + xorq $(X86_CR4_PGE), \scratch_reg + movq \scratch_reg, %cr4 + xorq $(X86_CR4_PGE), \scratch_reg + movq \scratch_reg, %cr4 +.Lend_tlbi_\@: movl $1, PER_CPU_VAR(kernel_cr3_loaded) .Lend_\@: #endif // CONFIG_COALESCE_TLBI @@ -192,7 +211,7 @@ For 32-bit we have the following conventions - kernel i= s built with mov %cr3, \scratch_reg ADJUST_KERNEL_CR3 \scratch_reg mov \scratch_reg, %cr3 - COALESCE_TLBI + COALESCE_TLBI \scratch_reg .Lend_\@: .endm @@ -260,7 +279,7 @@ For 32-bit we have the following conventions - kernel i= s built with ADJUST_KERNEL_CR3 \scratch_reg movq \scratch_reg, %cr3 - COALESCE_TLBI + COALESCE_TLBI \scratch_reg .Ldone_\@: .endm diff --git a/arch/x86/kernel/asm-offsets.c b/arch/x86/kernel/asm-offsets.c index 32ba599a51f88..deb92e9c8923d 100644 --- a/arch/x86/kernel/asm-offsets.c +++ b/arch/x86/kernel/asm-offsets.c @@ -106,6 +106,7 @@ static void __used common(void) /* TLB state for the entry code */ OFFSET(TLB_STATE_user_pcid_flush_mask, tlb_state, user_pcid_flush_mask); + OFFSET(TLB_STATE_cr4, tlb_state, cr4); /* Layout info for cpu_entry_area */ OFFSET(CPU_ENTRY_AREA_entry_stack, cpu_entry_area, entry_stack_page); -- 2.51.0 From nobody Wed Dec 17 04:17:08 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 47870326925 for ; Fri, 14 Nov 2025 15:17:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763133450; cv=none; b=LnSSbuH7F1xGfv6TZxAny5qhm5PnXkeG5JmLhoxi0C/p/hV5sc4uvn9nzNzam5KKRVOc3Kg3cBMHEXuC6zL4Xtn+2huyIvZs8Gq6vBzd5Qng3K6XMRKVXZXMvfwUrMN1yNnCAmwxMQmyz76ZVMpV4PpNPub9HzqKNA5XDnz4+OE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763133450; c=relaxed/simple; bh=vI3ACYOpvI5FKtEelcYoHvH4aME+Y/Cjq2GuVBwmwDg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=jYJonErkBKUbEz8lgYMcpE1dgZT4eBgiB228QqUNLb5XAqkk4VU5qcoOEPCkdS/R32M6wz2WtPlIWTE0qIibAAMxcLx8W6olplhjp7LhnnNMHyB3nj1j+dMeaXwAcV9dR9Sfx2mhl6fhxdcVgqtkMMj3ZEoSu5WOQtX2/aSNNrQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=ao8CTe1Q; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ao8CTe1Q" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1763133448; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8nnNDo7oy33aNFJ8GpidZCiS52jFHxLb2tZYlzIaaH0=; b=ao8CTe1QTKOHyYjtkiqG+TjPw7Dznzge0m4XDwQJdbN1u/pTLiS2FqsxQrt5fnJMtgPo79 bWvbMZCGDIAvzlVP4rmxGE9pF+Jb49QIzfsB9WG/1EZje6kCcs4GL0q1eRXBwV0DqMwtt3 n/YBDNlziN8KqXbiZYBjemZT/muAwdk= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-421-JVVClndjP8CFZcyMJNIxlg-1; Fri, 14 Nov 2025 10:17:23 -0500 X-MC-Unique: JVVClndjP8CFZcyMJNIxlg-1 X-Mimecast-MFC-AGG-ID: JVVClndjP8CFZcyMJNIxlg_1763133438 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 2113F195608E; Fri, 14 Nov 2025 15:17:18 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.45.226.10]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 6AFB41800451; Fri, 14 Nov 2025 15:17:03 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, rcu@vger.kernel.org, x86@kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Josh Poimboeuf , Paolo Bonzini , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [RFC PATCH v7 30/31] x86/mm, mm/vmalloc: Defer kernel TLB flush IPIs under CONFIG_COALESCE_TLBI=y Date: Fri, 14 Nov 2025 16:14:27 +0100 Message-ID: <20251114151428.1064524-10-vschneid@redhat.com> In-Reply-To: <20251114150133.1056710-1-vschneid@redhat.com> References: <20251114150133.1056710-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Content-Type: text/plain; charset="utf-8" Previous commits have added an unconditional TLB flush right after switching to the kernel CR3 on NOHZ_FULL CPUs, and a software signal to determine whether a CPU has its kernel CR3 loaded. Using these two components, we can now safely defer kernel TLB flush IPIs targeting NOHZ_FULL CPUs executing in userspace (i.e. with the user CR3 loaded). Note that the COALESCE_TLBI config option is introduced in a later commit, when the whole feature is implemented. Signed-off-by: Valentin Schneider --- arch/x86/include/asm/tlbflush.h | 3 +++ arch/x86/mm/tlb.c | 34 ++++++++++++++++++++++++++------- mm/vmalloc.c | 34 ++++++++++++++++++++++++++++----- 3 files changed, 59 insertions(+), 12 deletions(-) diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflus= h.h index e39ae95b85072..6d533afd70952 100644 --- a/arch/x86/include/asm/tlbflush.h +++ b/arch/x86/include/asm/tlbflush.h @@ -321,6 +321,9 @@ extern void flush_tlb_mm_range(struct mm_struct *mm, un= signed long start, unsigned long end, unsigned int stride_shift, bool freed_tables); extern void flush_tlb_kernel_range(unsigned long start, unsigned long end); +#ifdef CONFIG_COALESCE_TLBI +extern void flush_tlb_kernel_range_deferrable(unsigned long start, unsigne= d long end); +#endif static inline void flush_tlb_page(struct vm_area_struct *vma, unsigned lon= g a) { diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index 5d221709353e0..1ce80f8775e7a 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -12,6 +12,7 @@ #include #include #include +#include #include #include @@ -1529,23 +1530,24 @@ static void do_kernel_range_flush(void *info) flush_tlb_one_kernel(addr); } -static void kernel_tlb_flush_all(struct flush_tlb_info *info) +static void kernel_tlb_flush_all(smp_cond_func_t cond, struct flush_tlb_in= fo *info) { if (cpu_feature_enabled(X86_FEATURE_INVLPGB)) invlpgb_flush_all(); else - on_each_cpu(do_flush_tlb_all, NULL, 1); + on_each_cpu_cond(cond, do_flush_tlb_all, NULL, 1); } -static void kernel_tlb_flush_range(struct flush_tlb_info *info) +static void kernel_tlb_flush_range(smp_cond_func_t cond, struct flush_tlb_= info *info) { if (cpu_feature_enabled(X86_FEATURE_INVLPGB)) invlpgb_kernel_range_flush(info); else - on_each_cpu(do_kernel_range_flush, info, 1); + on_each_cpu_cond(cond, do_kernel_range_flush, info, 1); } -void flush_tlb_kernel_range(unsigned long start, unsigned long end) +static inline void +__flush_tlb_kernel_range(smp_cond_func_t cond, unsigned long start, unsign= ed long end) { struct flush_tlb_info *info; @@ -1555,13 +1557,31 @@ void flush_tlb_kernel_range(unsigned long start, un= signed long end) TLB_GENERATION_INVALID); if (info->end =3D=3D TLB_FLUSH_ALL) - kernel_tlb_flush_all(info); + kernel_tlb_flush_all(cond, info); else - kernel_tlb_flush_range(info); + kernel_tlb_flush_range(cond, info); put_flush_tlb_info(); } +void flush_tlb_kernel_range(unsigned long start, unsigned long end) +{ + __flush_tlb_kernel_range(NULL, start, end); +} + +#ifdef CONFIG_COALESCE_TLBI +static bool flush_tlb_kernel_cond(int cpu, void *info) +{ + return housekeeping_cpu(cpu, HK_TYPE_KERNEL_NOISE) || + per_cpu(kernel_cr3_loaded, cpu); +} + +void flush_tlb_kernel_range_deferrable(unsigned long start, unsigned long = end) +{ + __flush_tlb_kernel_range(flush_tlb_kernel_cond, start, end); +} +#endif + /* * This can be used from process context to figure out what the value of * CR3 is without needing to do a (slow) __read_cr3(). diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 798b2ed21e460..76ec10d56623b 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -494,6 +494,30 @@ void vunmap_range_noflush(unsigned long start, unsigne= d long end) __vunmap_range_noflush(start, end); } +#ifdef CONFIG_COALESCE_TLBI +/* + * !!! BIG FAT WARNING !!! + * + * The CPU is free to cache any part of the paging hierarchy it wants at a= ny + * time. It's also free to set accessed and dirty bits at any time, even f= or + * instructions that may never execute architecturally. + * + * This means that deferring a TLB flush affecting freed page-table-pages = (IOW, + * keeping them in a CPU's paging hierarchy cache) is a recipe for disaste= r. + * + * This isn't a problem for deferral of TLB flushes in vmalloc, because + * page-table-pages used for vmap() mappings are never freed - see how + * __vunmap_range_noflush() walks the whole mapping but only clears the le= af PTEs. + * If this ever changes, TLB flush deferral will cause misery. + */ +void __weak flush_tlb_kernel_range_deferrable(unsigned long start, unsigne= d long end) +{ + flush_tlb_kernel_range(start, end); +} +#else +#define flush_tlb_kernel_range_deferrable(start, end) flush_tlb_kernel_ran= ge(start, end) +#endif + /** * vunmap_range - unmap kernel virtual addresses * @addr: start of the VM area to unmap @@ -507,7 +531,7 @@ void vunmap_range(unsigned long addr, unsigned long end) { flush_cache_vunmap(addr, end); vunmap_range_noflush(addr, end); - flush_tlb_kernel_range(addr, end); + flush_tlb_kernel_range_deferrable(addr, end); } static int vmap_pages_pte_range(pmd_t *pmd, unsigned long addr, @@ -2339,7 +2363,7 @@ static bool __purge_vmap_area_lazy(unsigned long star= t, unsigned long end, nr_purge_nodes =3D cpumask_weight(&purge_nodes); if (nr_purge_nodes > 0) { - flush_tlb_kernel_range(start, end); + flush_tlb_kernel_range_deferrable(start, end); /* One extra worker is per a lazy_max_pages() full set minus one. */ nr_purge_helpers =3D atomic_long_read(&vmap_lazy_nr) / lazy_max_pages(); @@ -2442,7 +2466,7 @@ static void free_unmap_vmap_area(struct vmap_area *va) flush_cache_vunmap(va->va_start, va->va_end); vunmap_range_noflush(va->va_start, va->va_end); if (debug_pagealloc_enabled_static()) - flush_tlb_kernel_range(va->va_start, va->va_end); + flush_tlb_kernel_range_deferrable(va->va_start, va->va_end); free_vmap_area_noflush(va); } @@ -2890,7 +2914,7 @@ static void vb_free(unsigned long addr, unsigned long= size) vunmap_range_noflush(addr, addr + size); if (debug_pagealloc_enabled_static()) - flush_tlb_kernel_range(addr, addr + size); + flush_tlb_kernel_range_deferrable(addr, addr + size); spin_lock(&vb->lock); @@ -2955,7 +2979,7 @@ static void _vm_unmap_aliases(unsigned long start, un= signed long end, int flush) free_purged_blocks(&purge_list); if (!__purge_vmap_area_lazy(start, end, false) && flush) - flush_tlb_kernel_range(start, end); + flush_tlb_kernel_range_deferrable(start, end); mutex_unlock(&vmap_purge_lock); } -- 2.51.0 From nobody Wed Dec 17 04:17:08 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8D36B325727 for ; Fri, 14 Nov 2025 15:17:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763133461; cv=none; b=O0PbJoa+qh433RCFAFbo6NzLSDXgY0VCfCT0QMRAsQjRW2KU4cpIOsZSB/P5fX6sl2w4qsO5X0c1k1HNh2iVeNgdkrOHkrI6/XeWAyJGq+6F9p4CBVNWszLmaajEtxE+QL4tt9OSxhQdZtZw028rAOWpx+3CYyY9dvLAw7jQ+nQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763133461; c=relaxed/simple; bh=k4QzhlWoFW0nKlX+gCLSWARw5LDdpUWLbvAWfPBN+UA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Dyy8iZexTV541Ie2y92Sc2H4LALRy21j2fU12epJ2pf/W7a549Of/LA8NiK9oY9jTIfEn0ForTVye4/Ck7P/YcP3txVfwA+Km3FSf8UeG8m0R42qJ+HImpJLxqtiafO9JKUS3l26cfQPmRNwMkmTT/YYeqTaHLPuV5Z9D+CsPYk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=OWqg9cxJ; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="OWqg9cxJ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1763133458; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YHbOf6w5rlj656BCFSOxLzgqhHO7ycdTLJssDuPvEBo=; b=OWqg9cxJnOOxtxuvf1BiFG+7fNQvWCTSxBTKvd4NTdP1M88EU43ab1DsEOM/BW/XBtUA3j FktMHeeAhU/MgfBdx+FcoShUI934Wz1zzthuZwo+QEDm+NBVcjYabTlRt+mjmaPEfpwFtg EpWnTPvHpMUvlDf6VxuOqSGNpgCgiUY= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-531--Ii0M5UDM1-LvTUINdpoDg-1; Fri, 14 Nov 2025 10:17:37 -0500 X-MC-Unique: -Ii0M5UDM1-LvTUINdpoDg-1 X-Mimecast-MFC-AGG-ID: -Ii0M5UDM1-LvTUINdpoDg_1763133453 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 2DC7C1956096; Fri, 14 Nov 2025 15:17:33 +0000 (UTC) Received: from vschneid-thinkpadt14sgen2i.remote.csb (unknown [10.45.226.10]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id A94EB180049F; Fri, 14 Nov 2025 15:17:18 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, rcu@vger.kernel.org, x86@kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Arnaldo Carvalho de Melo , Josh Poimboeuf , Paolo Bonzini , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Sami Tolvanen , "David S. Miller" , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki , Mathieu Desnoyers , Mel Gorman , Andrew Morton , Masahiro Yamada , Han Shen , Rik van Riel , Jann Horn , Dan Carpenter , Oleg Nesterov , Juri Lelli , Clark Williams , Yair Podemsky , Marcelo Tosatti , Daniel Wagner , Petr Tesarik , Shrikanth Hegde Subject: [RFC PATCH v7 31/31] x86/entry: Add an option to coalesce TLB flushes Date: Fri, 14 Nov 2025 16:14:28 +0100 Message-ID: <20251114151428.1064524-11-vschneid@redhat.com> In-Reply-To: <20251114150133.1056710-1-vschneid@redhat.com> References: <20251114150133.1056710-1-vschneid@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Content-Type: text/plain; charset="utf-8" Previous patches have introduced a mechanism to prevent kernel text updates from inducing interference on isolated CPUs. A similar action is required for kernel-range TLB flushes in order to silence the biggest remaining cause of isolated CPU IPI interference. These flushes are mostly caused by vmalloc manipulations - e.g. on x86 with CONFIG_VMAP_STACK, spawning enough processes will easily trigger flushes. Unfortunately, the newly added context_tracking IPI deferral mechanism cannot be leveraged for TLB flushes, as the deferred work would be executed too late. Consider the following execution flow: !interrupt! SWITCH_TO_KERNEL_CR3 // vmalloc range becomes accessible idtentry_func_foo() irqentry_enter() irqentry_enter_from_user_mode() enter_from_user_mode() [...] ct_kernel_enter_state() ct_work_flush() // deferred flush would be done here Since there is no sane way to assert no stale entry is accessed during kernel entry, any code executed between SWITCH_TO_KERNEL_CR3 and ct_work_flush() is at risk of accessing a stale entry. Dave had suggested hacking up something within SWITCH_TO_KERNEL_CR3 itself, which is what has been implemented in the previous patches. Make kernel-range TLB flush deferral available via CONFIG_COALESCE_TLBI. Signed-off-by: Valentin Schneider --- arch/x86/Kconfig | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index fa9229c0e0939..04f9d6496bbbc 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -2189,6 +2189,23 @@ config ADDRESS_MASKING The capability can be used for efficient address sanitizers (ASAN) implementation and for optimizations in JITs. +config COALESCE_TLBI + def_bool n + prompt "Coalesce kernel TLB flushes for NOHZ-full CPUs" + depends on X86_64 && MITIGATION_PAGE_TABLE_ISOLATION && NO_HZ_FULL + help + TLB flushes for kernel addresses can lead to IPIs being sent to + NOHZ-full CPUs, thus kicking them out of userspace. + + This option coalesces kernel-range TLB flushes for NOHZ-full CPUs into + a single flush executed at kernel entry, right after switching to the + kernel page table. Note that this flush is unconditionnal, even if no + remote flush was issued during the previous userspace execution window. + + This obviously makes the user->kernel transition overhead even worse. + + If unsure, say N. + config HOTPLUG_CPU def_bool y depends on SMP -- 2.51.0