From nobody Tue Dec 30 12:50:30 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3961FC54FB9 for ; Wed, 15 Nov 2023 17:37:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232528AbjKORhd (ORCPT ); Wed, 15 Nov 2023 12:37:33 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48418 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231894AbjKORha (ORCPT ); Wed, 15 Nov 2023 12:37:30 -0500 Received: from mail-qk1-x735.google.com (mail-qk1-x735.google.com [IPv6:2607:f8b0:4864:20::735]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EF9BD83 for ; Wed, 15 Nov 2023 09:37:26 -0800 (PST) Received: by mail-qk1-x735.google.com with SMTP id af79cd13be357-77a277eb084so456483885a.2 for ; Wed, 15 Nov 2023 09:37:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700069845; x=1700674645; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=M04sUMEmylKhcxCvp1sLw6caUnGoADRX8jt4cdDowFM=; b=G8GbXkxnIRiQD1WtTb9sJV81aEwsEtbcJXXPVw16DZrQmZYO0OmIHbvyhBcDfjViSI 2i85+lhcunNYFRC3HUmQa+NwUFS9fI1C9ThowM0MDASDFs3HTLUFdfxuh6SpeB/UW1Pg tQgeEOXPVMl2cl5Pc859wHSWZ2GlREWZSKzTEgpdbZmMPWCl3PhMtKOJ5C+P2eBgZ+/s NKmMICZ9zfgiK9ebVbQIytyrys1Gz4lKMze6ZIf0xcE6lxpCLwtuzfJnSRixIh0eNABd rhKRrIZPA96q937ByzRy6wQ8WpbvWNjqdUMN4dWKmtXHCfQKNdb4x5TEW7rqxIYYqXap oCiA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700069845; x=1700674645; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=M04sUMEmylKhcxCvp1sLw6caUnGoADRX8jt4cdDowFM=; b=TX7CRToV6Z/0Gjl65/bmI0g0H9oTFa3uhlpDsw2EGXO14SX1lcr/DOosN91TV9qCMr SKZNIxVfFB6vBmjBiXBZ+TcRmlMlxUJniSDW6EfvFi12TDCD4OK27rCaHLCFJpLWSXQV y4pHeyLIGiArKsA1KpkfpkaF8ilPqS4cKQGvUDs//TB8Qv63y9n/YQLUsdQ0R3wkHvzc yB1v0KFY08RtfGZvbgoA53lEjb0v4o85/UzqTC3CiWQX6FXeDmmjx/jIN6noNGz17bM2 Hfr8wyVA/g38AH0YYAVjgl+EtogrJ+pH3SkUZN1rGM4fODOQmRKaCk9M67pBhw89f+cY QOvQ== X-Gm-Message-State: AOJu0YwR2Pqyo7X8ZguUwEhXWH+HmV0SijKq5dnYlljlf5w0Iye7ZBMi /GSjQZXj1NLpoLEmace2lljPj1dPfg== X-Google-Smtp-Source: AGHT+IGv/GvoH+u6ZPMC5Nk6aJ8mdZ5hpmFr7IUe4AjU06WSnBNR6QJDQGYlfcnMjmOcHTogaHrJYA== X-Received: by 2002:a05:620a:38c7:b0:778:ba73:1650 with SMTP id qq7-20020a05620a38c700b00778ba731650mr4745937qkn.30.1700069845162; Wed, 15 Nov 2023 09:37:25 -0800 (PST) Received: from citadel.lan ([2600:6c4a:4d3f:6d5c::1019]) by smtp.gmail.com with ESMTPSA id x20-20020a05620a0b5400b0077bda014d8esm3260099qkg.87.2023.11.15.09.37.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Nov 2023 09:37:24 -0800 (PST) From: Brian Gerst To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: Ingo Molnar , Thomas Gleixner , Borislav Petkov , "H . Peter Anvin" , Peter Zijlstra , Uros Bizjak , David.Laight@aculab.com, Brian Gerst Subject: [PATCH v3 01/14] x86/stackprotector/32: Remove stack protector test script Date: Wed, 15 Nov 2023 12:36:55 -0500 Message-ID: <20231115173708.108316-2-brgerst@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231115173708.108316-1-brgerst@gmail.com> References: <20231115173708.108316-1-brgerst@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Test for compiler support directly in Kconfig. Signed-off-by: Brian Gerst --- arch/x86/Kconfig | 2 +- scripts/gcc-x86_32-has-stack-protector.sh | 8 -------- 2 files changed, 1 insertion(+), 9 deletions(-) delete mode 100755 scripts/gcc-x86_32-has-stack-protector.sh diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 882b11eccbb3..46c55fd7ca86 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -411,7 +411,7 @@ config PGTABLE_LEVELS config CC_HAS_SANE_STACKPROTECTOR bool default $(success,$(srctree)/scripts/gcc-x86_64-has-stack-protector.sh $(= CC) $(CLANG_FLAGS)) if 64BIT - default $(success,$(srctree)/scripts/gcc-x86_32-has-stack-protector.sh $(= CC) $(CLANG_FLAGS)) + default $(cc-option,-mstack-protector-guard-reg=3Dfs -mstack-protector-gu= ard-symbol=3D__stack_chk_guard) help We have to make sure stack protector is unconditionally disabled if the compiler produces broken code or if it does not let us control diff --git a/scripts/gcc-x86_32-has-stack-protector.sh b/scripts/gcc-x86_32= -has-stack-protector.sh deleted file mode 100755 index 825c75c5b715..000000000000 --- a/scripts/gcc-x86_32-has-stack-protector.sh +++ /dev/null @@ -1,8 +0,0 @@ -#!/bin/sh -# SPDX-License-Identifier: GPL-2.0 - -# This requires GCC 8.1 or better. Specifically, we require -# -mstack-protector-guard-reg, added by -# https://gcc.gnu.org/bugzilla/show_bug.cgi?id=3D81708 - -echo "int foo(void) { char X[200]; return 3; }" | $* -S -x c -c -m32 -O0 -= fstack-protector -mstack-protector-guard-reg=3Dfs -mstack-protector-guard-s= ymbol=3D__stack_chk_guard - -o - 2> /dev/null | grep -q "%fs" --=20 2.41.0 From nobody Tue Dec 30 12:50:30 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1AEC6C2BB3F for ; Wed, 15 Nov 2023 17:37:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232636AbjKORhg (ORCPT ); Wed, 15 Nov 2023 12:37:36 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60776 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232401AbjKORhb (ORCPT ); Wed, 15 Nov 2023 12:37:31 -0500 Received: from mail-qk1-x72e.google.com (mail-qk1-x72e.google.com [IPv6:2607:f8b0:4864:20::72e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5BE81196 for ; Wed, 15 Nov 2023 09:37:28 -0800 (PST) Received: by mail-qk1-x72e.google.com with SMTP id af79cd13be357-77891f362cfso95334585a.1 for ; Wed, 15 Nov 2023 09:37:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700069846; x=1700674646; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QQoC9i+SJdGrvV/tO6Xn3N5JH39L1Nc9The4vgz1I64=; b=f15IXd/ORWdLM5P0ax5IPt5Rn6AAt+VKROAp5rBsKebk7SYsnvto1bsLPDP+NMQdvn dM90RUAEtRZ6e4fZm84E1a4Qgy4Z6qeXeuvLu9b9tx1auPq38H9KiXeOI2mr6km4xETg QpR+6FSxGWA/01Sieh5rQX5W7Vfl0rgE46MGTIraF0g+qmpZGNW0Dbu7/wtR1NyW2zMy zknZ4GmV0kKvi3NCCKsKGtZDQzrwXuvIRth4SG1a2/grOD31SUM7PxnLyGP7tFFtiiNi hvwoJ5vb8DH8V7xXDfT0gIvBkjebNZUHycgmLnwJPTjaaJZs+ScVqyTJEy07sn9eM/ek jRuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700069846; x=1700674646; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QQoC9i+SJdGrvV/tO6Xn3N5JH39L1Nc9The4vgz1I64=; b=m+HIuVntOyQA5vSWSmEd9djC8mIMYuubbeZuRwJmpIIdHn5zdiwbV4bIq1j9WA2IUk FrSyLcfc/ZnsGV1mcpsZgdGlShc+l13sn8LnPXfp2fO/VEM8Umk/VZ9RAEzl5kEHI0ed CN068Hc5io16tcAiHJnVKWsMhpP7tClNLZ4j/UG0C+YdMXhmnysBkwwsnJdQVQycOtOP gTqZZ0Ou4nAKFWEoPlPQ7cjhWGzHSjWCjUILmFEx9umzKIEB54N4mb17TO1g8DP5nVqV PcXqQe0uOFgC6Z4wb8ErJ+aQXc1NS7jMbtPoNrS5oJnhRdt401bpkVegpQ+1ozcFB/J6 FkqQ== X-Gm-Message-State: AOJu0YyjKdN/md5rwFd8cg43Jd4InFLRwmRK2l+G2OG15Atm+5qhe6dI 3x5MnEpsvoI82FLprgpQyj6lkf415A== X-Google-Smtp-Source: AGHT+IGMXjNqkuSzjoGvur/XIBWUx1Pyb6YhUOZ6uponQtwlbT8p7gl83U2+dPSqFNHhprGiL3tQdA== X-Received: by 2002:a05:620a:bd3:b0:778:969e:3a0c with SMTP id s19-20020a05620a0bd300b00778969e3a0cmr8867556qki.27.1700069846651; Wed, 15 Nov 2023 09:37:26 -0800 (PST) Received: from citadel.lan ([2600:6c4a:4d3f:6d5c::1019]) by smtp.gmail.com with ESMTPSA id x20-20020a05620a0b5400b0077bda014d8esm3260099qkg.87.2023.11.15.09.37.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Nov 2023 09:37:25 -0800 (PST) From: Brian Gerst To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: Ingo Molnar , Thomas Gleixner , Borislav Petkov , "H . Peter Anvin" , Peter Zijlstra , Uros Bizjak , David.Laight@aculab.com, Brian Gerst Subject: [PATCH v3 02/14] x86/stackprotector/64: Remove stack protector test script Date: Wed, 15 Nov 2023 12:36:56 -0500 Message-ID: <20231115173708.108316-3-brgerst@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231115173708.108316-1-brgerst@gmail.com> References: <20231115173708.108316-1-brgerst@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This test for the stack protector was added in 2006 to make sure the compiler had the PR28281 patch applied. With GCC 5.1 being the minimum supported compiler now, it is no longer necessary. Signed-off-by: Brian Gerst --- arch/x86/Kconfig | 5 ++--- scripts/gcc-x86_64-has-stack-protector.sh | 4 ---- 2 files changed, 2 insertions(+), 7 deletions(-) delete mode 100755 scripts/gcc-x86_64-has-stack-protector.sh diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 46c55fd7ca86..a1d2f7fe42bb 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -410,12 +410,11 @@ config PGTABLE_LEVELS =20 config CC_HAS_SANE_STACKPROTECTOR bool - default $(success,$(srctree)/scripts/gcc-x86_64-has-stack-protector.sh $(= CC) $(CLANG_FLAGS)) if 64BIT + default y if 64BIT default $(cc-option,-mstack-protector-guard-reg=3Dfs -mstack-protector-gu= ard-symbol=3D__stack_chk_guard) help We have to make sure stack protector is unconditionally disabled if - the compiler produces broken code or if it does not let us control - the segment on 32-bit kernels. + the compiler does not allow control of the segment and symbol. =20 menu "Processor type and features" =20 diff --git a/scripts/gcc-x86_64-has-stack-protector.sh b/scripts/gcc-x86_64= -has-stack-protector.sh deleted file mode 100755 index 75e4e22b986a..000000000000 --- a/scripts/gcc-x86_64-has-stack-protector.sh +++ /dev/null @@ -1,4 +0,0 @@ -#!/bin/sh -# SPDX-License-Identifier: GPL-2.0 - -echo "int foo(void) { char X[200]; return 3; }" | $* -S -x c -c -m64 -O0 -= mcmodel=3Dkernel -fno-PIE -fstack-protector - -o - 2> /dev/null | grep -q "= %gs" --=20 2.41.0 From nobody Tue Dec 30 12:50:30 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB846C072A2 for ; Wed, 15 Nov 2023 17:37:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232690AbjKORhj (ORCPT ); Wed, 15 Nov 2023 12:37:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60780 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232502AbjKORhd (ORCPT ); Wed, 15 Nov 2023 12:37:33 -0500 Received: from mail-qk1-x729.google.com (mail-qk1-x729.google.com [IPv6:2607:f8b0:4864:20::729]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6ACFF1BC for ; Wed, 15 Nov 2023 09:37:29 -0800 (PST) Received: by mail-qk1-x729.google.com with SMTP id af79cd13be357-779fb118fe4so433073785a.2 for ; Wed, 15 Nov 2023 09:37:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700069848; x=1700674648; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=lsKN0vAWTPkRoeR+0eW+M/Vw4EKK+fkuLodskFQx/7g=; b=Wu+HWNRJP1QjHx24kqcqFNFJCVp93WItLBa+xA7hYcTXz5WmGggUg9mPoYBL8sjgXa a2t6J//qXG5k21HdhLWhjaguy7okJtzL061xu/u39SK9ID/kb9Vc1LiTuEgoTuMRYtcB mxRpotpf7Kb83TwMgjpJ9sx8UDhEWrZfxBB7kl+04JB4xc2WFCv5CsS/0Q8IFKp8h8sN EHQdHT6zU0plpk6Y2Vz2vE2A10IcqGAm+apYWgVUxjXZNDlzivpIN2HTC/5EF1EFR55R wo6dMSgQVLpfQWMaDSSyFX4R+dNuqNVZB86LCODvUcHNyPaY5XIehzlmjUZKIr9Nrj30 xaRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700069848; x=1700674648; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lsKN0vAWTPkRoeR+0eW+M/Vw4EKK+fkuLodskFQx/7g=; b=m/eNDQWx/INaONHUHWYPkb84xWMdnJFIKrZ9cx+rQPWgnUHUxxY/bqXDH/EqY3EdPu +Nbx6EhhIghIahftoOcQglH4EGPBFAh+GPhCT/itCOLX13J6ze30au2YgvYWeF0cZiNq iZ3DY6YWaMMFSRvqJorzx+H/Chyy2pitO9bKa55jFLN4OJ74GwZAPyitGc+76dxAZOHC hjdebfvamXo4ffN749kxKuUTFVwEvTHsGnaUqeNR6SmvF0UPe327q9+3V0V3El0DvMGs nWaRd2do6NsGh1rbraW/TCpu8XAIdO7+Oc340FZOZRS1G9OaXKuBAIUcti0xKXVqXMpN jL4w== X-Gm-Message-State: AOJu0YxJLxYNJDfdus2onDaWbSYeF6gHVJzNsreH9SmIsurspnBADYAp ZvRzL8NsjuigsrgRboks/cciKA3MNQ== X-Google-Smtp-Source: AGHT+IGwig5qnejJPIdhkdSJOCxU62M+zRKtaqk3xMO3i4imN2kV5NiZ6RaML1EpS177QvcfBXiOcg== X-Received: by 2002:a05:620a:6409:b0:779:d143:c1a7 with SMTP id pz9-20020a05620a640900b00779d143c1a7mr6814784qkn.26.1700069848097; Wed, 15 Nov 2023 09:37:28 -0800 (PST) Received: from citadel.lan ([2600:6c4a:4d3f:6d5c::1019]) by smtp.gmail.com with ESMTPSA id x20-20020a05620a0b5400b0077bda014d8esm3260099qkg.87.2023.11.15.09.37.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Nov 2023 09:37:27 -0800 (PST) From: Brian Gerst To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: Ingo Molnar , Thomas Gleixner , Borislav Petkov , "H . Peter Anvin" , Peter Zijlstra , Uros Bizjak , David.Laight@aculab.com, Brian Gerst Subject: [PATCH v3 03/14] x86/boot: Disable stack protector for early boot code Date: Wed, 15 Nov 2023 12:36:57 -0500 Message-ID: <20231115173708.108316-4-brgerst@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231115173708.108316-1-brgerst@gmail.com> References: <20231115173708.108316-1-brgerst@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" On 64-bit, this will prevent crashes when the canary access is changed from %gs:40 to %gs:__stack_chk_guard(%rip). RIP-relative addresses from the identity-mapped early boot code will target the wrong address with zero-based percpu. KASLR could then shift that address to an unmapped page causing a crash on boot. This early boot code runs well before userspace is active and does not need stack protector enabled. Signed-off-by: Brian Gerst --- arch/x86/kernel/Makefile | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile index 0000325ab98f..aff619054e17 100644 --- a/arch/x86/kernel/Makefile +++ b/arch/x86/kernel/Makefile @@ -39,6 +39,8 @@ KMSAN_SANITIZE_nmi.o :=3D n KCOV_INSTRUMENT_head$(BITS).o :=3D n KCOV_INSTRUMENT_sev.o :=3D n =20 +CFLAGS_head32.o :=3D -fno-stack-protector +CFLAGS_head64.o :=3D -fno-stack-protector CFLAGS_irq.o :=3D -I $(srctree)/$(src)/../include/asm/trace =20 obj-y +=3D head_$(BITS).o --=20 2.41.0 From nobody Tue Dec 30 12:50:30 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61992C072A2 for ; Wed, 15 Nov 2023 17:37:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232502AbjKORhm (ORCPT ); Wed, 15 Nov 2023 12:37:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60808 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232628AbjKORhg (ORCPT ); Wed, 15 Nov 2023 12:37:36 -0500 Received: from mail-qk1-x733.google.com (mail-qk1-x733.google.com [IPv6:2607:f8b0:4864:20::733]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 57F511BD for ; Wed, 15 Nov 2023 09:37:31 -0800 (PST) Received: by mail-qk1-x733.google.com with SMTP id af79cd13be357-7781bc3783fso459159685a.1 for ; Wed, 15 Nov 2023 09:37:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700069850; x=1700674650; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PLUf2tkG8NZ4kfbeIrEm6olUf9NTE49Bj17ZfTFY3SY=; b=SEDG8xAQ8M4YUl7aNvW53zzlkGKixvtyFTOFSn4Q2KmfgCGUfp5jFJ6CbJBWg17b+S ufYPpeM22fNqQGNBXG+IS7fHm4DfU1N85bdV/mAaBURgtbI81yWx026QPnKbPxZ22VaP v+wOHouYq1pNqJrWp0mBr3Jx2/SfeFgSEyE2dtr9YM7HUxFczn72QYRqByATXKAUjcul RoqZ5iavlXMU7OK+tj8ZEX0oWkiPde2BOn/mep5NY8jXlAjHuAboFQzQZRjkHhayAZY1 Nj7OsbuNBhueJHsjxRqufDjeKAnmk3I0IumaoWcI4M/m3dB6/onw36HARrs9M5KuSXR0 I42A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700069850; x=1700674650; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PLUf2tkG8NZ4kfbeIrEm6olUf9NTE49Bj17ZfTFY3SY=; b=pKVQPyvgsAq09fmPfMRhIxnOO+NFTqHq858q8j4whvAOguVfxSCrqqkf3hbFm0YbxW EvtXn9NPvCmM+hjF2OL9/HWvMGrl3w7JwLD4L748tAVES3DtJwwwIHvbY0wLDOJ/LEYv 9J8Eiy7sxagrrHRGd/Q+JOUF3CNk5S1fnQzDvITbXihvCiZFpBNe8il/FQE96qRxB/q/ zxlCkos/66G3regTWS7s1P3E8sP5ZTbRQhWas3FOhBMQrpS1ur2P1fbr92p8KQedOqvw K+d82h0icr2psO8ZXhPV9OEtx8lYZrbyFV7+IO71nWkxKkxO+LYAnXI6rScLMHfdV5VO D6wA== X-Gm-Message-State: AOJu0YyCdpyC3bc0EixsuJNWHOEckr1DzhMSF1Q3WKJA2NlDMW5YmmII pgCNxIieVf6EqkhIxlKAFZaYO0fkDw== X-Google-Smtp-Source: AGHT+IH3FHEZ6WXjvOoIbOQbonAxGMCzxREdT8gD6rGN4CVKomo7hL9rr8ILmrb81OpvVFfEWLlwXA== X-Received: by 2002:a05:620a:284d:b0:778:9824:4b7a with SMTP id h13-20020a05620a284d00b0077898244b7amr6407829qkp.69.1700069850080; Wed, 15 Nov 2023 09:37:30 -0800 (PST) Received: from citadel.lan ([2600:6c4a:4d3f:6d5c::1019]) by smtp.gmail.com with ESMTPSA id x20-20020a05620a0b5400b0077bda014d8esm3260099qkg.87.2023.11.15.09.37.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Nov 2023 09:37:28 -0800 (PST) From: Brian Gerst To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: Ingo Molnar , Thomas Gleixner , Borislav Petkov , "H . Peter Anvin" , Peter Zijlstra , Uros Bizjak , David.Laight@aculab.com, Brian Gerst Subject: [PATCH v3 04/14] x86/pvh: Use fixed_percpu_data for early boot GSBASE Date: Wed, 15 Nov 2023 12:36:58 -0500 Message-ID: <20231115173708.108316-5-brgerst@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231115173708.108316-1-brgerst@gmail.com> References: <20231115173708.108316-1-brgerst@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Instead of having a private area for the stack canary, use fixed_percpu_data for GSBASE like the native kernel. Signed-off-by: Brian Gerst --- arch/x86/platform/pvh/head.S | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/arch/x86/platform/pvh/head.S b/arch/x86/platform/pvh/head.S index c4365a05ab83..fab90368481f 100644 --- a/arch/x86/platform/pvh/head.S +++ b/arch/x86/platform/pvh/head.S @@ -94,10 +94,15 @@ SYM_CODE_START_LOCAL(pvh_start_xen) /* 64-bit entry point. */ .code64 1: - /* Set base address in stack canary descriptor. */ + /* + * Set up GSBASE. + * Note that, on SMP, the boot cpu uses init data section until + * the per cpu areas are set up. + */ mov $MSR_GS_BASE,%ecx - mov $_pa(canary), %eax - xor %edx, %edx + lea INIT_PER_CPU_VAR(fixed_percpu_data)(%rip), %rdx + mov %edx, %eax + shr $32, %rdx wrmsr =20 call xen_prepare_pvh @@ -156,8 +161,6 @@ SYM_DATA_START_LOCAL(gdt_start) SYM_DATA_END_LABEL(gdt_start, SYM_L_LOCAL, gdt_end) =20 .balign 16 -SYM_DATA_LOCAL(canary, .fill 48, 1, 0) - SYM_DATA_START_LOCAL(early_stack) .fill BOOT_STACK_SIZE, 1, 0 SYM_DATA_END_LABEL(early_stack, SYM_L_LOCAL, early_stack_end) --=20 2.41.0 From nobody Tue Dec 30 12:50:30 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29180C072A2 for ; Wed, 15 Nov 2023 17:37:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232803AbjKORhq (ORCPT ); Wed, 15 Nov 2023 12:37:46 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60822 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232401AbjKORhg (ORCPT ); Wed, 15 Nov 2023 12:37:36 -0500 Received: from mail-qk1-x736.google.com (mail-qk1-x736.google.com [IPv6:2607:f8b0:4864:20::736]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 49BC1D40 for ; Wed, 15 Nov 2023 09:37:33 -0800 (PST) Received: by mail-qk1-x736.google.com with SMTP id af79cd13be357-778a47bc09aso456269685a.3 for ; Wed, 15 Nov 2023 09:37:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700069851; x=1700674651; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Zo9KuMlI56r1OXsGbX0xxREbBs5j59brj7SQCaFRmtQ=; b=YlSAaIisc2+Kwg/Dp7A1rbRAgvDK9SSM+xxXe3z603lkeapq1kA6CcUrsjwnonDDSC gl/I58gNoaQJy/tKlOsMgYKCycA4ZOVE19j0HruzQ/f13cDH1s7Bda1euCYhitcsAnyx WnelsdQKFAk7j0RbEiaxjJ60Jd+J6Nz8FzprbW89iH2iVnlly9kpU06I84xekpcPYXV6 5dw9FTtTgzkb69HHWFNdV8YDb6jOnwPuKKNLH1UDWcfYpmImfRgrS92qaXxopiD5bIxn W9L9qOX/+aCeZ/rs8J7a/H96WNLPqzG88Fz+5Zyvp726WUjWtTh6TCnBYNiR0VEUrJnS 0jaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700069851; x=1700674651; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Zo9KuMlI56r1OXsGbX0xxREbBs5j59brj7SQCaFRmtQ=; b=vzgbJCVCYIJhutiodfKRPjIT62k31/rP9KODNbWxfmwoTJTHM8LfCkzLDxOaZiPnMH VetF9S6KLwqTEll5tA7ky7bHZhVKpApZoNIgGxyHL3ilu2sADkmhrmilmPJPPqmapoRr cvLyzA+R5tdc0l0jda7nXQQkPLjbwhYu6BsKTmgvebeasKuwbRc6eGHi2Uc+BQlg+OLH lRZeB1PEQx9XloGLU9CgpFkh5b9Qh1u/hFCsX+4T/UrMVAEWbt1iq5ZoGWYYs+kiEzI8 AD6SA4J9ZnaLwgm+RgcqickBFoatw9DWV65iBoqaLitizYibgK7kaCljCL5l5tiPs9a7 rjjA== X-Gm-Message-State: AOJu0Yy0sbCpzCP/nt3wyH2tcBlHXyP6AHa28y7VaIGBcyR9xSVtuPbK bJZ9wrYBoPneDbDK+Rq175mYx7lfow== X-Google-Smtp-Source: AGHT+IG+38RoeXB/vuqSBSQOs8a9+IbwMtcj+WJGDLZ5zqnhopauoG4E8yH6u5ucPa15J1tJkxzgxA== X-Received: by 2002:a05:620a:8703:b0:777:ac06:ed91 with SMTP id px3-20020a05620a870300b00777ac06ed91mr5580042qkn.23.1700069851190; Wed, 15 Nov 2023 09:37:31 -0800 (PST) Received: from citadel.lan ([2600:6c4a:4d3f:6d5c::1019]) by smtp.gmail.com with ESMTPSA id x20-20020a05620a0b5400b0077bda014d8esm3260099qkg.87.2023.11.15.09.37.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Nov 2023 09:37:30 -0800 (PST) From: Brian Gerst To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: Ingo Molnar , Thomas Gleixner , Borislav Petkov , "H . Peter Anvin" , Peter Zijlstra , Uros Bizjak , David.Laight@aculab.com, Brian Gerst Subject: [PATCH v3 05/14] x86/relocs: Handle R_X86_64_REX_GOTPCRELX relocations Date: Wed, 15 Nov 2023 12:36:59 -0500 Message-ID: <20231115173708.108316-6-brgerst@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231115173708.108316-1-brgerst@gmail.com> References: <20231115173708.108316-1-brgerst@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Clang may produce R_X86_64_REX_GOTPCRELX relocations when redefining the stack protector location. Treat them as another type of PC-relative relocation. Signed-off-by: Brian Gerst --- arch/x86/tools/relocs.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/arch/x86/tools/relocs.c b/arch/x86/tools/relocs.c index d30949e25ebd..24ad10c62840 100644 --- a/arch/x86/tools/relocs.c +++ b/arch/x86/tools/relocs.c @@ -31,6 +31,11 @@ static struct relocs relocs32; static struct relocs relocs32neg; static struct relocs relocs64; #define FMT PRIu64 + +#ifndef R_X86_64_REX_GOTPCRELX +#define R_X86_64_REX_GOTPCRELX 42 +#endif + #else #define FMT PRIu32 #endif @@ -224,6 +229,7 @@ static const char *rel_type(unsigned type) REL_TYPE(R_X86_64_PC16), REL_TYPE(R_X86_64_8), REL_TYPE(R_X86_64_PC8), + REL_TYPE(R_X86_64_REX_GOTPCRELX), #else REL_TYPE(R_386_NONE), REL_TYPE(R_386_32), @@ -848,6 +854,7 @@ static int do_reloc64(struct section *sec, Elf_Rel *rel= , ElfW(Sym) *sym, =20 case R_X86_64_PC32: case R_X86_64_PLT32: + case R_X86_64_REX_GOTPCRELX: /* * PC relative relocations don't need to be adjusted unless * referencing a percpu symbol. --=20 2.41.0 From nobody Tue Dec 30 12:50:30 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DDF99C072A2 for ; Wed, 15 Nov 2023 17:37:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232754AbjKORhu (ORCPT ); Wed, 15 Nov 2023 12:37:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60808 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232661AbjKORhi (ORCPT ); Wed, 15 Nov 2023 12:37:38 -0500 Received: from mail-qk1-x729.google.com (mail-qk1-x729.google.com [IPv6:2607:f8b0:4864:20::729]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C6D82D42 for ; Wed, 15 Nov 2023 09:37:33 -0800 (PST) Received: by mail-qk1-x729.google.com with SMTP id af79cd13be357-777754138bdso463506385a.1 for ; Wed, 15 Nov 2023 09:37:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700069852; x=1700674652; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=RDtZpQP+OF9Ls+8v3KDmKE8Uj5TeqYlQFmL5zNUIneg=; b=gnShT4zMGR0c3+n4L/hevwdxmAABbJpv8jF3Oc/oqHKDky/oFKNzdnGUZEuqlIxY9M 1NBk2OKtza+mnOC34AL2odoj9uNXUpaMiB5JKgp5daXstN+qdWdAmDyNEwgNgi2m6ZHp VBI0MizUZqR0BJgr/HUTap0Uy5N2CAdndoRQkXzGYL3upDkkPbA9L0f7KF7231D2UEqK nbiOSObLCJpNg0pX3WTnQu6qyvDFZgpjueN6zlz4+OnruTaQde/a3tWq9OvZehGMtDy+ AWdVPF6ccBTLc/VrmnAVaAipH45rsiff3Tz0nSEFml6sSzTmXsB0n6dfa9T/5Ba8y8KS 049Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700069852; x=1700674652; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RDtZpQP+OF9Ls+8v3KDmKE8Uj5TeqYlQFmL5zNUIneg=; b=WMyU12fZ/gKiJIDMJHWyIGDaCd+XvlpSmr5g/dNMd63JAuEzn5eMY0nmzCWBYpokbF SWNOBuS1hw0zypYj1MGnn2hQwyuxqUr9F2pE3puiZe2FKvvUar72xDRz1Pl6RSx30XKK T17SHISB4A/kgjq6IE6wNo/AOW0ZhiyMXnOAeEO/XXZ62krr29dA3jx2BaaFhp8hOz13 FxIiiofLzRlN9okj1jIOtxYCkQ+4bwS0KglsN9btSUSeV2hddMlqzRQKQBvWKofEKGct q84L+3TLt2tsPkXzMK3yjIMlVrodfwC24Dy5GcHXVJ5q4ObpQkXrGKmV5l0gTeM6YzDA 8qyA== X-Gm-Message-State: AOJu0Yylnw9qt9LLIsk/IbY6wrFpqBRAymMOgqbCxFs5+6f/pLUmvPJB f+/B0u2boyQNWbVgNYLulbAVcNJeew== X-Google-Smtp-Source: AGHT+IG9Qzm4oanuaHXMTBkxXo9dMyTjXWqZwFaz23KZ036fGHKDJU5ReaE4228kHI4VhQrnVl7fCQ== X-Received: by 2002:a05:620a:8418:b0:778:92fb:a8c4 with SMTP id pc24-20020a05620a841800b0077892fba8c4mr5312357qkn.30.1700069852549; Wed, 15 Nov 2023 09:37:32 -0800 (PST) Received: from citadel.lan ([2600:6c4a:4d3f:6d5c::1019]) by smtp.gmail.com with ESMTPSA id x20-20020a05620a0b5400b0077bda014d8esm3260099qkg.87.2023.11.15.09.37.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Nov 2023 09:37:31 -0800 (PST) From: Brian Gerst To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: Ingo Molnar , Thomas Gleixner , Borislav Petkov , "H . Peter Anvin" , Peter Zijlstra , Uros Bizjak , David.Laight@aculab.com, Brian Gerst Subject: [PATCH v3 06/14] objtool: Allow adding relocations to an existing section Date: Wed, 15 Nov 2023 12:37:00 -0500 Message-ID: <20231115173708.108316-7-brgerst@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231115173708.108316-1-brgerst@gmail.com> References: <20231115173708.108316-1-brgerst@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" In order to add relocations to existing sections (e.g. ".rela.text"), encapsulate the reloc array in a block header to allow chaining blocks to add more relocs without moving and relinking the existing ones. This adds minimal memory overhead, while still being able to easily access the arrays by index. Signed-off-by: Brian Gerst --- tools/objtool/check.c | 2 +- tools/objtool/elf.c | 99 +++++++++++++++++++++++------ tools/objtool/include/objtool/elf.h | 84 +++++++++++++++++++----- 3 files changed, 148 insertions(+), 37 deletions(-) diff --git a/tools/objtool/check.c b/tools/objtool/check.c index e94756e09ca9..ac304140c395 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -4409,7 +4409,7 @@ static int validate_ibt_data_reloc(struct objtool_fil= e *file, return 0; =20 WARN_FUNC("data relocation to !ENDBR: %s", - reloc->sec->base, reloc_offset(reloc), + reloc->block->sec->base, reloc_offset(reloc), offstr(dest->sec, dest->offset)); =20 return 1; diff --git a/tools/objtool/elf.c b/tools/objtool/elf.c index 3d27983dc908..cfb970727c8a 100644 --- a/tools/objtool/elf.c +++ b/tools/objtool/elf.c @@ -277,7 +277,7 @@ struct reloc *find_reloc_by_dest_range(const struct elf= *elf, struct section *se for_offset_range(o, offset, offset + len) { elf_hash_for_each_possible(reloc, reloc, hash, sec_offset_hash(rsec, o)) { - if (reloc->sec !=3D rsec) + if (reloc->block->sec !=3D rsec) continue; =20 if (reloc_offset(reloc) >=3D offset && @@ -333,6 +333,7 @@ static int read_sections(struct elf *elf) sec =3D &elf->section_data[i]; =20 INIT_LIST_HEAD(&sec->symbol_list); + INIT_LIST_HEAD(&sec->reloc_list); =20 s =3D elf_getscn(elf->elf, i); if (!s) { @@ -850,7 +851,7 @@ static struct reloc *elf_init_reloc(struct elf *elf, st= ruct section *rsec, unsigned long offset, struct symbol *sym, s64 addend, unsigned int type) { - struct reloc *reloc, empty =3D { 0 }; + struct reloc *reloc; =20 if (reloc_idx >=3D sec_num_entries(rsec)) { WARN("%s: bad reloc_idx %u for %s with %d relocs", @@ -858,15 +859,18 @@ static struct reloc *elf_init_reloc(struct elf *elf, = struct section *rsec, return NULL; } =20 - reloc =3D &rsec->relocs[reloc_idx]; + reloc =3D get_reloc_by_index(rsec, reloc_idx); + if (!reloc) { + WARN("%s: %s: reloc %d out of range!", + __func__, rsec->name, reloc_idx); + return NULL; + } =20 - if (memcmp(reloc, &empty, sizeof(empty))) { + if (reloc->sym) { WARN("%s: %s: reloc %d already initialized!", __func__, rsec->name, reloc_idx); return NULL; } - - reloc->sec =3D rsec; reloc->sym =3D sym; =20 set_reloc_offset(elf, reloc, offset); @@ -930,19 +934,45 @@ struct reloc *elf_init_reloc_data_sym(struct elf *elf= , struct section *sec, elf_data_rela_type(elf)); } =20 +static struct reloc_block *alloc_reloc_block(struct section *rsec, size_t = num_relocs) +{ + struct reloc_block *block; + size_t block_size =3D sizeof(struct reloc_block) + sec_num_entries(rsec) = * sizeof(struct reloc); + int i; + + block =3D malloc(block_size); + if (!block) { + perror("malloc"); + return NULL; + } + + memset(block, 0, block_size); + INIT_LIST_HEAD(&block->list); + block->sec =3D rsec; + block->start_idx =3D rsec->num_relocs; + block->len =3D num_relocs; + + for (i =3D 0; i < num_relocs; i++) + block->relocs[i].block =3D block; + + rsec->num_relocs +=3D num_relocs; + list_add_tail(&block->list, &rsec->reloc_list); + + return block; +} + static int read_relocs(struct elf *elf) { unsigned long nr_reloc, max_reloc =3D 0; struct section *rsec; - struct reloc *reloc; - unsigned int symndx; - struct symbol *sym; int i; =20 if (!elf_alloc_hash(reloc, elf->num_relocs)) return -1; =20 list_for_each_entry(rsec, &elf->sections, list) { + struct reloc_block *block; + if (!is_reloc_sec(rsec)) continue; =20 @@ -956,15 +986,15 @@ static int read_relocs(struct elf *elf) rsec->base->rsec =3D rsec; =20 nr_reloc =3D 0; - rsec->relocs =3D calloc(sec_num_entries(rsec), sizeof(*reloc)); - if (!rsec->relocs) { - perror("calloc"); + block =3D alloc_reloc_block(rsec, sec_num_entries(rsec)); + if (!block) return -1; - } + for (i =3D 0; i < sec_num_entries(rsec); i++) { - reloc =3D &rsec->relocs[i]; + struct reloc *reloc =3D &block->relocs[i]; + struct symbol *sym; + unsigned int symndx; =20 - reloc->sec =3D rsec; symndx =3D reloc_sym(reloc); reloc->sym =3D sym =3D find_symbol_by_index(elf, symndx); if (!reloc->sym) { @@ -1100,6 +1130,7 @@ struct section *elf_create_section(struct elf *elf, c= onst char *name, memset(sec, 0, sizeof(*sec)); =20 INIT_LIST_HEAD(&sec->symbol_list); + INIT_LIST_HEAD(&sec->reloc_list); =20 s =3D elf_newscn(elf->elf); if (!s) { @@ -1170,6 +1201,7 @@ static struct section *elf_create_rela_section(struct= elf *elf, unsigned int reloc_nr) { struct section *rsec; + struct reloc_block *block; char *rsec_name; =20 rsec_name =3D malloc(strlen(sec->name) + strlen(".rela") + 1); @@ -1192,11 +1224,9 @@ static struct section *elf_create_rela_section(struc= t elf *elf, rsec->sh.sh_info =3D sec->idx; rsec->sh.sh_flags =3D SHF_INFO_LINK; =20 - rsec->relocs =3D calloc(sec_num_entries(rsec), sizeof(struct reloc)); - if (!rsec->relocs) { - perror("calloc"); + block =3D alloc_reloc_block(rsec, sec_num_entries(rsec)); + if (!block) return NULL; - } =20 sec->rsec =3D rsec; rsec->base =3D sec; @@ -1204,6 +1234,37 @@ static struct section *elf_create_rela_section(struc= t elf *elf, return rsec; } =20 +int elf_extend_rela_section(struct elf *elf, + struct section *rsec, + int add_relocs) +{ + int newnr =3D sec_num_entries(rsec) + add_relocs; + size_t oldsize =3D rsec->sh.sh_size; + size_t newsize =3D newnr * rsec->sh.sh_entsize; + void *buf; + struct reloc_block *block; + + buf =3D realloc(rsec->data->d_buf, newnr * rsec->sh.sh_entsize); + if (!buf) { + perror("realloc"); + return -1; + } + + memset(buf + oldsize, 0, newsize - oldsize); + + rsec->data->d_size =3D newsize; + rsec->data->d_buf =3D buf; + rsec->sh.sh_size =3D newsize; + + mark_sec_changed(elf, rsec, true); + + block =3D alloc_reloc_block(rsec, add_relocs); + if (!block) + return -1; + + return 0; +} + struct section *elf_create_section_pair(struct elf *elf, const char *name, size_t entsize, unsigned int nr, unsigned int reloc_nr) diff --git a/tools/objtool/include/objtool/elf.h b/tools/objtool/include/ob= jtool/elf.h index 9f71e988eca4..7851467f6878 100644 --- a/tools/objtool/include/objtool/elf.h +++ b/tools/objtool/include/objtool/elf.h @@ -43,7 +43,8 @@ struct section { char *name; int idx; bool _changed, text, rodata, noinstr, init, truncate; - struct reloc *relocs; + struct list_head reloc_list; + int num_relocs; }; =20 struct symbol { @@ -71,13 +72,23 @@ struct symbol { struct reloc *relocs; }; =20 +struct reloc_block; + struct reloc { struct elf_hash_node hash; - struct section *sec; + struct reloc_block *block; struct symbol *sym; struct reloc *sym_next_reloc; }; =20 +struct reloc_block { + struct list_head list; + struct section *sec; + int start_idx; + int len; + struct reloc relocs[0]; +}; + struct elf { Elf *elf; GElf_Ehdr ehdr; @@ -108,6 +119,11 @@ struct elf *elf_open_read(const char *name, int flags); =20 struct section *elf_create_section(struct elf *elf, const char *name, size_t entsize, unsigned int nr); + +int elf_extend_rela_section(struct elf *elf, + struct section *rsec, + int add_relocs); + struct section *elf_create_section_pair(struct elf *elf, const char *name, size_t entsize, unsigned int nr, unsigned int reloc_nr); @@ -197,12 +213,12 @@ static inline unsigned int sec_num_entries(struct sec= tion *sec) =20 static inline unsigned int reloc_idx(struct reloc *reloc) { - return reloc - reloc->sec->relocs; + return reloc->block->start_idx + (reloc - &reloc->block->relocs[0]); } =20 static inline void *reloc_rel(struct reloc *reloc) { - struct section *rsec =3D reloc->sec; + struct section *rsec =3D reloc->block->sec; =20 return rsec->data->d_buf + (reloc_idx(reloc) * rsec->sh.sh_entsize); } @@ -215,7 +231,7 @@ static inline bool is_32bit_reloc(struct reloc *reloc) * Elf64_Rel: 16 bytes * Elf64_Rela: 24 bytes */ - return reloc->sec->sh.sh_entsize < 16; + return reloc->block->sec->sh.sh_entsize < 16; } =20 #define __get_reloc_field(reloc, field) \ @@ -241,7 +257,7 @@ static inline u64 reloc_offset(struct reloc *reloc) static inline void set_reloc_offset(struct elf *elf, struct reloc *reloc, = u64 offset) { __set_reloc_field(reloc, r_offset, offset); - mark_sec_changed(elf, reloc->sec, true); + mark_sec_changed(elf, reloc->block->sec, true); } =20 static inline s64 reloc_addend(struct reloc *reloc) @@ -252,7 +268,7 @@ static inline s64 reloc_addend(struct reloc *reloc) static inline void set_reloc_addend(struct elf *elf, struct reloc *reloc, = s64 addend) { __set_reloc_field(reloc, r_addend, addend); - mark_sec_changed(elf, reloc->sec, true); + mark_sec_changed(elf, reloc->block->sec, true); } =20 =20 @@ -282,7 +298,7 @@ static inline void set_reloc_sym(struct elf *elf, struc= t reloc *reloc, unsigned =20 __set_reloc_field(reloc, r_info, info); =20 - mark_sec_changed(elf, reloc->sec, true); + mark_sec_changed(elf, reloc->block->sec, true); } static inline void set_reloc_type(struct elf *elf, struct reloc *reloc, un= signed int type) { @@ -292,7 +308,46 @@ static inline void set_reloc_type(struct elf *elf, str= uct reloc *reloc, unsigned =20 __set_reloc_field(reloc, r_info, info); =20 - mark_sec_changed(elf, reloc->sec, true); + mark_sec_changed(elf, reloc->block->sec, true); +} + +static inline struct reloc *get_reloc_by_index(struct section *rsec, int i= dx) +{ + struct reloc_block *block; + + list_for_each_entry(block, &rsec->reloc_list, list) { + if (idx < block->len) + return &block->relocs[idx]; + idx -=3D block->len; + } + + return NULL; +} + +static inline struct reloc *first_reloc(struct section *sec) +{ + struct reloc_block *block; + + if (list_empty(&sec->reloc_list)) + return NULL; + + block =3D list_first_entry(&sec->reloc_list, struct reloc_block, list); + return &block->relocs[0]; +} + +static inline struct reloc *next_reloc(struct reloc *reloc) +{ + struct reloc_block *block =3D reloc->block; + + reloc++; + if (reloc < &block->relocs[block->len]) + return reloc; + + if (list_is_last(&block->list, &block->sec->reloc_list)) + return NULL; + + block =3D list_next_entry(block, list); + return &block->relocs[0]; } =20 #define for_each_sec(file, sec) \ @@ -308,15 +363,10 @@ static inline void set_reloc_type(struct elf *elf, st= ruct reloc *reloc, unsigned sec_for_each_sym(__sec, sym) =20 #define for_each_reloc(rsec, reloc) \ - for (int __i =3D 0, __fake =3D 1; __fake; __fake =3D 0) \ - for (reloc =3D rsec->relocs; \ - __i < sec_num_entries(rsec); \ - __i++, reloc++) + for (reloc =3D first_reloc(rsec); reloc; reloc =3D next_reloc(reloc)) =20 #define for_each_reloc_from(rsec, reloc) \ - for (int __i =3D reloc_idx(reloc); \ - __i < sec_num_entries(rsec); \ - __i++, reloc++) + for (; reloc; reloc =3D next_reloc(reloc)) =20 #define OFFSET_STRIDE_BITS 4 #define OFFSET_STRIDE (1UL << OFFSET_STRIDE_BITS) @@ -344,7 +394,7 @@ static inline u32 sec_offset_hash(struct section *sec, = unsigned long offset) =20 static inline u32 reloc_hash(struct reloc *reloc) { - return sec_offset_hash(reloc->sec, reloc_offset(reloc)); + return sec_offset_hash(reloc->block->sec, reloc_offset(reloc)); } =20 #endif /* _OBJTOOL_ELF_H */ --=20 2.41.0 From nobody Tue Dec 30 12:50:30 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C90C8C54FB9 for ; Wed, 15 Nov 2023 17:37:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232487AbjKORhw (ORCPT ); Wed, 15 Nov 2023 12:37:52 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60836 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232682AbjKORhk (ORCPT ); Wed, 15 Nov 2023 12:37:40 -0500 Received: from mail-qk1-x72f.google.com (mail-qk1-x72f.google.com [IPv6:2607:f8b0:4864:20::72f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8FC0BD55 for ; Wed, 15 Nov 2023 09:37:35 -0800 (PST) Received: by mail-qk1-x72f.google.com with SMTP id af79cd13be357-7788fb06997so459443785a.0 for ; Wed, 15 Nov 2023 09:37:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700069854; x=1700674654; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=X5OOH8+riyDmJDIWiL4csbYtTgdsMwlIshYQZoHixI8=; b=GTSFNzq5QDqxZwoeNU7Y2lzMYRwLxlmEMqFpDzWCYyjk2CK6dH25FS+Yu3Lpw99GHi fmnUXTaC5OI+W6YQA92fJQdLxE2Zw+ugXkaAThKDHy7zHWogYerr7blujD0q59qZ9FNC eQw+7el6A86CfiQFuMdh5/GwMPaVtx4ZKUnQALcRpn9EKrTDrhQzEl6roWLYMXv9Dg0q kTq88XaQFRnL/Q1Sizl4z/ANCqX+L5q0KWMM3GabBUEh1ShpUBIXxmTxc5ElpITcWLis wAi9y2QQKehx36kYrb0bTyXEH5hpBVuVUBbfk8d8pbl6AMNIoqWcJJQO8OBnALDG01+u PE0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700069854; x=1700674654; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=X5OOH8+riyDmJDIWiL4csbYtTgdsMwlIshYQZoHixI8=; b=unuW8RriadVAIgIYPKPjBDPplCN5a6MDXn3I/qRTxg62Nsr+He1wpYLAcCgIaNeJIU OU1xPaDQjk22uPh59clVfP4Iycuss2lkksxYH8e3HXAHVP1kACk3x+bF/rVQyYX2YR3w L5EOWwT4EU5BlDqqEcov1FOMkTE60VY0XtH2Bp8ITIbqDIpSVUuPAEbHv37NV8tFZCVM DjZJDltz5A4NbD8G5shp8YboyWuzW0B7wEk7lTZKNdpGNrMtUE0GUuqQISCcLdXZhjLk Z/+OKUrHVzXlGtTl+3v5tv6XsW0YpAkrbPu0BrPFcSvMVvlm3NoBFSGeKqEpeYds55DR TVcg== X-Gm-Message-State: AOJu0Yy6o1vZUwYcFOwAiUFTkRZg67tqxuGITjjAO2hHptFHS1m1T2Cu pKROyIQk9oP9tpt2sLTXHA2Dztte1Q== X-Google-Smtp-Source: AGHT+IG0MuLZhLGHGwLKyltKxNLdgSLExQLasiB5NLof/2T0xpB1+r3+UKeHDCxFOXH+CbDO9huZgQ== X-Received: by 2002:a05:620a:20d5:b0:778:8b53:cda2 with SMTP id f21-20020a05620a20d500b007788b53cda2mr6267023qka.23.1700069853706; Wed, 15 Nov 2023 09:37:33 -0800 (PST) Received: from citadel.lan ([2600:6c4a:4d3f:6d5c::1019]) by smtp.gmail.com with ESMTPSA id x20-20020a05620a0b5400b0077bda014d8esm3260099qkg.87.2023.11.15.09.37.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Nov 2023 09:37:33 -0800 (PST) From: Brian Gerst To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: Ingo Molnar , Thomas Gleixner , Borislav Petkov , "H . Peter Anvin" , Peter Zijlstra , Uros Bizjak , David.Laight@aculab.com, Brian Gerst Subject: [PATCH v3 07/14] objtool: Convert fixed location stack protector accesses Date: Wed, 15 Nov 2023 12:37:01 -0500 Message-ID: <20231115173708.108316-8-brgerst@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231115173708.108316-1-brgerst@gmail.com> References: <20231115173708.108316-1-brgerst@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Older versions of GCC fixed the location of the stack protector canary at %gs:40. Use objtool to convert these accesses to normal percpu accesses to __stack_chk_guard. Signed-off-by: Brian Gerst --- arch/x86/Kconfig | 4 ++ scripts/Makefile.lib | 2 + tools/objtool/arch/x86/decode.c | 46 +++++++++++++ tools/objtool/arch/x86/special.c | 88 +++++++++++++++++++++++++ tools/objtool/builtin-check.c | 9 ++- tools/objtool/check.c | 12 ++++ tools/objtool/elf.c | 34 ++++++++-- tools/objtool/include/objtool/arch.h | 3 + tools/objtool/include/objtool/builtin.h | 2 + tools/objtool/include/objtool/elf.h | 6 ++ 10 files changed, 201 insertions(+), 5 deletions(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index a1d2f7fe42bb..6cee46127fd2 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -416,6 +416,10 @@ config CC_HAS_SANE_STACKPROTECTOR We have to make sure stack protector is unconditionally disabled if the compiler does not allow control of the segment and symbol. =20 +config STACKPROTECTOR_OBJTOOL + bool + default n + menu "Processor type and features" =20 config SMP diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib index 68d0134bdbf9..a15dc76c19d0 100644 --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -255,6 +255,8 @@ objtool :=3D $(objtree)/tools/objtool/objtool objtool-args-$(CONFIG_HAVE_JUMP_LABEL_HACK) +=3D --hacks=3Djump_label objtool-args-$(CONFIG_HAVE_NOINSTR_HACK) +=3D --hacks=3Dnoinstr objtool-args-$(CONFIG_CALL_DEPTH_TRACKING) +=3D --hacks=3Dskylake +objtool-args-$(CONFIG_STACKPROTECTOR_OBJTOOL) +=3D --hacks=3Dstackprotect= or +objtool-args-$(CONFIG_SMP) +=3D --smp objtool-args-$(CONFIG_X86_KERNEL_IBT) +=3D --ibt objtool-args-$(CONFIG_FINEIBT) +=3D --cfi objtool-args-$(CONFIG_FTRACE_MCOUNT_USE_OBJTOOL) +=3D --mcount diff --git a/tools/objtool/arch/x86/decode.c b/tools/objtool/arch/x86/decod= e.c index e327cd827135..53f3d7323259 100644 --- a/tools/objtool/arch/x86/decode.c +++ b/tools/objtool/arch/x86/decode.c @@ -144,6 +144,18 @@ static bool has_notrack_prefix(struct insn *insn) return false; } =20 +static bool has_gs_prefix(struct insn *insn) +{ + int i; + + for (i =3D 0; i < insn->prefixes.nbytes; i++) { + if (insn->prefixes.bytes[i] =3D=3D 0x65) + return true; + } + + return false; +} + int arch_decode_instruction(struct objtool_file *file, const struct sectio= n *sec, unsigned long offset, unsigned int maxlen, struct instruction *insn) @@ -408,10 +420,44 @@ int arch_decode_instruction(struct objtool_file *file= , const struct section *sec =20 break; =20 + case 0x2b: + case 0x3b: + case 0x39: + if (!rex_w) + break; + + /* sub %gs:0x28, reg */ + /* cmp %gs:0x28, reg */ + /* cmp reg, %gs:0x28 */ + if (has_gs_prefix(&ins) && + modrm_mod =3D=3D 0 && + modrm_rm =3D=3D 4 && + sib_index =3D=3D 4 && + sib_base =3D=3D 5 && + ins.displacement.value =3D=3D 0x28) + { + insn->type =3D INSN_STACKPROTECTOR; + break; + } + + break; + case 0x8b: if (!rex_w) break; =20 + /* mov %gs:0x28, reg */ + if (has_gs_prefix(&ins) && + modrm_mod =3D=3D 0 && + modrm_rm =3D=3D 4 && + sib_index =3D=3D 4 && + sib_base =3D=3D 5 && + ins.displacement.value =3D=3D 0x28) + { + insn->type =3D INSN_STACKPROTECTOR; + break; + } + if (rm_is_mem(CFI_BP)) { =20 /* mov disp(%rbp), reg */ diff --git a/tools/objtool/arch/x86/special.c b/tools/objtool/arch/x86/spec= ial.c index 29e949579ede..47c17452c899 100644 --- a/tools/objtool/arch/x86/special.c +++ b/tools/objtool/arch/x86/special.c @@ -3,6 +3,9 @@ =20 #include #include +#include +#include +#include =20 #define X86_FEATURE_POPCNT (4 * 32 + 23) #define X86_FEATURE_SMAP (9 * 32 + 20) @@ -137,3 +140,88 @@ struct reloc *arch_find_switch_table(struct objtool_fi= le *file, =20 return rodata_reloc; } + +/* + * Convert op %gs:0x28, reg -> op __stack_chk_guard(%rip), %reg + * op is mov, sub, or cmp. + */ +int arch_hack_stackprotector(struct objtool_file *file) +{ + struct section *sec; + struct symbol *__stack_chk_guard; + struct instruction *insn; + + int i; + + __stack_chk_guard =3D find_symbol_by_name(file->elf, "__stack_chk_guard"); + + for_each_sec(file, sec) { + int count =3D 0; + int idx; + struct section *rsec =3D sec->rsec; + + sec_for_each_insn(file, sec, insn) { + if (insn->type =3D=3D INSN_STACKPROTECTOR) + count++; + } + + if (!count) + continue; + + if (!__stack_chk_guard) + __stack_chk_guard =3D elf_create_undef_symbol(file->elf, "__stack_chk_g= uard"); + + if (!sec->rsec) { + idx =3D 0; + rsec =3D sec->rsec =3D elf_create_rela_section(file->elf, sec, count); + } else { + idx =3D sec_num_entries(rsec); + if (elf_extend_rela_section(file->elf, rsec, count)) + return -1; + } + + sec_for_each_insn(file, sec, insn) { + unsigned char *data =3D insn->sec->data->d_buf + insn->offset; + + if (insn->type !=3D INSN_STACKPROTECTOR) + continue; + + if (insn->len !=3D 9) + goto invalid; + + /* Remove GS prefix if !SMP */ + if (data[0] !=3D 0x65) + goto invalid; + if (!opts.smp) + data[0] =3D 0x90; + + /* Set Mod=3D00, R/M=3D101. Preserve Reg */ + data[3] =3D (data[3] & 0x38) | 5; + + /* Displacement 0 */ + data[4] =3D 0; + data[5] =3D 0; + data[6] =3D 0; + data[7] =3D 0; + + /* Pad with NOP */ + data[8] =3D 0x90; + + mark_sec_changed(file->elf, insn->sec, true); + + if (!elf_init_reloc_data_sym(file->elf, insn->sec, insn->offset + 4, id= x++, __stack_chk_guard, -4)) + return -1; + + continue; + +invalid: + fprintf(stderr, "Invalid stackprotector instruction at %s+0x%lx: ", ins= n->sec->name, insn->offset); + for (i =3D 0; i < insn->len; i++) + fprintf(stderr, "%02x ", data[i]); + fprintf(stderr, "\n"); + return -1; + } + } + + return 0; +} diff --git a/tools/objtool/builtin-check.c b/tools/objtool/builtin-check.c index 5e21cfb7661d..0ab2efb45c0e 100644 --- a/tools/objtool/builtin-check.c +++ b/tools/objtool/builtin-check.c @@ -62,12 +62,17 @@ static int parse_hacks(const struct option *opt, const = char *str, int unset) found =3D true; } =20 + if (!str || strstr(str, "stackprotector")) { + opts.hack_stackprotector =3D true; + found =3D true; + } + return found ? 0 : -1; } =20 static const struct option check_options[] =3D { OPT_GROUP("Actions:"), - OPT_CALLBACK_OPTARG('h', "hacks", NULL, NULL, "jump_label,noinstr,skylake= ", "patch toolchain bugs/limitations", parse_hacks), + OPT_CALLBACK_OPTARG('h', "hacks", NULL, NULL, "jump_label,noinstr,skylake= ,stackprotector", "patch toolchain bugs/limitations", parse_hacks), OPT_BOOLEAN('i', "ibt", &opts.ibt, "validate and annotate IBT"), OPT_BOOLEAN('m', "mcount", &opts.mcount, "annotate mcount/fentry calls fo= r ftrace"), OPT_BOOLEAN('n', "noinstr", &opts.noinstr, "validate noinstr rules"), @@ -94,6 +99,7 @@ static const struct option check_options[] =3D { OPT_BOOLEAN(0, "sec-address", &opts.sec_address, "print section addresses= in warnings"), OPT_BOOLEAN(0, "stats", &opts.stats, "print statistics"), OPT_BOOLEAN('v', "verbose", &opts.verbose, "verbose warnings"), + OPT_BOOLEAN(0, "smp", &opts.smp, "building an SMP kernel"), =20 OPT_END(), }; @@ -133,6 +139,7 @@ static bool opts_valid(void) { if (opts.hack_jump_label || opts.hack_noinstr || + opts.hack_stackprotector || opts.ibt || opts.mcount || opts.noinstr || diff --git a/tools/objtool/check.c b/tools/objtool/check.c index ac304140c395..57f080ca0195 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -1315,6 +1315,11 @@ __weak bool arch_is_embedded_insn(struct symbol *sym) return false; } =20 +__weak int arch_hack_stackprotector(struct objtool_file *file) +{ + return 0; +} + static struct reloc *insn_reloc(struct objtool_file *file, struct instruct= ion *insn) { struct reloc *reloc; @@ -4812,6 +4817,13 @@ int check(struct objtool_file *file) warnings +=3D ret; } =20 + if (opts.hack_stackprotector) { + ret =3D arch_hack_stackprotector(file); + if (ret < 0) + goto out; + warnings +=3D ret; + } + free_insns(file); =20 if (opts.verbose) diff --git a/tools/objtool/elf.c b/tools/objtool/elf.c index cfb970727c8a..2af99b2a054c 100644 --- a/tools/objtool/elf.c +++ b/tools/objtool/elf.c @@ -846,6 +846,32 @@ elf_create_prefix_symbol(struct elf *elf, struct symbo= l *orig, long size) return sym; } =20 +struct symbol * +elf_create_undef_symbol(struct elf *elf, const char *sym_name) +{ + struct symbol *sym =3D calloc(1, sizeof(*sym)); + char *name =3D strdup(sym_name); + + if (!sym || !name) { + perror("malloc"); + return NULL; + } + + sym->name =3D name; + sym->sec =3D find_section_by_index(elf, 0); + + sym->sym.st_name =3D elf_add_string(elf, NULL, name); + sym->sym.st_info =3D GELF_ST_INFO(STB_GLOBAL, STT_NOTYPE); + sym->sym.st_value =3D 0; + sym->sym.st_size =3D 0; + + sym =3D __elf_create_symbol(elf, sym); + if (sym) + elf_add_symbol(elf, sym); + + return sym; +} + static struct reloc *elf_init_reloc(struct elf *elf, struct section *rsec, unsigned int reloc_idx, unsigned long offset, struct symbol *sym, @@ -924,7 +950,7 @@ struct reloc *elf_init_reloc_data_sym(struct elf *elf, = struct section *sec, struct symbol *sym, s64 addend) { - if (sym->sec && (sec->sh.sh_flags & SHF_EXECINSTR)) { + if (sym->sec && (sym->sec->sh.sh_flags & SHF_EXECINSTR)) { WARN("bad call to %s() for text symbol %s", __func__, sym->name); return NULL; @@ -1196,9 +1222,9 @@ struct section *elf_create_section(struct elf *elf, c= onst char *name, return sec; } =20 -static struct section *elf_create_rela_section(struct elf *elf, - struct section *sec, - unsigned int reloc_nr) +struct section *elf_create_rela_section(struct elf *elf, + struct section *sec, + unsigned int reloc_nr) { struct section *rsec; struct reloc_block *block; diff --git a/tools/objtool/include/objtool/arch.h b/tools/objtool/include/o= bjtool/arch.h index 0b303eba660e..c60fec88b3af 100644 --- a/tools/objtool/include/objtool/arch.h +++ b/tools/objtool/include/objtool/arch.h @@ -28,6 +28,7 @@ enum insn_type { INSN_CLD, INSN_TRAP, INSN_ENDBR, + INSN_STACKPROTECTOR, INSN_OTHER, }; =20 @@ -96,4 +97,6 @@ int arch_rewrite_retpolines(struct objtool_file *file); =20 bool arch_pc_relative_reloc(struct reloc *reloc); =20 +int arch_hack_stackprotector(struct objtool_file *file); + #endif /* _ARCH_H */ diff --git a/tools/objtool/include/objtool/builtin.h b/tools/objtool/includ= e/objtool/builtin.h index fcca6662c8b4..5085d3135e6b 100644 --- a/tools/objtool/include/objtool/builtin.h +++ b/tools/objtool/include/objtool/builtin.h @@ -13,6 +13,7 @@ struct opts { bool hack_jump_label; bool hack_noinstr; bool hack_skylake; + bool hack_stackprotector; bool ibt; bool mcount; bool noinstr; @@ -38,6 +39,7 @@ struct opts { bool sec_address; bool stats; bool verbose; + bool smp; }; =20 extern struct opts opts; diff --git a/tools/objtool/include/objtool/elf.h b/tools/objtool/include/ob= jtool/elf.h index 7851467f6878..b5eec9e4a65d 100644 --- a/tools/objtool/include/objtool/elf.h +++ b/tools/objtool/include/objtool/elf.h @@ -120,6 +120,10 @@ struct elf *elf_open_read(const char *name, int flags); struct section *elf_create_section(struct elf *elf, const char *name, size_t entsize, unsigned int nr); =20 +struct section *elf_create_rela_section(struct elf *elf, + struct section *sec, + unsigned int reloc_nr); + int elf_extend_rela_section(struct elf *elf, struct section *rsec, int add_relocs); @@ -130,6 +134,8 @@ struct section *elf_create_section_pair(struct elf *elf= , const char *name, =20 struct symbol *elf_create_prefix_symbol(struct elf *elf, struct symbol *or= ig, long size); =20 +struct symbol *elf_create_undef_symbol(struct elf *elf, const char *sym_na= me); + struct reloc *elf_init_reloc_text_sym(struct elf *elf, struct section *sec, unsigned long offset, unsigned int reloc_idx, --=20 2.41.0 From nobody Tue Dec 30 12:50:30 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4B1F4C2BB3F for ; Wed, 15 Nov 2023 17:37:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231668AbjKORh4 (ORCPT ); Wed, 15 Nov 2023 12:37:56 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60880 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232718AbjKORhl (ORCPT ); Wed, 15 Nov 2023 12:37:41 -0500 Received: from mail-qk1-x729.google.com (mail-qk1-x729.google.com [IPv6:2607:f8b0:4864:20::729]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 55161D48 for ; Wed, 15 Nov 2023 09:37:37 -0800 (PST) Received: by mail-qk1-x729.google.com with SMTP id af79cd13be357-7781bc3783fso459166285a.1 for ; Wed, 15 Nov 2023 09:37:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700069855; x=1700674655; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=gtJ0rnAMMvu0qTTk2sGNJank9S4dE7egD/BT6LvuAOM=; b=gZeJ++nom6xVXbXIMXz0VIkKa1sAncCLIlYj5JtTC6ePRGw1xXyrHFGZeR57E4HkYf LXiO7bo8V8oDvcthsUmZjxHf6nZvQfEYIQVEYAiIyfGerfrKT8smaru22mDbg0wD0Jt/ GVlyaNs7LPjluOGEjcG/4+2UTtn6BN77TkJMemuJIFzHQbm2D1IV/LC0t3prAsaUOy5Y nW/hNZY5hRyG+dFTylA+y/Jb902iDMAWa+06XF7qb73UgwwOv6fzMA4bwwghxTNZvDWb hq3FTPYA6ixXGgnbXEhnwrzQv07ZEQCX6MJ9oXZOad4m/W/AhKTikbbYeRDKmWeRA9Qa 7EXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700069855; x=1700674655; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=gtJ0rnAMMvu0qTTk2sGNJank9S4dE7egD/BT6LvuAOM=; b=rgRM/B2JDIRYdUsIyA3MDPyGKLt+hHQwy8nWZ4/BfUvDg6NU4tvDU+VkS9UwQX7Eh6 vW/tJ9e2Mc31494eEab2q3TMqetODSEDymBuc5ogZoPJpV24orrWSMZSqnINqt+/WhoU hhCqKL0gI+wQKPdwrNFC1Yz37kuDkKUMDROj//vGVTG+dAm2eVOjz5O37/yuoa51NiTz r9/WU8BW+PuqKNrAJQDhfoECA3NllpfYX1Gb5YGOHXMS9Vysrb0IWeCu3tFO0oUU4O8C IXVw1IVKKB/qxnBH5CYiCxP6LAQidBDeHDLsFKAlBctwqcdiNu3yxKUUpEeoWCh4Thy/ C19A== X-Gm-Message-State: AOJu0YwDpYXAgZKBB48PqzGVVYT8B8dLSw8JdxTXr08xEaxgZ5atKo2+ h/CCR9ncJp28BcTaRQ6TxMWQGRF6Bw== X-Google-Smtp-Source: AGHT+IGoyEoIHIylVJJPlxpgTBEFasM2YeujOc6F2/G6+EUfcXBIXlq9zuNoBlNn/MVBwRskr5jCng== X-Received: by 2002:a05:620a:29c8:b0:76f:1d46:4a9f with SMTP id s8-20020a05620a29c800b0076f1d464a9fmr5982220qkp.4.1700069855299; Wed, 15 Nov 2023 09:37:35 -0800 (PST) Received: from citadel.lan ([2600:6c4a:4d3f:6d5c::1019]) by smtp.gmail.com with ESMTPSA id x20-20020a05620a0b5400b0077bda014d8esm3260099qkg.87.2023.11.15.09.37.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Nov 2023 09:37:34 -0800 (PST) From: Brian Gerst To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: Ingo Molnar , Thomas Gleixner , Borislav Petkov , "H . Peter Anvin" , Peter Zijlstra , Uros Bizjak , David.Laight@aculab.com, Brian Gerst Subject: [PATCH v3 08/14] x86/stackprotector/64: Convert to normal percpu variable Date: Wed, 15 Nov 2023 12:37:02 -0500 Message-ID: <20231115173708.108316-9-brgerst@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231115173708.108316-1-brgerst@gmail.com> References: <20231115173708.108316-1-brgerst@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Older versions of GCC fixed the location of the stack protector canary at %gs:40. This constraint forced the percpu section to be linked at virtual address 0 so that the canary could be the first data object in the percpu section. Supporting the zero-based percpu section requires additional code to handle relocations for RIP-relative references to percpu data, extra complexity to kallsyms, and workarounds for linker bugs due to the use of absolute symbols. Use compiler options to redefine the stack protector location if supported, otherwise use objtool. This will remove the contraint that the percpu section must be zero-based. Signed-off-by: Brian Gerst --- arch/x86/Kconfig | 11 ++++---- arch/x86/Makefile | 21 ++++++++++------ arch/x86/entry/entry_64.S | 2 +- arch/x86/include/asm/processor.h | 16 ++---------- arch/x86/include/asm/stackprotector.h | 36 ++++----------------------- arch/x86/kernel/asm-offsets_64.c | 6 ----- arch/x86/kernel/cpu/common.c | 4 +-- arch/x86/kernel/head_64.S | 3 +-- arch/x86/xen/xen-head.S | 3 +-- 9 files changed, 30 insertions(+), 72 deletions(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 6cee46127fd2..83404b741c0a 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -266,7 +266,7 @@ config X86 select HAVE_FUNCTION_ARG_ACCESS_API select HAVE_SETUP_PER_CPU_AREA select HAVE_SOFTIRQ_ON_OWN_STACK - select HAVE_STACKPROTECTOR if CC_HAS_SANE_STACKPROTECTOR + select HAVE_STACKPROTECTOR if X86_64 || CC_HAS_SANE_STACKPROTECTOR select HAVE_STACK_VALIDATION if HAVE_OBJTOOL select HAVE_STATIC_CALL select HAVE_STATIC_CALL_INLINE if HAVE_OBJTOOL @@ -410,15 +410,14 @@ config PGTABLE_LEVELS =20 config CC_HAS_SANE_STACKPROTECTOR bool - default y if 64BIT + default $(cc-option,-mstack-protector-guard-reg=3Dgs -mstack-protector-gu= ard-symbol=3D__stack_chk_guard) if 64BIT default $(cc-option,-mstack-protector-guard-reg=3Dfs -mstack-protector-gu= ard-symbol=3D__stack_chk_guard) - help - We have to make sure stack protector is unconditionally disabled if - the compiler does not allow control of the segment and symbol. =20 config STACKPROTECTOR_OBJTOOL bool - default n + depends on X86_64 && STACKPROTECTOR + default !CC_HAS_SANE_STACKPROTECTOR + prompt "Debug objtool stack protector conversion" if CC_HAS_SANE_STACKPRO= TECTOR && DEBUG_KERNEL =20 menu "Processor type and features" =20 diff --git a/arch/x86/Makefile b/arch/x86/Makefile index 1a068de12a56..06a79361e88f 100644 --- a/arch/x86/Makefile +++ b/arch/x86/Makefile @@ -112,13 +112,7 @@ ifeq ($(CONFIG_X86_32),y) # temporary until string.h is fixed KBUILD_CFLAGS +=3D -ffreestanding =20 - ifeq ($(CONFIG_STACKPROTECTOR),y) - ifeq ($(CONFIG_SMP),y) - KBUILD_CFLAGS +=3D -mstack-protector-guard-reg=3Dfs -mstack-protector-g= uard-symbol=3D__stack_chk_guard - else - KBUILD_CFLAGS +=3D -mstack-protector-guard=3Dglobal - endif - endif + percpu_seg :=3D fs else BITS :=3D 64 UTS_MACHINE :=3D x86_64 @@ -168,6 +162,19 @@ else KBUILD_CFLAGS +=3D -mcmodel=3Dkernel KBUILD_RUSTFLAGS +=3D -Cno-redzone=3Dy KBUILD_RUSTFLAGS +=3D -Ccode-model=3Dkernel + + percpu_seg :=3D gs +endif + +ifeq ($(CONFIG_STACKPROTECTOR),y) + ifneq ($(CONFIG_STACKPROTECTOR_OBJTOOL),y) + ifeq ($(CONFIG_SMP),y) + KBUILD_CFLAGS +=3D -mstack-protector-guard-reg=3D$(percpu_seg) + KBUILD_CFLAGS +=3D -mstack-protector-guard-symbol=3D__stack_chk_guard + else + KBUILD_CFLAGS +=3D -mstack-protector-guard=3Dglobal + endif + endif endif =20 # diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index 1a88ad8a7b48..cddcc236aaae 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -190,7 +190,7 @@ SYM_FUNC_START(__switch_to_asm) =20 #ifdef CONFIG_STACKPROTECTOR movq TASK_stack_canary(%rsi), %rbx - movq %rbx, PER_CPU_VAR(fixed_percpu_data + FIXED_stack_canary) + movq %rbx, PER_CPU_VAR(__stack_chk_guard) #endif =20 /* diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/proces= sor.h index 061aa86b4662..3ee091225904 100644 --- a/arch/x86/include/asm/processor.h +++ b/arch/x86/include/asm/processor.h @@ -394,16 +394,8 @@ struct irq_stack { =20 #ifdef CONFIG_X86_64 struct fixed_percpu_data { - /* - * GCC hardcodes the stack canary as %gs:40. Since the - * irq_stack is the object at %gs:0, we reserve the bottom - * 48 bytes of the irq stack for the canary. - * - * Once we are willing to require -mstack-protector-guard-symbol=3D - * support for x86_64 stackprotector, we can get rid of this. - */ char gs_base[40]; - unsigned long stack_canary; + unsigned long reserved; }; =20 DECLARE_PER_CPU_FIRST(struct fixed_percpu_data, fixed_percpu_data) __visib= le; @@ -418,11 +410,7 @@ extern asmlinkage void entry_SYSCALL32_ignore(void); =20 /* Save actual FS/GS selectors and bases to current->thread */ void current_save_fsgs(void); -#else /* X86_64 */ -#ifdef CONFIG_STACKPROTECTOR -DECLARE_PER_CPU(unsigned long, __stack_chk_guard); -#endif -#endif /* !X86_64 */ +#endif /* X86_64 */ =20 struct perf_event; =20 diff --git a/arch/x86/include/asm/stackprotector.h b/arch/x86/include/asm/s= tackprotector.h index 00473a650f51..d43fb589fcf6 100644 --- a/arch/x86/include/asm/stackprotector.h +++ b/arch/x86/include/asm/stackprotector.h @@ -2,26 +2,10 @@ /* * GCC stack protector support. * - * Stack protector works by putting predefined pattern at the start of + * Stack protector works by putting a predefined pattern at the start of * the stack frame and verifying that it hasn't been overwritten when - * returning from the function. The pattern is called stack canary - * and unfortunately gcc historically required it to be at a fixed offset - * from the percpu segment base. On x86_64, the offset is 40 bytes. - * - * The same segment is shared by percpu area and stack canary. On - * x86_64, percpu symbols are zero based and %gs (64-bit) points to the - * base of percpu area. The first occupant of the percpu area is always - * fixed_percpu_data which contains stack_canary at the appropriate - * offset. On x86_32, the stack canary is just a regular percpu - * variable. - * - * Putting percpu data in %fs on 32-bit is a minor optimization compared to - * using %gs. Since 32-bit userspace normally has %fs =3D=3D 0, we are li= kely - * to load 0 into %fs on exit to usermode, whereas with percpu data in - * %gs, we are likely to load a non-null %gs on return to user mode. - * - * Once we are willing to require GCC 8.1 or better for 64-bit stackprotec= tor - * support, we can remove some of this complexity. + * returning from the function. The pattern is called the stack canary + * and is a unique value for each task. */ =20 #ifndef _ASM_STACKPROTECTOR_H @@ -36,6 +20,8 @@ =20 #include =20 +DECLARE_PER_CPU(unsigned long, __stack_chk_guard); + /* * Initialize the stackprotector canary value. * @@ -51,25 +37,13 @@ static __always_inline void boot_init_stack_canary(void) { unsigned long canary =3D get_random_canary(); =20 -#ifdef CONFIG_X86_64 - BUILD_BUG_ON(offsetof(struct fixed_percpu_data, stack_canary) !=3D 40); -#endif - current->stack_canary =3D canary; -#ifdef CONFIG_X86_64 - this_cpu_write(fixed_percpu_data.stack_canary, canary); -#else this_cpu_write(__stack_chk_guard, canary); -#endif } =20 static inline void cpu_init_stack_canary(int cpu, struct task_struct *idle) { -#ifdef CONFIG_X86_64 - per_cpu(fixed_percpu_data.stack_canary, cpu) =3D idle->stack_canary; -#else per_cpu(__stack_chk_guard, cpu) =3D idle->stack_canary; -#endif } =20 #else /* STACKPROTECTOR */ diff --git a/arch/x86/kernel/asm-offsets_64.c b/arch/x86/kernel/asm-offsets= _64.c index bb65371ea9df..590b6cd0eac0 100644 --- a/arch/x86/kernel/asm-offsets_64.c +++ b/arch/x86/kernel/asm-offsets_64.c @@ -54,11 +54,5 @@ int main(void) BLANK(); #undef ENTRY =20 - BLANK(); - -#ifdef CONFIG_STACKPROTECTOR - OFFSET(FIXED_stack_canary, fixed_percpu_data, stack_canary); - BLANK(); -#endif return 0; } diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 4d4b87c6885d..b5b1d95b1399 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -2111,15 +2111,13 @@ void syscall_init(void) X86_EFLAGS_AC|X86_EFLAGS_ID); } =20 -#else /* CONFIG_X86_64 */ +#endif /* CONFIG_X86_64 */ =20 #ifdef CONFIG_STACKPROTECTOR DEFINE_PER_CPU(unsigned long, __stack_chk_guard); EXPORT_PER_CPU_SYMBOL(__stack_chk_guard); #endif =20 -#endif /* CONFIG_X86_64 */ - /* * Clear all 6 debug registers: */ diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index 3dcabbc49149..0d94d2a091fe 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -345,8 +345,7 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_G= LOBAL) =20 /* Set up %gs. * - * The base of %gs always points to fixed_percpu_data. If the - * stack protector canary is enabled, it is located at %gs:40. + * The base of %gs always points to fixed_percpu_data. * Note that, on SMP, the boot cpu uses init data section until * the per cpu areas are set up. */ diff --git a/arch/x86/xen/xen-head.S b/arch/x86/xen/xen-head.S index a0ea285878db..30f27e757354 100644 --- a/arch/x86/xen/xen-head.S +++ b/arch/x86/xen/xen-head.S @@ -53,8 +53,7 @@ SYM_CODE_START(startup_xen) =20 /* Set up %gs. * - * The base of %gs always points to fixed_percpu_data. If the - * stack protector canary is enabled, it is located at %gs:40. + * The base of %gs always points to fixed_percpu_data. * Note that, on SMP, the boot cpu uses init data section until * the per cpu areas are set up. */ --=20 2.41.0 From nobody Tue Dec 30 12:50:30 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3FDA2C2BB3F for ; Wed, 15 Nov 2023 17:37:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232734AbjKORh7 (ORCPT ); Wed, 15 Nov 2023 12:37:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60730 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232740AbjKORhn (ORCPT ); Wed, 15 Nov 2023 12:37:43 -0500 Received: from mail-qk1-x72d.google.com (mail-qk1-x72d.google.com [IPv6:2607:f8b0:4864:20::72d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B3BB683 for ; Wed, 15 Nov 2023 09:37:38 -0800 (PST) Received: by mail-qk1-x72d.google.com with SMTP id af79cd13be357-7789a4c01ddso456314085a.1 for ; Wed, 15 Nov 2023 09:37:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700069857; x=1700674657; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=pxm7rj1nNs1AxdmMRh1p1RVuTse+b2SlzLkV0LVvDBU=; b=iDAvRoTyGuyIGDfa3wfdJU3vhm8OjtJjhH8gZTj6uZXuZ5Z/C6cojYYCyjEObu243b rzt+Ay01W9KuZm4nl/iQCymlIuCtYrjx6aj+Z75kOWzquRRlcxusagEMa8+Yyodh1e/N O7XrhGxtjkStBDZvNADs9hQZ/bV0soX9heP5QBfkfBe3mgoCuKG2CT0vyBviI7Z8Ucu/ Ltt79ZnMFd1+T5wf3+X585c+cII6nJcl049CrmutnSPIDTAd+JcGfj4aqqldRCB2aqvG gnbcv86DKUE6E2n0zPa8vTvWsquxlxGyG4/BeekihPYE/9Out9+moOsoFpjRzuUDGvbC HisA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700069857; x=1700674657; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pxm7rj1nNs1AxdmMRh1p1RVuTse+b2SlzLkV0LVvDBU=; b=lv2XfPd6nEQXgbjkeWC3z6MdZ8Hltdd1MQ8yaJpXyjomls2xKM+bF1SUl2uZA2sz85 0Y2eh/u0gSd4EcGwxj3vVDFx/qFgyEHPzgnlCDM2c14NBWV8AUzp6hmmq2Rz93v8UoDy yzW2U4PGyKYJYwTR+iG3/G5C4+ZPYa4METatoXDVMgj2BMwZhNzW/eoNY+Oa1rQs+A32 /SlxHEnMOK6Lx2eznwVvdEhYqDW6s/2icGQYgA+ltJZBgl0lRjEiHevbalyqdYTH9VWl hhxvrt16FS8/GnAsMGbeXtbDhoFk1nHMXQe/3iGnP74P1CXTxzJ8D09dKkcior2qjI/L uBfQ== X-Gm-Message-State: AOJu0Ywk141eJ+aRY0gyI5dXm9yqevMPgXs5KfvHbq4SNIA/IUlEWuFw 3ZQ6rTUFs9DDSdnJ72iq7BLYY5amZg== X-Google-Smtp-Source: AGHT+IFWoKnkzh2sJI0YvIv6Zk4oTeGY1P2gC/wOCClYJ/hpiehbfxjiCRYsfqxU5qJ1L6qu2xAyBg== X-Received: by 2002:a05:620a:118e:b0:778:8da0:a6a2 with SMTP id b14-20020a05620a118e00b007788da0a6a2mr5872469qkk.28.1700069856945; Wed, 15 Nov 2023 09:37:36 -0800 (PST) Received: from citadel.lan ([2600:6c4a:4d3f:6d5c::1019]) by smtp.gmail.com with ESMTPSA id x20-20020a05620a0b5400b0077bda014d8esm3260099qkg.87.2023.11.15.09.37.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Nov 2023 09:37:36 -0800 (PST) From: Brian Gerst To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: Ingo Molnar , Thomas Gleixner , Borislav Petkov , "H . Peter Anvin" , Peter Zijlstra , Uros Bizjak , David.Laight@aculab.com, Brian Gerst Subject: [PATCH v3 09/14] x86/percpu/64: Use relative percpu offsets Date: Wed, 15 Nov 2023 12:37:03 -0500 Message-ID: <20231115173708.108316-10-brgerst@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231115173708.108316-1-brgerst@gmail.com> References: <20231115173708.108316-1-brgerst@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The percpu section is currently linked at virtual address 0, because older compilers hardcoded the stack protector canary value at a fixed offset from the start of the GS segment. Now that the canary is a normal percpu variable, the percpu section can be linked normally. This means that x86-64 will calculate percpu offsets like most other architectures, as the delta between the initial percpu address and the dynamically allocated memory. Signed-off-by: Brian Gerst --- arch/x86/include/asm/processor.h | 6 +++++- arch/x86/kernel/head_64.S | 19 +++++++++---------- arch/x86/kernel/setup_percpu.c | 12 ++---------- arch/x86/kernel/vmlinux.lds.S | 29 +---------------------------- arch/x86/platform/pvh/head.S | 5 ++--- arch/x86/tools/relocs.c | 10 +++------- arch/x86/xen/xen-head.S | 9 ++++----- init/Kconfig | 2 +- 8 files changed, 27 insertions(+), 65 deletions(-) diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/proces= sor.h index 3ee091225904..73fa9d4d2e16 100644 --- a/arch/x86/include/asm/processor.h +++ b/arch/x86/include/asm/processor.h @@ -403,7 +403,11 @@ DECLARE_INIT_PER_CPU(fixed_percpu_data); =20 static inline unsigned long cpu_kernelmode_gs_base(int cpu) { - return (unsigned long)per_cpu(fixed_percpu_data.gs_base, cpu); +#ifdef CONFIG_SMP + return per_cpu_offset(cpu); +#else + return 0; +#endif } =20 extern asmlinkage void entry_SYSCALL32_ignore(void); diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index 0d94d2a091fe..fe73e1c4cc5d 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -70,11 +70,14 @@ SYM_CODE_START_NOALIGN(startup_64) =20 leaq _text(%rip), %rdi =20 - /* Setup GSBASE to allow stack canary access for C code */ + /* + * Set up GSBASE. + * Note that, on SMP, the boot cpu uses init data section until + * the per cpu areas are set up. + */ movl $MSR_GS_BASE, %ecx - leaq INIT_PER_CPU_VAR(fixed_percpu_data)(%rip), %rdx - movl %edx, %eax - shrq $32, %rdx + xorl %eax, %eax + xorl %edx, %edx wrmsr =20 call startup_64_setup_env @@ -343,16 +346,12 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L= _GLOBAL) movl %eax,%fs movl %eax,%gs =20 - /* Set up %gs. - * - * The base of %gs always points to fixed_percpu_data. + /* + * Set up GSBASE. * Note that, on SMP, the boot cpu uses init data section until * the per cpu areas are set up. */ movl $MSR_GS_BASE,%ecx -#ifndef CONFIG_SMP - leaq INIT_PER_CPU_VAR(fixed_percpu_data)(%rip), %rdx -#endif movl %edx, %eax shrq $32, %rdx wrmsr diff --git a/arch/x86/kernel/setup_percpu.c b/arch/x86/kernel/setup_percpu.c index 2c97bf7b56ae..8707dd07b9ce 100644 --- a/arch/x86/kernel/setup_percpu.c +++ b/arch/x86/kernel/setup_percpu.c @@ -23,18 +23,10 @@ #include #include =20 -#ifdef CONFIG_X86_64 -#define BOOT_PERCPU_OFFSET ((unsigned long)__per_cpu_load) -#else -#define BOOT_PERCPU_OFFSET 0 -#endif - -DEFINE_PER_CPU_READ_MOSTLY(unsigned long, this_cpu_off) =3D BOOT_PERCPU_OF= FSET; +DEFINE_PER_CPU_READ_MOSTLY(unsigned long, this_cpu_off); EXPORT_PER_CPU_SYMBOL(this_cpu_off); =20 -unsigned long __per_cpu_offset[NR_CPUS] __ro_after_init =3D { - [0 ... NR_CPUS-1] =3D BOOT_PERCPU_OFFSET, -}; +unsigned long __per_cpu_offset[NR_CPUS] __ro_after_init; EXPORT_SYMBOL(__per_cpu_offset); =20 /* diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S index 1239be7cc8d8..57a83fb2d8a0 100644 --- a/arch/x86/kernel/vmlinux.lds.S +++ b/arch/x86/kernel/vmlinux.lds.S @@ -103,12 +103,6 @@ const_pcpu_hot =3D pcpu_hot; PHDRS { text PT_LOAD FLAGS(5); /* R_E */ data PT_LOAD FLAGS(6); /* RW_ */ -#ifdef CONFIG_X86_64 -#ifdef CONFIG_SMP - percpu PT_LOAD FLAGS(6); /* RW_ */ -#endif - init PT_LOAD FLAGS(7); /* RWE */ -#endif note PT_NOTE FLAGS(0); /* ___ */ } =20 @@ -224,21 +218,7 @@ SECTIONS __init_begin =3D .; /* paired with __init_end */ } =20 -#if defined(CONFIG_X86_64) && defined(CONFIG_SMP) - /* - * percpu offsets are zero-based on SMP. PERCPU_VADDR() changes the - * output PHDR, so the next output section - .init.text - should - * start another segment - init. - */ - PERCPU_VADDR(INTERNODE_CACHE_BYTES, 0, :percpu) - ASSERT(SIZEOF(.data..percpu) < CONFIG_PHYSICAL_START, - "per-CPU data too large - increase CONFIG_PHYSICAL_START") -#endif - INIT_TEXT_SECTION(PAGE_SIZE) -#ifdef CONFIG_X86_64 - :init -#endif =20 /* * Section for code used exclusively before alternatives are run. All @@ -368,9 +348,7 @@ SECTIONS EXIT_DATA } =20 -#if !defined(CONFIG_X86_64) || !defined(CONFIG_SMP) PERCPU_SECTION(INTERNODE_CACHE_BYTES) -#endif =20 . =3D ALIGN(PAGE_SIZE); =20 @@ -508,16 +486,11 @@ SECTIONS * Per-cpu symbols which need to be offset from __per_cpu_load * for the boot processor. */ -#define INIT_PER_CPU(x) init_per_cpu__##x =3D ABSOLUTE(x) + __per_cpu_load +#define INIT_PER_CPU(x) init_per_cpu__##x =3D ABSOLUTE(x) INIT_PER_CPU(gdt_page); INIT_PER_CPU(fixed_percpu_data); INIT_PER_CPU(irq_stack_backing_store); =20 -#ifdef CONFIG_SMP -. =3D ASSERT((fixed_percpu_data =3D=3D 0), - "fixed_percpu_data is not at start of per-cpu area"); -#endif - #ifdef CONFIG_CPU_UNRET_ENTRY . =3D ASSERT((retbleed_return_thunk & 0x3f) =3D=3D 0, "retbleed_return_thu= nk not cacheline-aligned"); #endif diff --git a/arch/x86/platform/pvh/head.S b/arch/x86/platform/pvh/head.S index fab90368481f..2ce07dffc314 100644 --- a/arch/x86/platform/pvh/head.S +++ b/arch/x86/platform/pvh/head.S @@ -100,9 +100,8 @@ SYM_CODE_START_LOCAL(pvh_start_xen) * the per cpu areas are set up. */ mov $MSR_GS_BASE,%ecx - lea INIT_PER_CPU_VAR(fixed_percpu_data)(%rip), %rdx - mov %edx, %eax - shr $32, %rdx + xor %eax, %eax + xor %edx, %edx wrmsr =20 call xen_prepare_pvh diff --git a/arch/x86/tools/relocs.c b/arch/x86/tools/relocs.c index 24ad10c62840..ef355242a8d8 100644 --- a/arch/x86/tools/relocs.c +++ b/arch/x86/tools/relocs.c @@ -822,12 +822,7 @@ static void percpu_init(void) */ static int is_percpu_sym(ElfW(Sym) *sym, const char *symname) { - int shndx =3D sym_index(sym); - - return (shndx =3D=3D per_cpu_shndx) && - strcmp(symname, "__init_begin") && - strcmp(symname, "__per_cpu_load") && - strncmp(symname, "init_per_cpu_", 13); + return 0; } =20 =20 @@ -1051,7 +1046,8 @@ static int cmp_relocs(const void *va, const void *vb) =20 static void sort_relocs(struct relocs *r) { - qsort(r->offset, r->count, sizeof(r->offset[0]), cmp_relocs); + if (r->count) + qsort(r->offset, r->count, sizeof(r->offset[0]), cmp_relocs); } =20 static int write32(uint32_t v, FILE *f) diff --git a/arch/x86/xen/xen-head.S b/arch/x86/xen/xen-head.S index 30f27e757354..7e8754c5fa1d 100644 --- a/arch/x86/xen/xen-head.S +++ b/arch/x86/xen/xen-head.S @@ -51,15 +51,14 @@ SYM_CODE_START(startup_xen) =20 leaq (__end_init_task - PTREGS_SIZE)(%rip), %rsp =20 - /* Set up %gs. - * - * The base of %gs always points to fixed_percpu_data. + /* + * Set up GSBASE. * Note that, on SMP, the boot cpu uses init data section until * the per cpu areas are set up. */ movl $MSR_GS_BASE,%ecx - movq $INIT_PER_CPU_VAR(fixed_percpu_data),%rax - cdq + xorl %eax, %eax + xorl %edx, %edx wrmsr =20 mov %rsi, %rdi diff --git a/init/Kconfig b/init/Kconfig index 9ffb103fc927..5f2c1f4a16aa 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -1718,7 +1718,7 @@ config KALLSYMS_ALL config KALLSYMS_ABSOLUTE_PERCPU bool depends on KALLSYMS - default X86_64 && SMP + default n =20 config KALLSYMS_BASE_RELATIVE bool --=20 2.41.0 From nobody Tue Dec 30 12:50:30 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 70944C2BB3F for ; Wed, 15 Nov 2023 17:38:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233007AbjKORiL (ORCPT ); Wed, 15 Nov 2023 12:38:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58004 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232682AbjKORhx (ORCPT ); Wed, 15 Nov 2023 12:37:53 -0500 Received: from mail-qk1-x733.google.com (mail-qk1-x733.google.com [IPv6:2607:f8b0:4864:20::733]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A25ABD6A for ; Wed, 15 Nov 2023 09:37:40 -0800 (PST) Received: by mail-qk1-x733.google.com with SMTP id af79cd13be357-778940531dbso429996485a.0 for ; Wed, 15 Nov 2023 09:37:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700069858; x=1700674658; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xReHSzApaFWYK7/D7xuGCs8slM1yGD2PO+yQfeDqI34=; b=QQIiXkDDREiKYzoha2vEoi0pEJvt6uxEtnmCMbfElZXjchqMra9ydKlM8naapJA4EZ NmZ69lzbAlHT2AWNb0KtvjCM2xhMROGNhXVXDp538QpX8QbOi2gNbmeQab5KCzwPsV3l 19xigslpyXrNJBlhPwbSFLOARNOGX4jSOStXdKUmhGcTB5fil4DvFNjTgY3LtW9iBMeF ezw9J8Mv0Tmn4eqCtq8fQoCtNvidUUB5MLZR0+ViHENWaO4PAmy4Wgh7YI+4pyYq08c1 VnfJHubn3Ea9px64JXiI5iTUf9oZ26HyfnYLuA+dqDWAwTjn3bsJc92x8fYK2B6jlICc 1+EA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700069858; x=1700674658; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xReHSzApaFWYK7/D7xuGCs8slM1yGD2PO+yQfeDqI34=; b=EASduukFl8766SKHG/A6VX0Xggs7v8jfpI1x78AXsPssovlGnItN6RzszTpvjXSwr5 RJmjiIyc+U/zoPPdb02V7EXHimsuBi/UIxLNfnU7DOUvW3XyX7/F5frkoMWjsvd8yO61 7GjtvyXBBovHTyiVp1ixfBEF5GMBWiFMooYvGKOwaNtM3j9DQwcWw8SNn1AeQzSqVTTU RXiEMPjL8Q9iGlgpYC3Hbw7b8U/US0VjSpkNfMm1i89iIpKsLJA+7oYQA163/VfIJ78r C4RAncdBBQn0LxyuSBpleJAk8vtGpW0NoUVfsxjk/53ZgYRuh4WfDBG5hITmOIIcnwqp QM2g== X-Gm-Message-State: AOJu0YzukQCOLR09qCgEf8iH2wcuGFdpeIP3Cyqg6ws9aXN0+2RPQE2D R0cSEq2QjkFgjWWcv+48jqdk4gF/gQ== X-Google-Smtp-Source: AGHT+IGPE9obzWO8K9sGVIqKBZ2smTQ1773aqiUlD2q2gxu38kLTvubuGlOPY1A5pPrtaFe5tQ2ZlA== X-Received: by 2002:a05:6214:9c1:b0:66f:b8a2:521a with SMTP id dp1-20020a05621409c100b0066fb8a2521amr5632326qvb.17.1700069858535; Wed, 15 Nov 2023 09:37:38 -0800 (PST) Received: from citadel.lan ([2600:6c4a:4d3f:6d5c::1019]) by smtp.gmail.com with ESMTPSA id x20-20020a05620a0b5400b0077bda014d8esm3260099qkg.87.2023.11.15.09.37.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Nov 2023 09:37:37 -0800 (PST) From: Brian Gerst To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: Ingo Molnar , Thomas Gleixner , Borislav Petkov , "H . Peter Anvin" , Peter Zijlstra , Uros Bizjak , David.Laight@aculab.com, Brian Gerst Subject: [PATCH v3 10/14] x86/percpu/64: Remove fixed_percpu_data Date: Wed, 15 Nov 2023 12:37:04 -0500 Message-ID: <20231115173708.108316-11-brgerst@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231115173708.108316-1-brgerst@gmail.com> References: <20231115173708.108316-1-brgerst@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Now that the stack protector canary value is a normal percpu variable, fixed_percpu_data is unused and can be removed. Signed-off-by: Brian Gerst --- arch/x86/include/asm/processor.h | 8 -------- arch/x86/kernel/cpu/common.c | 4 ---- arch/x86/kernel/vmlinux.lds.S | 1 - arch/x86/tools/relocs.c | 1 - 4 files changed, 14 deletions(-) diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/proces= sor.h index 73fa9d4d2e16..f84c8d3ca75d 100644 --- a/arch/x86/include/asm/processor.h +++ b/arch/x86/include/asm/processor.h @@ -393,14 +393,6 @@ struct irq_stack { } __aligned(IRQ_STACK_SIZE); =20 #ifdef CONFIG_X86_64 -struct fixed_percpu_data { - char gs_base[40]; - unsigned long reserved; -}; - -DECLARE_PER_CPU_FIRST(struct fixed_percpu_data, fixed_percpu_data) __visib= le; -DECLARE_INIT_PER_CPU(fixed_percpu_data); - static inline unsigned long cpu_kernelmode_gs_base(int cpu) { #ifdef CONFIG_SMP diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index b5b1d95b1399..a7792479ebe1 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -2059,10 +2059,6 @@ EXPORT_PER_CPU_SYMBOL(pcpu_hot); EXPORT_PER_CPU_SYMBOL(const_pcpu_hot); =20 #ifdef CONFIG_X86_64 -DEFINE_PER_CPU_FIRST(struct fixed_percpu_data, - fixed_percpu_data) __aligned(PAGE_SIZE) __visible; -EXPORT_PER_CPU_SYMBOL_GPL(fixed_percpu_data); - static void wrmsrl_cstar(unsigned long val) { /* diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S index 57a83fb2d8a0..efa4885060b5 100644 --- a/arch/x86/kernel/vmlinux.lds.S +++ b/arch/x86/kernel/vmlinux.lds.S @@ -488,7 +488,6 @@ SECTIONS */ #define INIT_PER_CPU(x) init_per_cpu__##x =3D ABSOLUTE(x) INIT_PER_CPU(gdt_page); -INIT_PER_CPU(fixed_percpu_data); INIT_PER_CPU(irq_stack_backing_store); =20 #ifdef CONFIG_CPU_UNRET_ENTRY diff --git a/arch/x86/tools/relocs.c b/arch/x86/tools/relocs.c index ef355242a8d8..3b0cfddd8b27 100644 --- a/arch/x86/tools/relocs.c +++ b/arch/x86/tools/relocs.c @@ -817,7 +817,6 @@ static void percpu_init(void) * __per_cpu_load * * The "gold" linker incorrectly associates: - * init_per_cpu__fixed_percpu_data * init_per_cpu__gdt_page */ static int is_percpu_sym(ElfW(Sym) *sym, const char *symname) --=20 2.41.0 From nobody Tue Dec 30 12:50:30 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D44B6C2BB3F for ; Wed, 15 Nov 2023 17:38:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232661AbjKORiS (ORCPT ); Wed, 15 Nov 2023 12:38:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60688 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232919AbjKORiI (ORCPT ); Wed, 15 Nov 2023 12:38:08 -0500 Received: from mail-oo1-xc36.google.com (mail-oo1-xc36.google.com [IPv6:2607:f8b0:4864:20::c36]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DE6951715 for ; Wed, 15 Nov 2023 09:37:42 -0800 (PST) Received: by mail-oo1-xc36.google.com with SMTP id 006d021491bc7-5842a7fdc61so3583604eaf.3 for ; Wed, 15 Nov 2023 09:37:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700069860; x=1700674660; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rM3WKUS19C6H61LiRqPZba/3FsTYnCRGf0Iggb95Tzk=; b=fQTn0A876g5qiGlEZDA797uktb1Gb5FR85rdkwvEQQWTBlllKrJ/e7kVKqBWPExrOP gUoYji5u5ZWQJ/EKtwo4YJbM+PpGBsLz6MlhMuEm86VGPYbzNavtGkF9vzwIVdOqRcko l1088g8no2/Ik/xohGrVK4Ma6seOvzsdeGnbK8PeuWUSRq6Eiq+2/mPJqWbqnsgBoQwf fqQUVIUUbrf9uNfBYWhOxAbnwGhaMvkrNzBbVuMfpvDgO9ZL5FAub+SumBz5Oa7zKGtp 51NQs4JswdvmOeGWRzhnN4n2hkGdP68z+UuNj7DIC/QEMFO5w7nYbMUSnZt39cmc6yIx xtEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700069860; x=1700674660; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rM3WKUS19C6H61LiRqPZba/3FsTYnCRGf0Iggb95Tzk=; b=cc7xDpFX1F+8b6WandOaUT+CDxv3gW7Be9izLuXwa9LfhYiTS4AfxzcrYF55otbuAv Xlun4TCRgIHvFIbhKqKlr3Qyys4fbXhYlo60ksWVXVKYKQV5SRNAEs+z8ge6nkV/sHTJ jix0/lc3eRs3l4yI7yXrvcH6Uu3w6fwtCNa22e7jj1hfheoBUgZ9Hs1QP2eTBaRA1fmk 5EX7r6XZo/qJawtejxGdZGmotz8I8TnPnnfN4aA1fR3RYvYaofekCiZlEBfrIO4kUGzf yj2qKqviWsS6dl9fXGDLpEhUzsSo12EkPdJevnD/kY1hrZ2avjeohHM2AKDfYrrqwmYY 0VNQ== X-Gm-Message-State: AOJu0YxfnR5AhlWXDjiP0/ErA04tjQ0oeu+CwqfUawjc8VglVofTdK49 2MVo1Jzu9YG7XKVBlK45EncSY2KoEA== X-Google-Smtp-Source: AGHT+IGDNerEzTHNzojJnC39t/wRd3hyjOhnAWFGMiSSfdd2wqsdNGdI+oILO+Kbi48J4gu2+A8rzw== X-Received: by 2002:a05:6359:1581:b0:16b:fd1c:fdba with SMTP id jv1-20020a056359158100b0016bfd1cfdbamr5343991rwb.27.1700069860075; Wed, 15 Nov 2023 09:37:40 -0800 (PST) Received: from citadel.lan ([2600:6c4a:4d3f:6d5c::1019]) by smtp.gmail.com with ESMTPSA id x20-20020a05620a0b5400b0077bda014d8esm3260099qkg.87.2023.11.15.09.37.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Nov 2023 09:37:39 -0800 (PST) From: Brian Gerst To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: Ingo Molnar , Thomas Gleixner , Borislav Petkov , "H . Peter Anvin" , Peter Zijlstra , Uros Bizjak , David.Laight@aculab.com, Brian Gerst Subject: [PATCH v3 11/14] x86/boot/64: Remove inverse relocations Date: Wed, 15 Nov 2023 12:37:05 -0500 Message-ID: <20231115173708.108316-12-brgerst@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231115173708.108316-1-brgerst@gmail.com> References: <20231115173708.108316-1-brgerst@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Now that the percpu section is not at a fixed virtual address, inverse relocations, which were needed to offset the effects of relocation on RIP-relative percpu references, are no longer needed. Signed-off-by: Brian Gerst --- arch/x86/boot/compressed/misc.c | 14 +--- arch/x86/tools/relocs.c | 126 +------------------------------- 2 files changed, 2 insertions(+), 138 deletions(-) diff --git a/arch/x86/boot/compressed/misc.c b/arch/x86/boot/compressed/mis= c.c index b99e08e6815b..2de345a236c0 100644 --- a/arch/x86/boot/compressed/misc.c +++ b/arch/x86/boot/compressed/misc.c @@ -221,7 +221,7 @@ static void handle_relocations(void *output, unsigned l= ong output_len, =20 /* * Process relocations: 32 bit relocations first then 64 bit after. - * Three sets of binary relocations are added to the end of the kernel + * Two sets of binary relocations are added to the end of the kernel * before compression. Each relocation table entry is the kernel * address of the location which needs to be updated stored as a * 32-bit value which is sign extended to 64 bits. @@ -231,8 +231,6 @@ static void handle_relocations(void *output, unsigned l= ong output_len, * kernel bits... * 0 - zero terminator for 64 bit relocations * 64 bit relocation repeated - * 0 - zero terminator for inverse 32 bit relocations - * 32 bit inverse relocation repeated * 0 - zero terminator for 32 bit relocations * 32 bit relocation repeated * @@ -249,16 +247,6 @@ static void handle_relocations(void *output, unsigned = long output_len, *(uint32_t *)ptr +=3D delta; } #ifdef CONFIG_X86_64 - while (*--reloc) { - long extended =3D *reloc; - extended +=3D map; - - ptr =3D (unsigned long)extended; - if (ptr < min_addr || ptr > max_addr) - error("inverse 32-bit relocation outside of kernel!\n"); - - *(int32_t *)ptr -=3D delta; - } for (reloc--; *reloc; reloc--) { long extended =3D *reloc; extended +=3D map; diff --git a/arch/x86/tools/relocs.c b/arch/x86/tools/relocs.c index 3b0cfddd8b27..ae9bbf634826 100644 --- a/arch/x86/tools/relocs.c +++ b/arch/x86/tools/relocs.c @@ -28,7 +28,6 @@ struct relocs { static struct relocs relocs16; static struct relocs relocs32; #if ELF_BITS =3D=3D 64 -static struct relocs relocs32neg; static struct relocs relocs64; #define FMT PRIu64 =20 @@ -89,7 +88,6 @@ static const char * const sym_regex_kernel[S_NSYMTYPES] = =3D { "__initramfs_start|" "(jiffies|jiffies_64)|" #if ELF_BITS =3D=3D 64 - "__per_cpu_load|" "init_per_cpu__.*|" "__end_rodata_hpage_align|" #endif @@ -287,33 +285,6 @@ static const char *sym_name(const char *sym_strtab, El= f_Sym *sym) return name; } =20 -static Elf_Sym *sym_lookup(const char *symname) -{ - int i; - for (i =3D 0; i < shnum; i++) { - struct section *sec =3D &secs[i]; - long nsyms; - char *strtab; - Elf_Sym *symtab; - Elf_Sym *sym; - - if (sec->shdr.sh_type !=3D SHT_SYMTAB) - continue; - - nsyms =3D sec->shdr.sh_size/sizeof(Elf_Sym); - symtab =3D sec->symtab; - strtab =3D sec->link->strtab; - - for (sym =3D symtab; --nsyms >=3D 0; sym++) { - if (!sym->st_name) - continue; - if (strcmp(symname, strtab + sym->st_name) =3D=3D 0) - return sym; - } - } - return 0; -} - #if BYTE_ORDER =3D=3D LITTLE_ENDIAN #define le16_to_cpu(val) (val) #define le32_to_cpu(val) (val) @@ -756,75 +727,8 @@ static void walk_relocs(int (*process)(struct section = *sec, Elf_Rel *rel, } } =20 -/* - * The .data..percpu section is a special case for x86_64 SMP kernels. - * It is used to initialize the actual per_cpu areas and to provide - * definitions for the per_cpu variables that correspond to their offsets - * within the percpu area. Since the values of all of the symbols need - * to be offsets from the start of the per_cpu area the virtual address - * (sh_addr) of .data..percpu is 0 in SMP kernels. - * - * This means that: - * - * Relocations that reference symbols in the per_cpu area do not - * need further relocation (since the value is an offset relative - * to the start of the per_cpu area that does not change). - * - * Relocations that apply to the per_cpu area need to have their - * offset adjusted by by the value of __per_cpu_load to make them - * point to the correct place in the loaded image (because the - * virtual address of .data..percpu is 0). - * - * For non SMP kernels .data..percpu is linked as part of the normal - * kernel data and does not require special treatment. - * - */ -static int per_cpu_shndx =3D -1; -static Elf_Addr per_cpu_load_addr; - -static void percpu_init(void) -{ - int i; - for (i =3D 0; i < shnum; i++) { - ElfW(Sym) *sym; - if (strcmp(sec_name(i), ".data..percpu")) - continue; - - if (secs[i].shdr.sh_addr !=3D 0) /* non SMP kernel */ - return; - - sym =3D sym_lookup("__per_cpu_load"); - if (!sym) - die("can't find __per_cpu_load\n"); - - per_cpu_shndx =3D i; - per_cpu_load_addr =3D sym->st_value; - return; - } -} - #if ELF_BITS =3D=3D 64 =20 -/* - * Check to see if a symbol lies in the .data..percpu section. - * - * The linker incorrectly associates some symbols with the - * .data..percpu section so we also need to check the symbol - * name to make sure that we classify the symbol correctly. - * - * The GNU linker incorrectly associates: - * __init_begin - * __per_cpu_load - * - * The "gold" linker incorrectly associates: - * init_per_cpu__gdt_page - */ -static int is_percpu_sym(ElfW(Sym) *sym, const char *symname) -{ - return 0; -} - - static int do_reloc64(struct section *sec, Elf_Rel *rel, ElfW(Sym) *sym, const char *symname) { @@ -835,12 +739,6 @@ static int do_reloc64(struct section *sec, Elf_Rel *re= l, ElfW(Sym) *sym, if (sym->st_shndx =3D=3D SHN_UNDEF) return 0; =20 - /* - * Adjust the offset if this reloc applies to the percpu section. - */ - if (sec->shdr.sh_info =3D=3D per_cpu_shndx) - offset +=3D per_cpu_load_addr; - switch (r_type) { case R_X86_64_NONE: /* NONE can be ignored. */ @@ -850,33 +748,21 @@ static int do_reloc64(struct section *sec, Elf_Rel *r= el, ElfW(Sym) *sym, case R_X86_64_PLT32: case R_X86_64_REX_GOTPCRELX: /* - * PC relative relocations don't need to be adjusted unless - * referencing a percpu symbol. + * PC relative relocations don't need to be adjusted. * * NB: R_X86_64_PLT32 can be treated as R_X86_64_PC32. */ - if (is_percpu_sym(sym, symname)) - add_reloc(&relocs32neg, offset); break; =20 case R_X86_64_PC64: /* * Only used by jump labels */ - if (is_percpu_sym(sym, symname)) - die("Invalid R_X86_64_PC64 relocation against per-CPU symbol %s\n", - symname); break; =20 case R_X86_64_32: case R_X86_64_32S: case R_X86_64_64: - /* - * References to the percpu area don't need to be adjusted. - */ - if (is_percpu_sym(sym, symname)) - break; - if (shn_abs) { /* * Whitelisted absolute symbols do not require @@ -1090,7 +976,6 @@ static void emit_relocs(int as_text, int use_real_mode) /* Order the relocations for more efficient processing */ sort_relocs(&relocs32); #if ELF_BITS =3D=3D 64 - sort_relocs(&relocs32neg); sort_relocs(&relocs64); #else sort_relocs(&relocs16); @@ -1122,13 +1007,6 @@ static void emit_relocs(int as_text, int use_real_mo= de) /* Now print each relocation */ for (i =3D 0; i < relocs64.count; i++) write_reloc(relocs64.offset[i], stdout); - - /* Print a stop */ - write_reloc(0, stdout); - - /* Now print each inverse 32-bit relocation */ - for (i =3D 0; i < relocs32neg.count; i++) - write_reloc(relocs32neg.offset[i], stdout); #endif =20 /* Print a stop */ @@ -1179,8 +1057,6 @@ void process(FILE *fp, int use_real_mode, int as_text, read_strtabs(fp); read_symtabs(fp); read_relocs(fp); - if (ELF_BITS =3D=3D 64) - percpu_init(); if (show_absolute_syms) { print_absolute_symbols(); return; --=20 2.41.0 From nobody Tue Dec 30 12:50:30 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E758BC2BB3F for ; Wed, 15 Nov 2023 17:38:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233064AbjKORiP (ORCPT ); Wed, 15 Nov 2023 12:38:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60806 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232907AbjKORiI (ORCPT ); Wed, 15 Nov 2023 12:38:08 -0500 Received: from mail-qk1-x72f.google.com (mail-qk1-x72f.google.com [IPv6:2607:f8b0:4864:20::72f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 68F9F1724 for ; Wed, 15 Nov 2023 09:37:43 -0800 (PST) Received: by mail-qk1-x72f.google.com with SMTP id af79cd13be357-778ac9c898dso371860885a.0 for ; Wed, 15 Nov 2023 09:37:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700069861; x=1700674661; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PhFnq9Ngmt9wj9jeMbM3jBJDuaOt4o4B8wK/xZ02mGU=; b=T/cxrHaG0mZMvP2jpdukb0SLqXkvsj1XoaDnGyezN8MPu0+Ueb7/bfljOaEYwWQJXU CnRcWxJXiKltjAy/IrJk8Q30BF8PYKeDRiK+290y2N5yPXVgP3PC/W/jLVIam8Dm4nt3 q2GSjnF13FeZ6N6IqxGuuF202G9vdMrvYbo184vgACYTVkjFrRuHjFwp5dghK1dg04Nh MQeiil0hyendsMIey1fgWr4RGqkRzugEKIcEmAfR0/45BRyMOny93O4gCiI5rzftXPbc uN4wsU9kXIsKPIJ7kKH1dLNIhPOLuwJStN51JnXKwOJuWufpbFD29susP+i1OSwDiF9v G5Lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700069861; x=1700674661; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PhFnq9Ngmt9wj9jeMbM3jBJDuaOt4o4B8wK/xZ02mGU=; b=eTN8t330X5xyEv7Es8P2Q0iL4V+sWIUjQ4p6MaXeSeToDMIBOJvQ//l3JMwKRqK1+s 0XMxaIEacIMDGlb8BStDBKEVVSpvhPmye8U6n2e/C8nn5LsPgmNjanSsQIrU1s+I3aR6 2xhLjJaTGa9E/a5HHy90QvlWdBIBQdF9nSkrLj1axOW6O1SKeBuREn2Mfe9elvKhBdmG Z4iNQbJXRVljlLkNXwcFn9oq6wCiHkmkLktIVYIMEwh8NJXIwcWk/OfZ58YzVemnzNts RvNPhmZcdkczMw6ADtEvy/PB3w29K65st1NEfW2osjnTNuldATKjDYZp6cyEkbs1uYcG ymqg== X-Gm-Message-State: AOJu0Yy3CR7x0UNVgy4IdhrK8bSuu92yto9mUpe3bimESA4AbWA7djsw UrdY4BEazZQJ2/F+CHiVYo5iK9Lf7g== X-Google-Smtp-Source: AGHT+IEAC5Qw0c2DGcdRu+XAfLZHT0LeaPcaMiUc/D+j7PvvIhx0XA4IDNz5R3/au6vbzlcFh/dj9w== X-Received: by 2002:a05:620a:f02:b0:778:91f0:e559 with SMTP id v2-20020a05620a0f0200b0077891f0e559mr6431724qkl.78.1700069861182; Wed, 15 Nov 2023 09:37:41 -0800 (PST) Received: from citadel.lan ([2600:6c4a:4d3f:6d5c::1019]) by smtp.gmail.com with ESMTPSA id x20-20020a05620a0b5400b0077bda014d8esm3260099qkg.87.2023.11.15.09.37.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Nov 2023 09:37:40 -0800 (PST) From: Brian Gerst To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: Ingo Molnar , Thomas Gleixner , Borislav Petkov , "H . Peter Anvin" , Peter Zijlstra , Uros Bizjak , David.Laight@aculab.com, Brian Gerst Subject: [PATCH v3 12/14] x86/percpu/64: Remove INIT_PER_CPU macros Date: Wed, 15 Nov 2023 12:37:06 -0500 Message-ID: <20231115173708.108316-13-brgerst@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231115173708.108316-1-brgerst@gmail.com> References: <20231115173708.108316-1-brgerst@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The load and link addresses of percpu variables are now the same, so these macros are no longer necessary. Signed-off-by: Brian Gerst --- arch/x86/include/asm/percpu.h | 22 ---------------------- arch/x86/kernel/irq_64.c | 1 - arch/x86/kernel/vmlinux.lds.S | 7 ------- arch/x86/tools/relocs.c | 1 - 4 files changed, 31 deletions(-) diff --git a/arch/x86/include/asm/percpu.h b/arch/x86/include/asm/percpu.h index b86b27d15e52..7a176381ee01 100644 --- a/arch/x86/include/asm/percpu.h +++ b/arch/x86/include/asm/percpu.h @@ -20,12 +20,6 @@ =20 #define PER_CPU_VAR(var) __percpu(var)__percpu_rel =20 -#ifdef CONFIG_X86_64_SMP -#define INIT_PER_CPU_VAR(var) init_per_cpu__##var -#else -#define INIT_PER_CPU_VAR(var) var -#endif - #else /* ...!ASSEMBLY */ =20 #include @@ -96,22 +90,6 @@ #define __percpu_arg(x) __percpu_prefix "%" #x #define __force_percpu_arg(x) __force_percpu_prefix "%" #x =20 -/* - * Initialized pointers to per-cpu variables needed for the boot - * processor need to use these macros to get the proper address - * offset from __per_cpu_load on SMP. - * - * There also must be an entry in vmlinux_64.lds.S - */ -#define DECLARE_INIT_PER_CPU(var) \ - extern typeof(var) init_per_cpu_var(var) - -#ifdef CONFIG_X86_64_SMP -#define init_per_cpu_var(var) init_per_cpu__##var -#else -#define init_per_cpu_var(var) var -#endif - /* For arch-specific code, we can use direct single-insn ops (they * don't give an lvalue though). */ =20 diff --git a/arch/x86/kernel/irq_64.c b/arch/x86/kernel/irq_64.c index fe0c859873d1..30424f9876bc 100644 --- a/arch/x86/kernel/irq_64.c +++ b/arch/x86/kernel/irq_64.c @@ -26,7 +26,6 @@ #include =20 DEFINE_PER_CPU_PAGE_ALIGNED(struct irq_stack, irq_stack_backing_store) __v= isible; -DECLARE_INIT_PER_CPU(irq_stack_backing_store); =20 #ifdef CONFIG_VMAP_STACK /* diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S index efa4885060b5..9aea7b6b02c7 100644 --- a/arch/x86/kernel/vmlinux.lds.S +++ b/arch/x86/kernel/vmlinux.lds.S @@ -482,13 +482,6 @@ SECTIONS "kernel image bigger than KERNEL_IMAGE_SIZE"); =20 #ifdef CONFIG_X86_64 -/* - * Per-cpu symbols which need to be offset from __per_cpu_load - * for the boot processor. - */ -#define INIT_PER_CPU(x) init_per_cpu__##x =3D ABSOLUTE(x) -INIT_PER_CPU(gdt_page); -INIT_PER_CPU(irq_stack_backing_store); =20 #ifdef CONFIG_CPU_UNRET_ENTRY . =3D ASSERT((retbleed_return_thunk & 0x3f) =3D=3D 0, "retbleed_return_thu= nk not cacheline-aligned"); diff --git a/arch/x86/tools/relocs.c b/arch/x86/tools/relocs.c index ae9bbf634826..70b7b0bf33d0 100644 --- a/arch/x86/tools/relocs.c +++ b/arch/x86/tools/relocs.c @@ -88,7 +88,6 @@ static const char * const sym_regex_kernel[S_NSYMTYPES] = =3D { "__initramfs_start|" "(jiffies|jiffies_64)|" #if ELF_BITS =3D=3D 64 - "init_per_cpu__.*|" "__end_rodata_hpage_align|" #endif "__vvar_page|" --=20 2.41.0 From nobody Tue Dec 30 12:50:30 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6E71C072A2 for ; Wed, 15 Nov 2023 17:38:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232871AbjKORi0 (ORCPT ); Wed, 15 Nov 2023 12:38:26 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54558 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232823AbjKORiI (ORCPT ); Wed, 15 Nov 2023 12:38:08 -0500 Received: from mail-qk1-x732.google.com (mail-qk1-x732.google.com [IPv6:2607:f8b0:4864:20::732]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 68433173B for ; Wed, 15 Nov 2023 09:37:44 -0800 (PST) Received: by mail-qk1-x732.google.com with SMTP id af79cd13be357-7781bc3783fso459173485a.1 for ; Wed, 15 Nov 2023 09:37:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700069862; x=1700674662; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Pv8uTN4aHjsRuvoEGWM8GCEtdvaJFwUVZz3IVlIYzsA=; b=nJqMBNDDm80Mc+hthJoqWkPkxqQXFZWfWTrkbyCp8IBMYn6lBhkBiC5FtTz/hwl4TY +v0ckxFYS6QXHe2ZdYiGbTlo7ZUD5hfgnE9cojKZ11kiNBtadb2IQilXlrt0NS4R7rsV m26+aTppe6lwxvSWSVBniMDFjD6zNnh7OOlYCPyVyI4tRYdGUK9loC+iKsEGfrSEHRmF 36UH0UJzUq+DWb0dI8uYrPy9LcM6gVFAaOT0v10Kt0ukNZMV1VMqLQnaGXV+DLFGrZ7G IKfkFlOyqS40bHlhIhKxJHh9eFVQFk+SjlMPTLUzaD5Msc7dNSEdWapzQP03ubOKEMpm 9KEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700069862; x=1700674662; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Pv8uTN4aHjsRuvoEGWM8GCEtdvaJFwUVZz3IVlIYzsA=; b=F5sm9ss3dmFuu0G8NcLnvDfy41OkWmf8vQuW2dViLIh2VNFJyBxklxvauh0kx128sw 1FoIMAAHMBbW8YEVf8+SilIHVxlHdzGl2LNfBTZVRWFQgtVHWZNyn2a8fxBFPZlSDXXQ +JdpBgWzzogvQ0Vg+4nJkh+KHZQL4sUPH86PoiUZvL5GS498W9EG96r/SAxJ3qULJOU7 GhWPwkloDze3z6bisZ7+I/BqF+YyTpUNso4crkoEVchV7vRuKzDsgBzyUHcjWOZuvIn+ I306Kn75sZ9AvOFXda6uCLtAJe6Tf0kInUP3kncTkuMYipolIVDqbIj7riOK1XNegyh/ u/FQ== X-Gm-Message-State: AOJu0YzKNoOUeFk1fYUoTT38ytmlL2f+xAhue8RMJ0SU0VX5PmBtzxzD 0IaIMPRBdpEb01gFhdhEYRTGuMUnug== X-Google-Smtp-Source: AGHT+IEDZsp9UakDhaCQ083s0oOcpalrQ55cJXqz8rBgiwh1nm1vQuVM64EOFrfe4e03iMP0/rU6Ww== X-Received: by 2002:a05:6214:c62:b0:671:3493:61e8 with SMTP id t2-20020a0562140c6200b00671349361e8mr8034104qvj.26.1700069862230; Wed, 15 Nov 2023 09:37:42 -0800 (PST) Received: from citadel.lan ([2600:6c4a:4d3f:6d5c::1019]) by smtp.gmail.com with ESMTPSA id x20-20020a05620a0b5400b0077bda014d8esm3260099qkg.87.2023.11.15.09.37.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Nov 2023 09:37:41 -0800 (PST) From: Brian Gerst To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: Ingo Molnar , Thomas Gleixner , Borislav Petkov , "H . Peter Anvin" , Peter Zijlstra , Uros Bizjak , David.Laight@aculab.com, Brian Gerst Subject: [PATCH v3 13/14] percpu: Remove PER_CPU_FIRST_SECTION Date: Wed, 15 Nov 2023 12:37:07 -0500 Message-ID: <20231115173708.108316-14-brgerst@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231115173708.108316-1-brgerst@gmail.com> References: <20231115173708.108316-1-brgerst@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" x86-64 was the only user. Signed-off-by: Brian Gerst --- include/asm-generic/vmlinux.lds.h | 1 - include/linux/percpu-defs.h | 12 ------------ 2 files changed, 13 deletions(-) diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinu= x.lds.h index bae0fe4d499b..579bd5ad09b9 100644 --- a/include/asm-generic/vmlinux.lds.h +++ b/include/asm-generic/vmlinux.lds.h @@ -1026,7 +1026,6 @@ */ #define PERCPU_INPUT(cacheline) \ __per_cpu_start =3D .; \ - *(.data..percpu..first) \ . =3D ALIGN(PAGE_SIZE); \ *(.data..percpu..page_aligned) \ . =3D ALIGN(cacheline); \ diff --git a/include/linux/percpu-defs.h b/include/linux/percpu-defs.h index ec3573119923..b9ddee91e6c7 100644 --- a/include/linux/percpu-defs.h +++ b/include/linux/percpu-defs.h @@ -26,13 +26,11 @@ #define PER_CPU_SHARED_ALIGNED_SECTION "..shared_aligned" #define PER_CPU_ALIGNED_SECTION "..shared_aligned" #endif -#define PER_CPU_FIRST_SECTION "..first" =20 #else =20 #define PER_CPU_SHARED_ALIGNED_SECTION "" #define PER_CPU_ALIGNED_SECTION "..shared_aligned" -#define PER_CPU_FIRST_SECTION "" =20 #endif =20 @@ -114,16 +112,6 @@ #define DEFINE_PER_CPU(type, name) \ DEFINE_PER_CPU_SECTION(type, name, "") =20 -/* - * Declaration/definition used for per-CPU variables that must come first = in - * the set of variables. - */ -#define DECLARE_PER_CPU_FIRST(type, name) \ - DECLARE_PER_CPU_SECTION(type, name, PER_CPU_FIRST_SECTION) - -#define DEFINE_PER_CPU_FIRST(type, name) \ - DEFINE_PER_CPU_SECTION(type, name, PER_CPU_FIRST_SECTION) - /* * Declaration/definition used for per-CPU variables that must be cacheline * aligned under SMP conditions so that, whilst a particular instance of t= he --=20 2.41.0 From nobody Tue Dec 30 12:50:30 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 764DDC072A2 for ; Wed, 15 Nov 2023 17:38:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232894AbjKORi3 (ORCPT ); Wed, 15 Nov 2023 12:38:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58034 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232992AbjKORiK (ORCPT ); Wed, 15 Nov 2023 12:38:10 -0500 Received: from mail-qk1-x72f.google.com (mail-qk1-x72f.google.com [IPv6:2607:f8b0:4864:20::72f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 16DD01988 for ; Wed, 15 Nov 2023 09:37:44 -0800 (PST) Received: by mail-qk1-x72f.google.com with SMTP id af79cd13be357-77bc5d8490dso374587985a.2 for ; Wed, 15 Nov 2023 09:37:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700069864; x=1700674664; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wg6IcMPinfxxVmG7Rw0rCvEQaQgxpQ0/U8bjgfX6Jak=; b=GIsUJIqQhVdexdgGgnovdIw3DWnwYXEdNjRmspisHqT+Rf9VWqfRcB6MYt/j/pYpnt C1D9360KWx83ITXq8XGCYCKsqGELXIT2Ma3/V3/zvgq0ObDPWDgS9A/dKJhKiPq2z60d NVLINqlVZ6mgI7l3YVqqeYU4/74j1Pt9g6Q5UsVIUPYXxTrTmHAZCicLCu4QSNHZPWh+ Coyhd6i5KfpGRYpRZtXyQmSLxi7dekj+96A2aCGiAJyUw7v/NQahFByGO9S4oHNarjhs p2qEaHLYZX5BLlthpd7m9cz46l8mCi4ShKKCsmIhdMEZkrABOLpdJ/29P3dKaD1oBKPx shWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700069864; x=1700674664; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wg6IcMPinfxxVmG7Rw0rCvEQaQgxpQ0/U8bjgfX6Jak=; b=ks7bFQxNEgafKtONUJZE+zS+jOdY7U9nb/sOipQfj8F4RduScD07BGeBWRb6R+6MKV kFR2gsbAtDSLJOMKMjins0P3WWMaaHb6cri7m4yF7vAqsoZYEN2ja27v4FqYr2Ijn4ZI 5C/1VXbNgbprUS6bWEIZTaCr/Y7paXsOR9Niv4qdxi0MxXFBk3i44U3ILK+jLruyw757 qu90LDPMRMkKaqeetp99sSmqVXkLJYrtCRme3qtNFKMYbX9se5+oyIyi2X7gEk651idz U/1blu89BTJLSzKlO8qt+VlE7nEeJrNiWGJsAozjWK57v2RBikjY9IUnlxC14yA4/8lg bUhw== X-Gm-Message-State: AOJu0YyoKPwFUqiAK4wdNZS18SaNBevoHd3LnYA69SC4oojmX3TgyCUw B5vLhWKFxLEwYYWa9W3Hqvt5TtnnLw== X-Google-Smtp-Source: AGHT+IESiHQdNuF3/8va/IQmw1lth2LHHkQoi6KWb2uwv7CXolbOGRNkH+NszfBUvCm3oJEQGcSHKg== X-Received: by 2002:a05:620a:199d:b0:779:efb4:73ad with SMTP id bm29-20020a05620a199d00b00779efb473admr6973423qkb.53.1700069863679; Wed, 15 Nov 2023 09:37:43 -0800 (PST) Received: from citadel.lan ([2600:6c4a:4d3f:6d5c::1019]) by smtp.gmail.com with ESMTPSA id x20-20020a05620a0b5400b0077bda014d8esm3260099qkg.87.2023.11.15.09.37.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Nov 2023 09:37:43 -0800 (PST) From: Brian Gerst To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: Ingo Molnar , Thomas Gleixner , Borislav Petkov , "H . Peter Anvin" , Peter Zijlstra , Uros Bizjak , David.Laight@aculab.com, Brian Gerst Subject: [PATCH v3 14/14] kallsyms: Remove KALLSYMS_ABSOLUTE_PERCPU Date: Wed, 15 Nov 2023 12:37:08 -0500 Message-ID: <20231115173708.108316-15-brgerst@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231115173708.108316-1-brgerst@gmail.com> References: <20231115173708.108316-1-brgerst@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" x86-64 was the only user. Signed-off-by: Brian Gerst --- init/Kconfig | 11 +----- kernel/kallsyms.c | 12 ++----- scripts/kallsyms.c | 80 ++++++++--------------------------------- scripts/link-vmlinux.sh | 4 --- 4 files changed, 18 insertions(+), 89 deletions(-) diff --git a/init/Kconfig b/init/Kconfig index 5f2c1f4a16aa..b55a8f237b24 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -1715,11 +1715,6 @@ config KALLSYMS_ALL =20 Say N unless you really need all symbols, or kernel live patching. =20 -config KALLSYMS_ABSOLUTE_PERCPU - bool - depends on KALLSYMS - default n - config KALLSYMS_BASE_RELATIVE bool depends on KALLSYMS @@ -1727,11 +1722,7 @@ config KALLSYMS_BASE_RELATIVE help Instead of emitting them as absolute values in the native word size, emit the symbol references in the kallsyms table as 32-bit entries, - each containing a relative value in the range [base, base + U32_MAX] - or, when KALLSYMS_ABSOLUTE_PERCPU is in effect, each containing either - an absolute value in the range [0, S32_MAX] or a relative value in the - range [base, base + S32_MAX], where base is the lowest relative symbol - address encountered in the image. + each containing a relative value in the range [base, base + U32_MAX]. =20 On 64-bit builds, this reduces the size of the address table by 50%, but more importantly, it results in entries whose values are build diff --git a/kernel/kallsyms.c b/kernel/kallsyms.c index 18edd57b5fe8..f4e8e531052a 100644 --- a/kernel/kallsyms.c +++ b/kernel/kallsyms.c @@ -151,16 +151,8 @@ unsigned long kallsyms_sym_address(int idx) if (!IS_ENABLED(CONFIG_KALLSYMS_BASE_RELATIVE)) return kallsyms_addresses[idx]; =20 - /* values are unsigned offsets if --absolute-percpu is not in effect */ - if (!IS_ENABLED(CONFIG_KALLSYMS_ABSOLUTE_PERCPU)) - return kallsyms_relative_base + (u32)kallsyms_offsets[idx]; - - /* ...otherwise, positive offsets are absolute values */ - if (kallsyms_offsets[idx] >=3D 0) - return kallsyms_offsets[idx]; - - /* ...and negative offsets are relative to kallsyms_relative_base - 1 */ - return kallsyms_relative_base - 1 - kallsyms_offsets[idx]; + /* values are unsigned offsets */ + return kallsyms_relative_base + (u32)kallsyms_offsets[idx]; } =20 static void cleanup_symbol_name(char *s) diff --git a/scripts/kallsyms.c b/scripts/kallsyms.c index 653b92f6d4c8..501f978abf4b 100644 --- a/scripts/kallsyms.c +++ b/scripts/kallsyms.c @@ -5,8 +5,8 @@ * This software may be used and distributed according to the terms * of the GNU General Public License, incorporated herein by reference. * - * Usage: kallsyms [--all-symbols] [--absolute-percpu] - * [--base-relative] [--lto-clang] in.map > out.S + * Usage: kallsyms [--all-symbols] [--base-relative] [--lto-clang] + * in.map > out.S * * Table compression uses all the unused char codes on the symbols and * maps these to the most used substrings (tokens). For instance, it might @@ -37,7 +37,6 @@ struct sym_entry { unsigned int len; unsigned int seq; unsigned int start_pos; - unsigned int percpu_absolute; unsigned char sym[]; }; =20 @@ -55,14 +54,9 @@ static struct addr_range text_ranges[] =3D { #define text_range_text (&text_ranges[0]) #define text_range_inittext (&text_ranges[1]) =20 -static struct addr_range percpu_range =3D { - "__per_cpu_start", "__per_cpu_end", -1ULL, 0 -}; - static struct sym_entry **table; static unsigned int table_size, table_cnt; static int all_symbols; -static int absolute_percpu; static int base_relative; static int lto_clang; =20 @@ -75,7 +69,7 @@ static unsigned char best_table_len[256]; =20 static void usage(void) { - fprintf(stderr, "Usage: kallsyms [--all-symbols] [--absolute-percpu] " + fprintf(stderr, "Usage: kallsyms [--all-symbols] " "[--base-relative] [--lto-clang] in.map > out.S\n"); exit(1); } @@ -167,7 +161,6 @@ static struct sym_entry *read_symbol(FILE *in, char **b= uf, size_t *buf_len) return NULL; =20 check_symbol_range(name, addr, text_ranges, ARRAY_SIZE(text_ranges)); - check_symbol_range(name, addr, &percpu_range, 1); =20 /* include the type field in the symbol name, so that it gets * compressed together */ @@ -183,7 +176,6 @@ static struct sym_entry *read_symbol(FILE *in, char **b= uf, size_t *buf_len) sym->len =3D len; sym->sym[0] =3D type; strcpy(sym_name(sym), name); - sym->percpu_absolute =3D 0; =20 return sym; } @@ -334,11 +326,6 @@ static int expand_symbol(const unsigned char *data, in= t len, char *result) return total; } =20 -static int symbol_absolute(const struct sym_entry *s) -{ - return s->percpu_absolute; -} - static void cleanup_symbol_name(char *s) { char *p; @@ -499,30 +486,17 @@ static void write_src(void) */ =20 long long offset; - int overflow; - - if (!absolute_percpu) { - offset =3D table[i]->addr - relative_base; - overflow =3D (offset < 0 || offset > UINT_MAX); - } else if (symbol_absolute(table[i])) { - offset =3D table[i]->addr; - overflow =3D (offset < 0 || offset > INT_MAX); - } else { - offset =3D relative_base - table[i]->addr - 1; - overflow =3D (offset < INT_MIN || offset >=3D 0); - } - if (overflow) { + + offset =3D table[i]->addr - relative_base; + if (offset < 0 || offset > UINT_MAX) { fprintf(stderr, "kallsyms failure: " - "%s symbol value %#llx out of range in relative mode\n", - symbol_absolute(table[i]) ? "absolute" : "relative", + "symbol value %#llx out of range in relative mode\n", table[i]->addr); exit(EXIT_FAILURE); } printf("\t.long\t%#x /* %s */\n", (int)offset, table[i]->sym); - } else if (!symbol_absolute(table[i])) { - output_address(table[i]->addr); } else { - printf("\tPTR\t%#llx\n", table[i]->addr); + output_address(table[i]->addr); } } printf("\n"); @@ -775,36 +749,15 @@ static void sort_symbols(void) qsort(table, table_cnt, sizeof(table[0]), compare_symbols); } =20 -static void make_percpus_absolute(void) -{ - unsigned int i; - - for (i =3D 0; i < table_cnt; i++) - if (symbol_in_range(table[i], &percpu_range, 1)) { - /* - * Keep the 'A' override for percpu symbols to - * ensure consistent behavior compared to older - * versions of this tool. - */ - table[i]->sym[0] =3D 'A'; - table[i]->percpu_absolute =3D 1; - } -} - -/* find the minimum non-absolute symbol address */ +/* find the minimum symbol address */ static void record_relative_base(void) { - unsigned int i; - - for (i =3D 0; i < table_cnt; i++) - if (!symbol_absolute(table[i])) { - /* - * The table is sorted by address. - * Take the first non-absolute symbol value. - */ - relative_base =3D table[i]->addr; - return; - } + /* + * The table is sorted by address. + * Take the first symbol value. + */ + if (table_cnt) + relative_base =3D table[0]->addr; } =20 int main(int argc, char **argv) @@ -812,7 +765,6 @@ int main(int argc, char **argv) while (1) { static const struct option long_options[] =3D { {"all-symbols", no_argument, &all_symbols, 1}, - {"absolute-percpu", no_argument, &absolute_percpu, 1}, {"base-relative", no_argument, &base_relative, 1}, {"lto-clang", no_argument, <o_clang, 1}, {}, @@ -831,8 +783,6 @@ int main(int argc, char **argv) =20 read_map(argv[optind]); shrink_table(); - if (absolute_percpu) - make_percpus_absolute(); sort_symbols(); if (base_relative) record_relative_base(); diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh index a432b171be82..d25b6d5de45e 100755 --- a/scripts/link-vmlinux.sh +++ b/scripts/link-vmlinux.sh @@ -148,10 +148,6 @@ kallsyms() kallsymopt=3D"${kallsymopt} --all-symbols" fi =20 - if is_enabled CONFIG_KALLSYMS_ABSOLUTE_PERCPU; then - kallsymopt=3D"${kallsymopt} --absolute-percpu" - fi - if is_enabled CONFIG_KALLSYMS_BASE_RELATIVE; then kallsymopt=3D"${kallsymopt} --base-relative" fi --=20 2.41.0