From nobody Fri May 3 04:42:38 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2987EC433F5 for ; Wed, 8 Dec 2021 11:08:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232071AbhLHLMB (ORCPT ); Wed, 8 Dec 2021 06:12:01 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50314 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232113AbhLHLMA (ORCPT ); Wed, 8 Dec 2021 06:12:00 -0500 Received: from mail-pf1-x433.google.com (mail-pf1-x433.google.com [IPv6:2607:f8b0:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 477E0C061746 for ; Wed, 8 Dec 2021 03:08:29 -0800 (PST) Received: by mail-pf1-x433.google.com with SMTP id g18so2159783pfk.5 for ; Wed, 08 Dec 2021 03:08:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=TnQefY0SqcDF3kPq8KJwIjgVaMNVB8DughUGvzU8o18=; b=ZkLOIb1g7XpNcLLrO2xyloqg63+eDW8c1N54mOfbRuFF15rJgWz2gE91JKdJydbYqG 8eL+r+UhBdoKdktkRc6tlgzDzsYJ6NEY/1TBuKklxnRsWEthwhlQQghbv4vM7av0dghG YG6zw3TvtD0aPRSriB0gRBQvxZg2vVXXuprrdD3o0j3Qbt2CneEC83iXlxyIWawDUhZw D0iIjqE7yyGv7egqdw5VoqBiJosZhNCpE68DlbCNteQlDu4Axl6SfUW4B/3Jil8jexsd tyJwF/W/PbqJoQPqUimewZeAPGV5Cx7WcudydaUIl40hyGzVhigr/W2WRHOIV/UiSSyJ MqXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TnQefY0SqcDF3kPq8KJwIjgVaMNVB8DughUGvzU8o18=; b=aBhQ7MjgcLwDq90Ao2V+LAJK/f5dM2zFVNJ4jgj1alYDnLpNn7lsi9vVCcgfEabqWw mqvmkhQEAXOQsPzDBGo6UMlpKQxj+EVRRpOkN1cSycg1FjWM0iNKrofCAWAFV6msgkXP fqPBUAEMKJGDl0o4ADKkYiQaVnWxg462oAKT2cGJLIFILWY4MjPVU/ujj+M/fdhG0JWj gCeJ5NJ8ek/F3YnQP0aLtQXYZaffm6KtAMTBmMrSZB4O3iz9h+A1RiFfsPOkvPmQuLh9 cmcGC5EyeAgvnAhhauyX7itdHSfVG/2oQq6mEzwctfUFXZ79Iw1cj+9KwHxX58Q3OQcF /N9A== X-Gm-Message-State: AOAM530t5dG5X92hwccETX2e9XLoqs2xCKrzNKQihbfI4WmxHQFLGMyX BoUHm5vaRE59BAnzffxQGzQBCivLWbc= X-Google-Smtp-Source: ABdhPJy9TsqtHV5JdjlSkcLb9NlmtdqMIVu0NVHyePzQqWTpNS604qrVcpVV8sNyEJaBLMfE3D+HqQ== X-Received: by 2002:a05:6a00:a18:b0:4a7:ef65:ddfb with SMTP id p24-20020a056a000a1800b004a7ef65ddfbmr4953445pfh.17.1638961708633; Wed, 08 Dec 2021 03:08:28 -0800 (PST) Received: from localhost ([47.251.3.230]) by smtp.gmail.com with ESMTPSA id b10sm2960927pft.179.2021.12.08.03.08.27 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 08 Dec 2021 03:08:28 -0800 (PST) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, xen-devel@lists.xenproject.org, Lai Jiangshan , Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" Subject: [PATCH 01/11] x86/entry: Use swapgs and native_iret directly in swapgs_restore_regs_and_return_to_usermode Date: Wed, 8 Dec 2021 19:08:23 +0800 Message-Id: <20211208110833.65366-2-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20211208110833.65366-1-jiangshanlai@gmail.com> References: <20211208110833.65366-1-jiangshanlai@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Lai Jiangshan swapgs_restore_regs_and_return_to_usermode() is used in native code (non-xenpv) only now, so it doesn't need the PV-aware SWAPGS and INTERRUPT_RETURN. Signed-off-by: Lai Jiangshan Reviewed-by: Juergen Gross --- arch/x86/entry/entry_64.S | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index 97b1f84bb53f..d95a02ecee16 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -608,8 +608,8 @@ SYM_INNER_LABEL(swapgs_restore_regs_and_return_to_userm= ode, SYM_L_GLOBAL) =20 /* Restore RDI. */ popq %rdi - SWAPGS - INTERRUPT_RETURN + swapgs + jmp native_iret =20 =20 SYM_INNER_LABEL(restore_regs_and_return_to_kernel, SYM_L_GLOBAL) --=20 2.19.1.6.gb485710b From nobody Fri May 3 04:42:38 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06B6EC433F5 for ; Wed, 8 Dec 2021 11:08:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232088AbhLHLMK (ORCPT ); Wed, 8 Dec 2021 06:12:10 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50358 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232078AbhLHLMJ (ORCPT ); Wed, 8 Dec 2021 06:12:09 -0500 Received: from mail-pj1-x1036.google.com (mail-pj1-x1036.google.com [IPv6:2607:f8b0:4864:20::1036]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 828AEC061746 for ; Wed, 8 Dec 2021 03:08:37 -0800 (PST) Received: by mail-pj1-x1036.google.com with SMTP id w33-20020a17090a6ba400b001a722a06212so3874077pjj.0 for ; Wed, 08 Dec 2021 03:08:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=v0Rti8J9O/DOw+Q8wf2xojRDpLT8xHVZ9Lxf2e0cKZk=; b=PsLrL5Aie1ZtnPQWX5QQBbXXhbTKSUKu0PazbTlH+ol9j3w7v0vsbVraLGwI8/WUBo 4JFmDJF1Et2cG6j/knBCajQznDaPYEV9uSmcNxrp0vZqTMFNZiS2SDO75XkRhd5Vvo9d iLJGeDUeMxGOuQ9nEeBVyRfyOPmkTlLH91yzjTTVSbHnK38Z+QOAMnm5Va2wTSFbnJfJ L8u2+h3MCb0UMPsvd3lGAk9BW7XraGPnmMNDp1DmWaALPZBg//tjNUNJfUiPJyE54GZA uoM15nO2f6/MMfuxIpBsbi2/4pmj+7Z/2GspSMt57V6sUf7bF36VCpuwxizpG3+DZLvh vyew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=v0Rti8J9O/DOw+Q8wf2xojRDpLT8xHVZ9Lxf2e0cKZk=; b=Ob0BhjMtRvgVZ/zE7sqLxNm60+65k4n+/hURivfWQ3y5yLz2zxC6MqV0JB+JWSnpMT kpBFVbhXu1isULHJihmq3gJAdOEHXaLyk9bDyEkurqgm5R0S489y7wCH0CtsFX6354gV NXkKAYAAIMGMtXBtfr27CuvmEcwMHUV5LCsyNDmswoONUyBXTNDOVWtlKVrlOE/vmKcM 8r1q60o98KErymKyEOxRk7iGsE/7rH8rA3XxQxJIHevDixtfsJLHwPBS6U+khwht1xER zSJc94krXxByVckFpp+ULJ+jQ92DDgwslacco9M/2XYuaBD5G35lgCBjZiEN6ucoup07 HS/g== X-Gm-Message-State: AOAM5305sYfQfY32o9Ptl+MTxXVPqUClA9ciCKUH1I/ZQAuT+CT8+Ujz aQgSOOg6A4yUMPqeQ1zRSFEUslf0nhI= X-Google-Smtp-Source: ABdhPJxpeELdxfo0JipJhaLLz4786jh3lr2fXOrOICnzD3O4qx8UMsz1Z1IGRVnoqDN8NDdQgTJz/g== X-Received: by 2002:a17:90b:3889:: with SMTP id mu9mr6279578pjb.160.1638961716971; Wed, 08 Dec 2021 03:08:36 -0800 (PST) Received: from localhost ([47.251.3.230]) by smtp.gmail.com with ESMTPSA id e29sm2222262pge.17.2021.12.08.03.08.35 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 08 Dec 2021 03:08:36 -0800 (PST) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, xen-devel@lists.xenproject.org, Lai Jiangshan , Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Peter Zijlstra , Joerg Roedel , "Chang S. Bae" Subject: [PATCH 02/11] x86/traps: Move pt_regs only in fixup_bad_iret() Date: Wed, 8 Dec 2021 19:08:24 +0800 Message-Id: <20211208110833.65366-3-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20211208110833.65366-1-jiangshanlai@gmail.com> References: <20211208110833.65366-1-jiangshanlai@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Lai Jiangshan fixup_bad_iret() and sync_regs() have similar arguments and do similar work that copies full or partial pt_regs to a place and switches stack after return. They are quite the same, but fixup_bad_iret() not only copies the pt_regs but also the return address of error_entry() while sync_regs() copies the pt_regs only and the return address of error_entry() was preserved and handled in ASM code. This patch makes fixup_bad_iret() work like sync_regs() and the handling of the return address of error_entry() is moved in ASM code. It removes the need to use the struct bad_iret_stack, simplifies fixup_bad_iret() and makes the ASM error_entry() call fixup_bad_iret() as the same as calling sync_regs() which adds readability because the calling patterns are exactly the same. It is prepared for later patch to do the stack switch after the error_entry() which simplifies the code further. Signed-off-by: Lai Jiangshan --- arch/x86/entry/entry_64.S | 5 ++++- arch/x86/include/asm/traps.h | 2 +- arch/x86/kernel/traps.c | 17 ++++++----------- 3 files changed, 11 insertions(+), 13 deletions(-) diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index d95a02ecee16..f8faff3ca61e 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -1040,9 +1040,12 @@ SYM_CODE_START_LOCAL(error_entry) * Pretend that the exception came from user mode: set up pt_regs * as if we faulted immediately after IRET. */ - mov %rsp, %rdi + popq %r12 /* save return addr in %12 */ + movq %rsp, %rdi /* arg0 =3D pt_regs pointer */ call fixup_bad_iret mov %rax, %rsp + ENCODE_FRAME_POINTER + pushq %r12 jmp .Lerror_entry_from_usermode_after_swapgs SYM_CODE_END(error_entry) =20 diff --git a/arch/x86/include/asm/traps.h b/arch/x86/include/asm/traps.h index 6221be7cafc3..1cdd7e8bcba7 100644 --- a/arch/x86/include/asm/traps.h +++ b/arch/x86/include/asm/traps.h @@ -13,7 +13,7 @@ #ifdef CONFIG_X86_64 asmlinkage __visible notrace struct pt_regs *sync_regs(struct pt_regs *ere= gs); asmlinkage __visible notrace -struct bad_iret_stack *fixup_bad_iret(struct bad_iret_stack *s); +struct pt_regs *fixup_bad_iret(struct pt_regs *bad_regs); void __init trap_init(void); asmlinkage __visible noinstr struct pt_regs *vc_switch_off_ist(struct pt_r= egs *eregs); #endif diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index c9d566dcf89a..a6c0bc9ee36f 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -761,13 +761,8 @@ asmlinkage __visible noinstr struct pt_regs *vc_switch= _off_ist(struct pt_regs *r } #endif =20 -struct bad_iret_stack { - void *error_entry_ret; - struct pt_regs regs; -}; - asmlinkage __visible noinstr -struct bad_iret_stack *fixup_bad_iret(struct bad_iret_stack *s) +struct pt_regs *fixup_bad_iret(struct pt_regs *bad_regs) { /* * This is called from entry_64.S early in handling a fault @@ -777,19 +772,19 @@ struct bad_iret_stack *fixup_bad_iret(struct bad_iret= _stack *s) * just below the IRET frame) and we want to pretend that the * exception came from the IRET target. */ - struct bad_iret_stack tmp, *new_stack =3D - (struct bad_iret_stack *)__this_cpu_read(cpu_tss_rw.x86_tss.sp0) - 1; + struct pt_regs tmp, *new_stack =3D + (struct pt_regs *)__this_cpu_read(cpu_tss_rw.x86_tss.sp0) - 1; =20 /* Copy the IRET target to the temporary storage. */ - __memcpy(&tmp.regs.ip, (void *)s->regs.sp, 5*8); + __memcpy(&tmp.ip, (void *)bad_regs->sp, 5*8); =20 /* Copy the remainder of the stack from the current stack. */ - __memcpy(&tmp, s, offsetof(struct bad_iret_stack, regs.ip)); + __memcpy(&tmp, bad_regs, offsetof(struct pt_regs, ip)); =20 /* Update the entry stack */ __memcpy(new_stack, &tmp, sizeof(tmp)); =20 - BUG_ON(!user_mode(&new_stack->regs)); + BUG_ON(!user_mode(new_stack)); return new_stack; } #endif --=20 2.19.1.6.gb485710b From nobody Fri May 3 04:42:38 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F9B0C433EF for ; Wed, 8 Dec 2021 11:08:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232103AbhLHLMQ (ORCPT ); Wed, 8 Dec 2021 06:12:16 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50382 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232078AbhLHLMO (ORCPT ); Wed, 8 Dec 2021 06:12:14 -0500 Received: from mail-pf1-x432.google.com (mail-pf1-x432.google.com [IPv6:2607:f8b0:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 05557C061746 for ; Wed, 8 Dec 2021 03:08:43 -0800 (PST) Received: by mail-pf1-x432.google.com with SMTP id k26so2128548pfp.10 for ; Wed, 08 Dec 2021 03:08:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=74fcl84eMF3trMMN2JVz1x4AFBWc9HU1GFAGLmUM5eE=; b=VKlr4hE6P3/KVxbIhcLJDA0sF52nL0lnTLSlQue1EOi6nOPZmlq7+N6t34Z4nlAn6S 6NYswFiLZFVt72m7dLmKsXjcv49Tzn4Vfnpcat4WRibMq/7lLfmYUvxqlDrF2qI/vqdA 5aiicTqsTeo5rZI/iqEDgXPe0X06ShRim5daAufoj+27dBC6vjm3F6QWqdhhrBeccdlx vioUF/TKnCwOTAFHjtjXkA2ZHEBTUJBfBC9YS2QXCQak02LJEkZyHt3r3mxI7EG3xklk 0XSnYu5Z2L1fze0kQgB0lt62j14Jn3CZPbd5zi5beWbTOrpQnWraqlF9nD9BmgP3hxz8 xxDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=74fcl84eMF3trMMN2JVz1x4AFBWc9HU1GFAGLmUM5eE=; b=yx0NqUKp1QeKqGOirE0FASlMt/WQ49qR23OG3klt3vCsTgcO41GNwBUmKqcU7Kbmjx JKLX5H+qabTOeltc5jU3uVmhT4LIT+f3kXf3IMZFnqFApkpP8t824NTt2Ag6K9Nukcw7 f5CdXSu7DnFY1bj/3or3Rlu0H7FVnd0a4PeGNt1d+l4/j9V9PTkag+DAz66rhRkLhvEL +KxjQFdVQPT49kDNjoNAAmMGNPiX+5TLu/iSyxVIdA/1EJvXEkzu1TslLe9rfyilx2oN Gb73FPECXpsf1ugTkUVpFxZmOz7Kl1+bLECt4NRurcx+Lj99P5OSCt8OH1+Q7+NvbZMW zLqQ== X-Gm-Message-State: AOAM532KdX5meCGviIANW2ME2UCJpHPtkDczEOxpLocXECmy9qtbzr+y onLrwj7txWUG0msqIq9x0rdhaBAimXI= X-Google-Smtp-Source: ABdhPJwdwD0NWVFtmrZKBdCkBw1nVwlMCHpLBoT9j05zLnwJyoppx7uciS4Kg9RdTDjqkCWrZh05WA== X-Received: by 2002:a63:e43:: with SMTP id 3mr29449478pgo.73.1638961722423; Wed, 08 Dec 2021 03:08:42 -0800 (PST) Received: from localhost ([47.251.3.230]) by smtp.gmail.com with ESMTPSA id a16sm2254924pgm.57.2021.12.08.03.08.41 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 08 Dec 2021 03:08:41 -0800 (PST) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, xen-devel@lists.xenproject.org, Lai Jiangshan , Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" Subject: [PATCH 03/11] x86/entry: Switch the stack after error_entry() returns Date: Wed, 8 Dec 2021 19:08:25 +0800 Message-Id: <20211208110833.65366-4-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20211208110833.65366-1-jiangshanlai@gmail.com> References: <20211208110833.65366-1-jiangshanlai@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Lai Jiangshan error_entry() calls sync_regs() to settle/copy the pt_regs and switches the stack directly after sync_regs(). But error_entry() itself is also a function call, the switching has to handle the return address of it together, which causes the work complicated and tangly. Switching to the stack after error_entry() makes the code simpler and intuitive. Signed-off-by: Lai Jiangshan --- arch/x86/entry/entry_64.S | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-) diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index f8faff3ca61e..6baa7413f7eb 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -323,6 +323,8 @@ SYM_CODE_END(ret_from_fork) .macro idtentry_body cfunc has_error_code:req =20 call error_entry + movq %rax, %rsp /* switch stack settled by sync_regs() */ + ENCODE_FRAME_POINTER UNWIND_HINT_REGS =20 movq %rsp, %rdi /* pt_regs pointer into 1st argument*/ @@ -981,14 +983,10 @@ SYM_CODE_START_LOCAL(error_entry) /* We have user CR3. Change to kernel CR3. */ SWITCH_TO_KERNEL_CR3 scratch_reg=3D%rax =20 + leaq 8(%rsp), %rdi /* arg0 =3D pt_regs pointer */ .Lerror_entry_from_usermode_after_swapgs: /* Put us onto the real thread stack. */ - popq %r12 /* save return addr in %12 */ - movq %rsp, %rdi /* arg0 =3D pt_regs pointer */ call sync_regs - movq %rax, %rsp /* switch stack */ - ENCODE_FRAME_POINTER - pushq %r12 ret =20 /* @@ -1020,6 +1018,7 @@ SYM_CODE_START_LOCAL(error_entry) */ .Lerror_entry_done_lfence: FENCE_SWAPGS_KERNEL_ENTRY + leaq 8(%rsp), %rax /* return pt_regs pointer */ ret =20 .Lbstep_iret: @@ -1040,12 +1039,9 @@ SYM_CODE_START_LOCAL(error_entry) * Pretend that the exception came from user mode: set up pt_regs * as if we faulted immediately after IRET. */ - popq %r12 /* save return addr in %12 */ - movq %rsp, %rdi /* arg0 =3D pt_regs pointer */ + leaq 8(%rsp), %rdi /* arg0 =3D pt_regs pointer */ call fixup_bad_iret - mov %rax, %rsp - ENCODE_FRAME_POINTER - pushq %r12 + mov %rax, %rdi jmp .Lerror_entry_from_usermode_after_swapgs SYM_CODE_END(error_entry) =20 --=20 2.19.1.6.gb485710b From nobody Fri May 3 04:42:38 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 10431C433F5 for ; Wed, 8 Dec 2021 11:08:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232119AbhLHLMV (ORCPT ); Wed, 8 Dec 2021 06:12:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50412 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232078AbhLHLMU (ORCPT ); Wed, 8 Dec 2021 06:12:20 -0500 Received: from mail-pg1-x52d.google.com (mail-pg1-x52d.google.com [IPv6:2607:f8b0:4864:20::52d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CABE5C061746 for ; Wed, 8 Dec 2021 03:08:48 -0800 (PST) Received: by mail-pg1-x52d.google.com with SMTP id g16so1795239pgi.1 for ; Wed, 08 Dec 2021 03:08:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=RcBSE4oPibzeIhyf2Qp5YqTgR/bF+VmAeF/brxTC5FU=; b=D0KfkWiPHhKhRJUZdR5kq7I4HpNBJdRwa55sv9BMfhZ3vbJVIPNqKGnE1ONGnrlI34 2uQvzZB8c1W/Hzmk9lJIBvPuBZIZBDzlhZ6uIuU9VZOKqWnOAHB9aMyyXNK8KcmbcQV/ bQjVtI9HtkySFmOwWp9onHdlvBxih0vvgFOJHMQjs9U8tAoDeS0Tt29U9Ye6sau5kAxz J/DeQH5dwbfQl7bRYn+aIrOkMti5RoJtWalneYj8OgUE/4cpA9PctgdhPmFe30GA+AfV HgbSnUq6EfLPRCS7X+q0+QvEnT3ZXqXLMRNwgKgKcUcvvo/VDdu5ix0k/Mo8225S/GHA bJ1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RcBSE4oPibzeIhyf2Qp5YqTgR/bF+VmAeF/brxTC5FU=; b=5oDK+0WIlByCkmqEGzFYy+aIxoKR7kAD39YQwJcny9W78I5INr6o4DBaRIRXRNtN6D paUrAlSTkZT3ewFpXUA8w9BjXLPIe6qU+OGSvu2JyrI3iu1jVRdyGVx3J/0BMBKk/ONj k+xOujcOpOlJP1XqNuJV2YmFaESCseZnoqAFKvUFVoU3zyLemc/pNiCuKNVAJ7wuZpgs XUfNmnYE+eeT7elTxTa+IhEYoF/aexfPfpfq5AD5hcWstgjNMn2GwtjrLa5gQTMCwcXa iNMQ4k5FjxcixNbJTXZsJIA+23BGM5HNZZ6WqQmUNrFa5aup3jeJcdk7CvOafD50WAIN 9lvg== X-Gm-Message-State: AOAM533yI9ZQbz+JhN2fKsOb6tlCXwXqd5Wqk0TR0Z06XYg6xe/WAvOi indny3XkV70UmUzxSElnO05aYo/3cKs= X-Google-Smtp-Source: ABdhPJwB9xEYc6yaPMgbaAjXzRhRbjCMiDT7xaBXucJ4jA6rNMMEh2pDhUz6bzCHuWkJVnkpO7Wxhg== X-Received: by 2002:a05:6a00:1a8e:b0:49f:a4a9:8f1e with SMTP id e14-20020a056a001a8e00b0049fa4a98f1emr4890452pfv.67.1638961727985; Wed, 08 Dec 2021 03:08:47 -0800 (PST) Received: from localhost ([47.251.3.230]) by smtp.gmail.com with ESMTPSA id p17sm3404857pfh.86.2021.12.08.03.08.47 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 08 Dec 2021 03:08:47 -0800 (PST) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, xen-devel@lists.xenproject.org, Lai Jiangshan , Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" Subject: [PATCH 04/11] x86/entry: move PUSH_AND_CLEAR_REGS out of error_entry Date: Wed, 8 Dec 2021 19:08:26 +0800 Message-Id: <20211208110833.65366-5-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20211208110833.65366-1-jiangshanlai@gmail.com> References: <20211208110833.65366-1-jiangshanlai@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Lai Jiangshan Moving PUSH_AND_CLEAR_REGS out of error_entry doesn't change any functionality. It will enlarge the size: size arch/x86/entry/entry_64.o.before: text data bss dec hex filename 17916 384 0 18300 477c arch/x86/entry/entry_64.o size --format=3DSysV arch/x86/entry/entry_64.o.before: .entry.text 5528 0 .orc_unwind 6456 0 .orc_unwind_ip 4304 0 size arch/x86/entry/entry_64.o.after: text data bss dec hex filename 26868 384 0 27252 6a74 arch/x86/entry/entry_64.o size --format=3DSysV arch/x86/entry/entry_64.o.after: .entry.text 8200 0 .orc_unwind 10224 0 .orc_unwind_ip 6816 0 But .entry.text in x86_64 is 2M aligned, enlarging it to 8.2k doesn't enlarge the final text size. The tables .orc_unwind[_ip] are enlarged due to it adds many pushes. It is prepared for not calling error_entry() from XENPV in later patch and for future converting the whole error_entry into C code. Signed-off-by: Lai Jiangshan --- arch/x86/entry/entry_64.S | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index 6baa7413f7eb..72ceb4b70822 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -322,6 +322,9 @@ SYM_CODE_END(ret_from_fork) */ .macro idtentry_body cfunc has_error_code:req =20 + PUSH_AND_CLEAR_REGS + ENCODE_FRAME_POINTER + call error_entry movq %rax, %rsp /* switch stack settled by sync_regs() */ ENCODE_FRAME_POINTER @@ -969,8 +972,6 @@ SYM_CODE_END(paranoid_exit) SYM_CODE_START_LOCAL(error_entry) UNWIND_HINT_FUNC cld - PUSH_AND_CLEAR_REGS save_ret=3D1 - ENCODE_FRAME_POINTER 8 testb $3, CS+8(%rsp) jz .Lerror_kernelspace =20 --=20 2.19.1.6.gb485710b From nobody Fri May 3 04:42:38 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 02AABC433EF for ; Wed, 8 Dec 2021 11:08:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232124AbhLHLM1 (ORCPT ); Wed, 8 Dec 2021 06:12:27 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50442 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232111AbhLHLM0 (ORCPT ); Wed, 8 Dec 2021 06:12:26 -0500 Received: from mail-pf1-x42f.google.com (mail-pf1-x42f.google.com [IPv6:2607:f8b0:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7A95CC0617A1 for ; Wed, 8 Dec 2021 03:08:54 -0800 (PST) Received: by mail-pf1-x42f.google.com with SMTP id i12so2152798pfd.6 for ; Wed, 08 Dec 2021 03:08:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=94qtwsHdBUcVzdr14fQH4QqeaELVSejvb9k+dDp4o2k=; b=gdJwJexPSIx1/o0kGlFH/3siK2BWmqN/Qya2wWAwRpJBABJCf7cZVPY1vV9uKphzCM wDiL42S9zLojg/9rYkV+8Gcyug0bNkhyprI3HL5IVv6FPdbDYpG0Q+9f1fcLvsLyHqlJ 6d+3RCpxWxTSvIGQUyD3iKktmbGyZSKRYFR81PlrbYQogKQmi6fZOJTId9b93ecmVF2Q eT3UjTw/vZKqvqEL/qGLIrdAM1QDCdiqpnZVGVu3jSXv5teHhN5/T/g/zvUjMZ6G722x jjKFxOz0lcm/NslZErC/EI4Yi0rOzfoSzQ5g0N+HuIY4WjIjEBS0MHTo4NO9TZ3LRbFW TNpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=94qtwsHdBUcVzdr14fQH4QqeaELVSejvb9k+dDp4o2k=; b=p96UbQyolJtXZHYWZ0/Q7KRDnL7fTR+VbaDAbC3BI6AOPvy/mubI34jrgBq+/P81Z9 BXOGPpLWeSa/artrIBZZxRCHrB0Tx9BpjdQ4bNVRzI+DnDzB8lGOlbM7dfOmnaNby7jh liEG5YF7ywWlC5m4/7w2G12Y+CU/SuVV60Z/EB4WjTOlmObNcp5f5nBt7PfUg0GslfMY 89zzFymXWvx7+ojHU6D4D1uISyXCFaexzALNzk6vaMWSRbYz70w8qz6mPHSMupx3nHu1 A84L/L7ZwwyhL86lYhI/u3CVJkySu2kVbMpFB5GeRcbckFQYAltMI1gTfmltUCv3HlBi WxFg== X-Gm-Message-State: AOAM5320YsXGpGMdjJHyTP64N6X9EnUYOEBfJMaljl6/XHlJER8cZr5S HJflzt6ZB+Wp2NBeKNAcDYvoiAbyhnE= X-Google-Smtp-Source: ABdhPJzMp2m1CIOe5Z9utqdinpuLznocfHk1h5+AWYqs9CoFg7FElyyIpiFD9TM9L0u194BTWXvhUw== X-Received: by 2002:a62:8449:0:b0:4ad:56ba:2f1d with SMTP id k70-20020a628449000000b004ad56ba2f1dmr4928492pfd.37.1638961733585; Wed, 08 Dec 2021 03:08:53 -0800 (PST) Received: from localhost ([47.251.3.230]) by smtp.gmail.com with ESMTPSA id d3sm3454696pfv.57.2021.12.08.03.08.52 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 08 Dec 2021 03:08:53 -0800 (PST) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, xen-devel@lists.xenproject.org, Lai Jiangshan , Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" Subject: [PATCH 05/11] x86/entry: Move cld to the start of idtentry Date: Wed, 8 Dec 2021 19:08:27 +0800 Message-Id: <20211208110833.65366-6-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20211208110833.65366-1-jiangshanlai@gmail.com> References: <20211208110833.65366-1-jiangshanlai@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Lai Jiangshan Make it next to CLAC Suggested-by: Peter Zijlstra Signed-off-by: Lai Jiangshan --- arch/x86/entry/entry_64.S | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index 72ceb4b70822..ee1d4adcdab0 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -356,6 +356,7 @@ SYM_CODE_END(ret_from_fork) SYM_CODE_START(\asmsym) UNWIND_HINT_IRET_REGS offset=3D\has_error_code*8 ASM_CLAC + cld =20 .if \has_error_code =3D=3D 0 pushq $-1 /* ORIG_RAX: no syscall to restart */ @@ -423,6 +424,7 @@ SYM_CODE_END(\asmsym) SYM_CODE_START(\asmsym) UNWIND_HINT_IRET_REGS ASM_CLAC + cld =20 pushq $-1 /* ORIG_RAX: no syscall to restart */ =20 @@ -478,6 +480,7 @@ SYM_CODE_END(\asmsym) SYM_CODE_START(\asmsym) UNWIND_HINT_IRET_REGS ASM_CLAC + cld =20 /* * If the entry is from userspace, switch stacks and treat it as @@ -539,6 +542,7 @@ SYM_CODE_END(\asmsym) SYM_CODE_START(\asmsym) UNWIND_HINT_IRET_REGS offset=3D8 ASM_CLAC + cld =20 /* paranoid_entry returns GS information for paranoid_exit in EBX. */ call paranoid_entry @@ -853,7 +857,6 @@ SYM_CODE_END(xen_failsafe_callback) */ SYM_CODE_START_LOCAL(paranoid_entry) UNWIND_HINT_FUNC - cld PUSH_AND_CLEAR_REGS save_ret=3D1 ENCODE_FRAME_POINTER 8 =20 @@ -971,7 +974,6 @@ SYM_CODE_END(paranoid_exit) */ SYM_CODE_START_LOCAL(error_entry) UNWIND_HINT_FUNC - cld testb $3, CS+8(%rsp) jz .Lerror_kernelspace =20 @@ -1104,6 +1106,7 @@ SYM_CODE_START(asm_exc_nmi) */ =20 ASM_CLAC + cld =20 /* Use %rdx as our temp variable throughout */ pushq %rdx @@ -1123,7 +1126,6 @@ SYM_CODE_START(asm_exc_nmi) */ =20 swapgs - cld FENCE_SWAPGS_USER_ENTRY SWITCH_TO_KERNEL_CR3 scratch_reg=3D%rdx movq %rsp, %rdx --=20 2.19.1.6.gb485710b From nobody Fri May 3 04:42:38 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C8EC4C4332F for ; Wed, 8 Dec 2021 11:09:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229503AbhLHLMk (ORCPT ); Wed, 8 Dec 2021 06:12:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50468 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232145AbhLHLMb (ORCPT ); Wed, 8 Dec 2021 06:12:31 -0500 Received: from mail-pf1-x42d.google.com (mail-pf1-x42d.google.com [IPv6:2607:f8b0:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A02B4C061A72 for ; Wed, 8 Dec 2021 03:08:59 -0800 (PST) Received: by mail-pf1-x42d.google.com with SMTP id r130so2189162pfc.1 for ; Wed, 08 Dec 2021 03:08:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=GTzYOcv7GXMsuudeRCCVKvQE4p4RTznbhZKVu0xYKUU=; b=km30WDN3p/x6O1LN4IM4fy77QF1NVRUPK5sP06Gal6+QGtjqpUUYFI0ArnBEImqtBx AuUANxaE5TNvqfzwlIxrjFaBsxfD6p1eT/U6KemlwxP4SBJRnkOXwocKlRobPiXwphiK nnKNx8a17oUYe1aBF9xVYjEfT0JHyVlsyWpuNHWRttKft7eHMogyjA4hHEpDixYozui/ Yp9HWHVmMbUx0pUgEzC7X+43ukc3qPqWPrcwKNA92/m1IkWkKcYC2169qQXVTznVDh9C 8FdQ3N0QnfLfiyMM1qnRw3/LhI95W2mPf5Ae8J85boc7Vk+qIFVR2qqDY1HLZcu21xTg Tx+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=GTzYOcv7GXMsuudeRCCVKvQE4p4RTznbhZKVu0xYKUU=; b=rk51IO8Z2pFsBJOZLXmcVFC8a57RB3cRfZa2SI6WsMsg2WmcPL+0JVaNmZ1zGc8NLp GdrI/oPD7D8OCgjfdrck/x4Tx/VYKmKBdO1twosKy7DfNK7xf8jhUm7ZvNSed6jEg11R uLrHCPvpAWGj6rHSGoYJRuUGfJXvTQxv73jgdjE93USAlG+x4mCFnvDMAMSw4Uo2qe+z 9eVFe0YGnI7nXD3h6tg6pkj/fkFbpQDjOal0IB/+ff5qsE1cAlMVlFUhWzLUNwsR7OlQ Wp0WuodVosiek0whQIUvik+6PBMWB6AyE2iBIqVGoUl2GX/U1VycFkLpw3FIQMCF4KmS qj9A== X-Gm-Message-State: AOAM533wl+Tb82663nCre/EVHuAxNpTYjjD+XHKg/0iKUeS+OuW+k1Dv zN0+GRS3ANTeEEbkf7VTYGYk+s5dd7o= X-Google-Smtp-Source: ABdhPJyG8RspzExM0YLJHIGBovB6b/ZNv9oPLLxl7bIY/uiUpBpyPdvWlt/3qAb05wwKy/zxp0w/Lw== X-Received: by 2002:a05:6a00:ad0:b0:4a3:1056:e2ae with SMTP id c16-20020a056a000ad000b004a31056e2aemr4931125pfl.2.1638961739024; Wed, 08 Dec 2021 03:08:59 -0800 (PST) Received: from localhost ([47.251.3.230]) by smtp.gmail.com with ESMTPSA id gc22sm5791712pjb.57.2021.12.08.03.08.57 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 08 Dec 2021 03:08:58 -0800 (PST) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, xen-devel@lists.xenproject.org, Lai Jiangshan , Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" Subject: [PATCH 06/11] x86/entry: Don't call error_entry for XENPV Date: Wed, 8 Dec 2021 19:08:28 +0800 Message-Id: <20211208110833.65366-7-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20211208110833.65366-1-jiangshanlai@gmail.com> References: <20211208110833.65366-1-jiangshanlai@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Lai Jiangshan When in XENPV, it is already in the task stack, and it can't fault for native_iret() nor native_load_gs_index() since XENPV uses its own pvops for iret and load_gs_index(). And it doesn't need to switch CR3. So there is no reason to call error_entry() in XENPV. Signed-off-by: Lai Jiangshan --- arch/x86/entry/entry_64.S | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index ee1d4adcdab0..383070aa9272 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -325,8 +325,17 @@ SYM_CODE_END(ret_from_fork) PUSH_AND_CLEAR_REGS ENCODE_FRAME_POINTER =20 - call error_entry - movq %rax, %rsp /* switch stack settled by sync_regs() */ + /* + * Call error_entry and switch stack settled by sync_regs(). + * + * When in XENPV, it is already in the task stack, and it can't fault + * for native_iret() nor native_load_gs_index() since XENPV uses its + * own pvops for iret and load_gs_index(). And it doesn't need to + * switch CR3. So it can skip invoking error_entry(). + */ + ALTERNATIVE "call error_entry; movq %rax, %rsp", \ + "", X86_FEATURE_XENPV + ENCODE_FRAME_POINTER UNWIND_HINT_REGS =20 --=20 2.19.1.6.gb485710b From nobody Fri May 3 04:42:38 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EAC6BC433EF for ; Wed, 8 Dec 2021 11:09:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232141AbhLHLMl (ORCPT ); Wed, 8 Dec 2021 06:12:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50496 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230229AbhLHLMh (ORCPT ); Wed, 8 Dec 2021 06:12:37 -0500 Received: from mail-pj1-x1030.google.com (mail-pj1-x1030.google.com [IPv6:2607:f8b0:4864:20::1030]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5F7F2C061D5E for ; Wed, 8 Dec 2021 03:09:05 -0800 (PST) Received: by mail-pj1-x1030.google.com with SMTP id p18-20020a17090ad31200b001a78bb52876so4094359pju.3 for ; Wed, 08 Dec 2021 03:09:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8qfk8rEd7OE7FkctKlJZZ9kD8w0vOU1vAluUq/sa8Oc=; b=l7nygggwydXZUAu4zkqtbtTRWY39vFf8FLQk4PTAPrhXI+tzr6CLSDOcbjAIFholoR ZflKnIP1dr60YP3ogLki0mS5fzvW2s2sJ5JJjEOnEcnqaWFXcDIqGKfbpVG5umYF7IuV XbGokOAZ7qo1h+G1Bfk0/lmfvpWFvQF3DWBtRJl8/CnxVy53GK6huw6txhzHw8osD5Bs OMUKn0L37dEZLYVTT7lapANuDqunPxOpcQlA+VF0ffqg433s/FhOPxw6l+uZTJK2URXY u7aPuRJNHxAhjbmWX/mle+316jDesbf0dAlNAH5fyU9efV13TpPnGDNyw76aFmOqi4by 5IXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8qfk8rEd7OE7FkctKlJZZ9kD8w0vOU1vAluUq/sa8Oc=; b=IcrPkU8DE1IOlaMRLBaJEToZLrVmLR0lifRSkY1nm8wnLcL8p7m4wmZHEzvmXrGrAS hO+zjsVyGNVnz5vtgl2as5XFH8ezveqe/h71A7SKtJzNgilSiFEUaPMBpL71dX7X5x3J 5Dl90Jfmbz4ZYZf+DhEFmaVfo2ATTJcRmalyDPnJIHJ19WQRxBRXWL2rZyUoRNB6kFm8 54xtyR338nYZ1LGWHODPt0IKBQuuRaoXtm2W3C2tZ9Fyc3dAjjezvfn7IwtgI5offE8t htE6wUKGVJlY3DtmG2wQ9zuZjvK0N1xD19rWF8nqgkAxSByK4ADqwWVPSYRPuwaNUfqL 7SXQ== X-Gm-Message-State: AOAM531zbkHL5pGqHQgxMh0OD41NWPIoQz3Eun6MjSubu6UZDdh6iwPI PH0e/1IT30sbcpr0435180o4nQwzC3E= X-Google-Smtp-Source: ABdhPJyX9L5YCir717yBDgGNa36Bhmc2dDItA2/uZBarVsl66m2fKt+O9ZG0+5iKAc//bNxdDvNjmA== X-Received: by 2002:a17:90b:612:: with SMTP id gb18mr6498365pjb.0.1638961744681; Wed, 08 Dec 2021 03:09:04 -0800 (PST) Received: from localhost ([47.251.3.230]) by smtp.gmail.com with ESMTPSA id b18sm2455866pjo.31.2021.12.08.03.09.03 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 08 Dec 2021 03:09:04 -0800 (PST) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, xen-devel@lists.xenproject.org, Lai Jiangshan , Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" Subject: [PATCH 07/11] x86/entry: Convert SWAPGS to swapgs in error_entry() Date: Wed, 8 Dec 2021 19:08:29 +0800 Message-Id: <20211208110833.65366-8-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20211208110833.65366-1-jiangshanlai@gmail.com> References: <20211208110833.65366-1-jiangshanlai@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Lai Jiangshan XENPV doesn't use error_entry() anymore, so the pv-aware SWAPGS can be changed to native swapgs. Signed-off-by: Lai Jiangshan --- arch/x86/entry/entry_64.S | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index 383070aa9272..6100007fd03f 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -990,7 +990,7 @@ SYM_CODE_START_LOCAL(error_entry) * We entered from user mode or we're pretending to have entered * from user mode due to an IRET fault. */ - SWAPGS + swapgs FENCE_SWAPGS_USER_ENTRY /* We have user CR3. Change to kernel CR3. */ SWITCH_TO_KERNEL_CR3 scratch_reg=3D%rax @@ -1022,7 +1022,7 @@ SYM_CODE_START_LOCAL(error_entry) * gsbase and proceed. We'll fix up the exception and land in * .Lgs_change's error handler with kernel gsbase. */ - SWAPGS + swapgs =20 /* * Issue an LFENCE to prevent GS speculation, regardless of whether it is= a @@ -1043,7 +1043,7 @@ SYM_CODE_START_LOCAL(error_entry) * We came from an IRET to user mode, so we have user * gsbase and CR3. Switch to kernel gsbase and CR3: */ - SWAPGS + swapgs FENCE_SWAPGS_USER_ENTRY SWITCH_TO_KERNEL_CR3 scratch_reg=3D%rax =20 --=20 2.19.1.6.gb485710b From nobody Fri May 3 04:42:38 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4621DC433FE for ; Wed, 8 Dec 2021 11:09:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232194AbhLHLMu (ORCPT ); Wed, 8 Dec 2021 06:12:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50530 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232186AbhLHLMp (ORCPT ); Wed, 8 Dec 2021 06:12:45 -0500 Received: from mail-pf1-x431.google.com (mail-pf1-x431.google.com [IPv6:2607:f8b0:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C7444C0617A2 for ; Wed, 8 Dec 2021 03:09:13 -0800 (PST) Received: by mail-pf1-x431.google.com with SMTP id r130so2189621pfc.1 for ; Wed, 08 Dec 2021 03:09:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4OSMT+TQw2RUHxNW8atVbB9ZMIOeuqyP7ER0LvB+dp8=; b=p9amC8ioLdWqBeAJ4CjkDRZ1SNu2S6IWtNPrQgya84oZpDa9clfN6COADE//h/Kczz kQ+fmFgTFQoLYUpO3SxmFlpuPVleUj1yU37SK/cIZ0B+1mzWQP1bXTdmwTYJqBqgidym s3Qnw3+ezN8X07/YBXDOFJMTCWVdRYuXNtqsZb9MYstubPgHdxVejo3WZ/+NjwbHXH2U 8tjQd71AvCYyeR33oziwHtkT65rxg9yzijXFTi0+rKnlPvNRCFSokL+RsxQRBvOrVIrU w/GkvVUeb2F4qxGFfEyZK3vg73sGRS4uXqjhQZh5lj5xic5xJsV2YQ+0E79kCZaZeBK8 qj8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4OSMT+TQw2RUHxNW8atVbB9ZMIOeuqyP7ER0LvB+dp8=; b=Kr9AAgHDfS+yIIHkuqBxEjj0/87XYmZ/4LZASYCvzleyPuMwwSTehGi0RC5G50BnXq E4iudaH4tYDr7skoJF1Ohf5cV/zTYGegSivHsSFUWruAM/ElLBYpSbgp+968+AYbnuDf XULyMbDCOw9jph2yR1rZRH0GF64s1HkHw4IYFM+KRa7udkpm8HKTCR451fDWOnFabJQY UhiCmLWHmzgslkBtZM2aN8So1tXDx1NYbPiSn9zgd7pc24BcIl++q3TFZ013CMuQiQL4 4mEMCEo3VPE0SlrBysWz2ToYjMErEyZ+jD+dEZYL5m5mHqK6Zk1G2Sdp1BzqEcvMLlw3 pJ+Q== X-Gm-Message-State: AOAM530VrGJNv37M9YZ0t3i8BIO7oICwAGUxqZpUwW9waLyuIsQrpgEG S4GMLL3x2P2pUBs0OvNNn6y+TMeNSe4= X-Google-Smtp-Source: ABdhPJyXGPiRK3kSpyhR6KL61u0teYkSKfkL2XAKTjzosU2gZ8Lz/nBoNiGVciQ2kf7/fPZtckPd8w== X-Received: by 2002:a63:6a03:: with SMTP id f3mr28895124pgc.618.1638961752671; Wed, 08 Dec 2021 03:09:12 -0800 (PST) Received: from localhost ([47.251.3.230]) by smtp.gmail.com with ESMTPSA id q10sm3232415pfk.180.2021.12.08.03.09.11 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 08 Dec 2021 03:09:12 -0800 (PST) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, xen-devel@lists.xenproject.org, Lai Jiangshan , Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Juergen Gross , "Peter Zijlstra (Intel)" , Joerg Roedel , "Chang S. Bae" , Jan Kiszka Subject: [PATCH 08/11] x86/entry: Use idtentry macro for entry_INT80_compat Date: Wed, 8 Dec 2021 19:08:30 +0800 Message-Id: <20211208110833.65366-9-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20211208110833.65366-1-jiangshanlai@gmail.com> References: <20211208110833.65366-1-jiangshanlai@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Lai Jiangshan entry_INT80_compat is identical to idtentry macro except a special handling for %rax in the prolog. Add the prolog to idtentry and use idtentry for entry_INT80_compat. Signed-off-by: Lai Jiangshan --- arch/x86/entry/entry_64.S | 18 ++++++ arch/x86/entry/entry_64_compat.S | 102 ------------------------------- arch/x86/include/asm/idtentry.h | 47 ++++++++++++++ arch/x86/include/asm/proto.h | 4 -- 4 files changed, 65 insertions(+), 106 deletions(-) diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index 6100007fd03f..1fd5efaa99b3 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -371,6 +371,24 @@ SYM_CODE_START(\asmsym) pushq $-1 /* ORIG_RAX: no syscall to restart */ .endif =20 + .if \vector =3D=3D IA32_SYSCALL_VECTOR + /* + * User tracing code (ptrace or signal handlers) might assume + * that the saved RAX contains a 32-bit number when we're + * invoking a 32-bit syscall. Just in case the high bits are + * nonzero, zero-extend the syscall number. (This could almost + * certainly be deleted with no ill effects.) + */ + movl %eax, %eax + + /* + * do_int80_syscall_32() expects regs->orig_ax to be user ax, + * and regs->ax to be $-ENOSYS. + */ + movq %rax, (%rsp) + movq $-ENOSYS, %rax + .endif + .if \vector =3D=3D X86_TRAP_BP /* * If coming from kernel space, create a 6-word gap to allow the diff --git a/arch/x86/entry/entry_64_compat.S b/arch/x86/entry/entry_64_com= pat.S index 0051cf5c792d..a4fcea0cab14 100644 --- a/arch/x86/entry/entry_64_compat.S +++ b/arch/x86/entry/entry_64_compat.S @@ -311,105 +311,3 @@ sysret32_from_system_call: swapgs sysretl SYM_CODE_END(entry_SYSCALL_compat) - -/* - * 32-bit legacy system call entry. - * - * 32-bit x86 Linux system calls traditionally used the INT $0x80 - * instruction. INT $0x80 lands here. - * - * This entry point can be used by 32-bit and 64-bit programs to perform - * 32-bit system calls. Instances of INT $0x80 can be found inline in - * various programs and libraries. It is also used by the vDSO's - * __kernel_vsyscall fallback for hardware that doesn't support a faster - * entry method. Restarted 32-bit system calls also fall back to INT - * $0x80 regardless of what instruction was originally used to do the - * system call. - * - * This is considered a slow path. It is not used by most libc - * implementations on modern hardware except during process startup. - * - * Arguments: - * eax system call number - * ebx arg1 - * ecx arg2 - * edx arg3 - * esi arg4 - * edi arg5 - * ebp arg6 - */ -SYM_CODE_START(entry_INT80_compat) - UNWIND_HINT_EMPTY - /* - * Interrupts are off on entry. - */ - ASM_CLAC /* Do this early to minimize exposure */ - SWAPGS - - /* - * User tracing code (ptrace or signal handlers) might assume that - * the saved RAX contains a 32-bit number when we're invoking a 32-bit - * syscall. Just in case the high bits are nonzero, zero-extend - * the syscall number. (This could almost certainly be deleted - * with no ill effects.) - */ - movl %eax, %eax - - /* switch to thread stack expects orig_ax and rdi to be pushed */ - pushq %rax /* pt_regs->orig_ax */ - pushq %rdi /* pt_regs->di */ - - /* Need to switch before accessing the thread stack. */ - SWITCH_TO_KERNEL_CR3 scratch_reg=3D%rdi - - /* In the Xen PV case we already run on the thread stack. */ - ALTERNATIVE "", "jmp .Lint80_keep_stack", X86_FEATURE_XENPV - - movq %rsp, %rdi - movq PER_CPU_VAR(cpu_current_top_of_stack), %rsp - - pushq 6*8(%rdi) /* regs->ss */ - pushq 5*8(%rdi) /* regs->rsp */ - pushq 4*8(%rdi) /* regs->eflags */ - pushq 3*8(%rdi) /* regs->cs */ - pushq 2*8(%rdi) /* regs->ip */ - pushq 1*8(%rdi) /* regs->orig_ax */ - pushq (%rdi) /* pt_regs->di */ -.Lint80_keep_stack: - - pushq %rsi /* pt_regs->si */ - xorl %esi, %esi /* nospec si */ - pushq %rdx /* pt_regs->dx */ - xorl %edx, %edx /* nospec dx */ - pushq %rcx /* pt_regs->cx */ - xorl %ecx, %ecx /* nospec cx */ - pushq $-ENOSYS /* pt_regs->ax */ - pushq %r8 /* pt_regs->r8 */ - xorl %r8d, %r8d /* nospec r8 */ - pushq %r9 /* pt_regs->r9 */ - xorl %r9d, %r9d /* nospec r9 */ - pushq %r10 /* pt_regs->r10*/ - xorl %r10d, %r10d /* nospec r10 */ - pushq %r11 /* pt_regs->r11 */ - xorl %r11d, %r11d /* nospec r11 */ - pushq %rbx /* pt_regs->rbx */ - xorl %ebx, %ebx /* nospec rbx */ - pushq %rbp /* pt_regs->rbp */ - xorl %ebp, %ebp /* nospec rbp */ - pushq %r12 /* pt_regs->r12 */ - xorl %r12d, %r12d /* nospec r12 */ - pushq %r13 /* pt_regs->r13 */ - xorl %r13d, %r13d /* nospec r13 */ - pushq %r14 /* pt_regs->r14 */ - xorl %r14d, %r14d /* nospec r14 */ - pushq %r15 /* pt_regs->r15 */ - xorl %r15d, %r15d /* nospec r15 */ - - UNWIND_HINT_REGS - - cld - - movq %rsp, %rdi - call do_int80_syscall_32 - jmp swapgs_restore_regs_and_return_to_usermode -SYM_CODE_END(entry_INT80_compat) diff --git a/arch/x86/include/asm/idtentry.h b/arch/x86/include/asm/idtentr= y.h index 1345088e9902..38cb2e0dc2c7 100644 --- a/arch/x86/include/asm/idtentry.h +++ b/arch/x86/include/asm/idtentry.h @@ -204,6 +204,20 @@ __visible noinstr void func(struct pt_regs *regs, \ \ static noinline void __##func(struct pt_regs *regs, u32 vector) =20 +/** + * DECLARE_IDTENTRY_IA32_EMULATION - Declare functions for int80 + * @vector: Vector number (ignored for C) + * @asm_func: Function name of the entry point + * @cfunc: The C handler called from the ASM entry point (ignored for C) + * + * Declares two functions: + * - The ASM entry point: asm_func + * - The XEN PV trap entry point: xen_##asm_func (maybe unused) + */ +#define DECLARE_IDTENTRY_IA32_EMULATION(vector, asm_func, cfunc) \ + asmlinkage void asm_func(void); \ + asmlinkage void xen_##asm_func(void) + /** * DECLARE_IDTENTRY_SYSVEC - Declare functions for system vector entry poi= nts * @vector: Vector number (ignored for C) @@ -430,6 +444,35 @@ __visible noinstr void func(struct pt_regs *regs, \ #define DECLARE_IDTENTRY_ERRORCODE(vector, func) \ idtentry vector asm_##func func has_error_code=3D1 =20 +/* + * 32-bit legacy system call entry. + * + * 32-bit x86 Linux system calls traditionally used the INT $0x80 + * instruction. INT $0x80 lands here. + * + * This entry point can be used by 32-bit and 64-bit programs to perform + * 32-bit system calls. Instances of INT $0x80 can be found inline in + * various programs and libraries. It is also used by the vDSO's + * __kernel_vsyscall fallback for hardware that doesn't support a faster + * entry method. Restarted 32-bit system calls also fall back to INT + * $0x80 regardless of what instruction was originally used to do the + * system call. + * + * This is considered a slow path. It is not used by most libc + * implementations on modern hardware except during process startup. + * + * Arguments: + * eax system call number + * ebx arg1 + * ecx arg2 + * edx arg3 + * esi arg4 + * edi arg5 + * ebp arg6 + */ +#define DECLARE_IDTENTRY_IA32_EMULATION(vector, asm_func, cfunc) \ + idtentry vector asm_func cfunc has_error_code=3D0 + /* Special case for 32bit IRET 'trap'. Do not emit ASM code */ #define DECLARE_IDTENTRY_SW(vector, func) =20 @@ -631,6 +674,10 @@ DECLARE_IDTENTRY_IRQ(X86_TRAP_OTHER, common_interrupt); DECLARE_IDTENTRY_IRQ(X86_TRAP_OTHER, spurious_interrupt); #endif =20 +#ifdef CONFIG_IA32_EMULATION +DECLARE_IDTENTRY_IA32_EMULATION(IA32_SYSCALL_VECTOR, entry_INT80_compat, d= o_int80_syscall_32); +#endif + /* System vector entry points */ #ifdef CONFIG_X86_LOCAL_APIC DECLARE_IDTENTRY_SYSVEC(ERROR_APIC_VECTOR, sysvec_error_interrupt); diff --git a/arch/x86/include/asm/proto.h b/arch/x86/include/asm/proto.h index feed36d44d04..c4d331fe65ff 100644 --- a/arch/x86/include/asm/proto.h +++ b/arch/x86/include/asm/proto.h @@ -28,10 +28,6 @@ void entry_SYSENTER_compat(void); void __end_entry_SYSENTER_compat(void); void entry_SYSCALL_compat(void); void entry_SYSCALL_compat_safe_stack(void); -void entry_INT80_compat(void); -#ifdef CONFIG_XEN_PV -void xen_entry_INT80_compat(void); -#endif #endif =20 void x86_configure_nx(void); --=20 2.19.1.6.gb485710b From nobody Fri May 3 04:42:38 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3B0E2C433EF for ; Wed, 8 Dec 2021 11:09:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232112AbhLHLM5 (ORCPT ); Wed, 8 Dec 2021 06:12:57 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50552 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232210AbhLHLMv (ORCPT ); Wed, 8 Dec 2021 06:12:51 -0500 Received: from mail-pl1-x636.google.com (mail-pl1-x636.google.com [IPv6:2607:f8b0:4864:20::636]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 14263C0617A2 for ; Wed, 8 Dec 2021 03:09:20 -0800 (PST) Received: by mail-pl1-x636.google.com with SMTP id q17so1288469plr.11 for ; Wed, 08 Dec 2021 03:09:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WphTuTVslZby+oDEm50XCSesh2uHM5bzyfp9NXbgOWI=; b=htwdJLPu0gbuBqzNfz7sUj3efCTwW2LVgoh7ceQ0BMbkeYhPhXwXqOasG0Bd8RJnve XAAXE8iaSVBK6I6+t8twEOP/RHaK4mAkcL91G9J2LxApElT1FlYqrg1s/Gg2QuqLEqAt Xp4KqW62Iq1m5547lpOp1SrpBcM3mYOrFm91iX6Fb7OiOY1ScjdTivEC2Oy/lIyYlKLm gZAsMcM2ZGf8cfQoKLV6309j6vQq+DLsm0OVA24wlnmsN4Ki2w52UEmZGesy0mkRCwlx D6oBpssBTUieU9+0PWVIpucZ1qx6OzKjivCUNTWYSj2ChMJ0rvHeSI5cVlCD5IXjw/IB 1w0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WphTuTVslZby+oDEm50XCSesh2uHM5bzyfp9NXbgOWI=; b=NGHhSIEo8N8V6pnZ3XnIrDcQ8JyNVHQsTgsUhXJdbVwj4jiZ1iMowNVstcLY4wWg72 yFukkQOWCxJUtojlJ8VyCYNN53b/6CZ5sa+0qE7ShTSEYhmk5A3NVrTte1CPk41lP3SD +I/qmAN/t1ItuAYyNzpWy9SI3L38Ax9hPoPT1FZsKJfPazI7Gnc6WV2f/ukeImqouw1W XjlsVX0qMC6CH4FKn983DrW3dhQ4N8DfSuANgEcEBz4oKVCcvcqYdavdm0qv9vxRDdV8 fs4pfAm6mj2T+OH653BO3JM9fDYajG6J+JAW+lv0wYJSxTg4OP0fsXlSSODj56LZvUzS gFIw== X-Gm-Message-State: AOAM532eiSQNZiSo5pKipZlvjBwKHSxibkbk+q4uWrKDm6eCCdASDOIb e6OaOUOM+H2OvGmA8MLZ/B08aiRxfM8= X-Google-Smtp-Source: ABdhPJygiu3o+VoYWbdot6ZOwAuTCklCWFi5NzkjvztY5lMtfZYeyM7huQwfqLtMKHLX/ykwc3TFVw== X-Received: by 2002:a17:90b:357:: with SMTP id fh23mr6446123pjb.238.1638961759479; Wed, 08 Dec 2021 03:09:19 -0800 (PST) Received: from localhost ([47.251.3.230]) by smtp.gmail.com with ESMTPSA id s15sm2665417pjs.51.2021.12.08.03.09.18 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 08 Dec 2021 03:09:19 -0800 (PST) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, xen-devel@lists.xenproject.org, Lai Jiangshan , Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" Subject: [PATCH 09/11] x86/entry: Convert SWAPGS to swapgs in entry_SYSENTER_compat() Date: Wed, 8 Dec 2021 19:08:31 +0800 Message-Id: <20211208110833.65366-10-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20211208110833.65366-1-jiangshanlai@gmail.com> References: <20211208110833.65366-1-jiangshanlai@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Lai Jiangshan XENPV has its own entry point for SYSENTER, it doesn't use entry_SYSENTER_compat. So the pv-awared SWAPGS can be changed to swapgs. Signed-off-by: Lai Jiangshan --- arch/x86/entry/entry_64_compat.S | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/entry/entry_64_compat.S b/arch/x86/entry/entry_64_com= pat.S index a4fcea0cab14..72e017c3941f 100644 --- a/arch/x86/entry/entry_64_compat.S +++ b/arch/x86/entry/entry_64_compat.S @@ -49,7 +49,7 @@ SYM_CODE_START(entry_SYSENTER_compat) UNWIND_HINT_EMPTY /* Interrupts are off on entry. */ - SWAPGS + swapgs =20 pushq %rax SWITCH_TO_KERNEL_CR3 scratch_reg=3D%rax --=20 2.19.1.6.gb485710b From nobody Fri May 3 04:42:38 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3D01AC433F5 for ; Wed, 8 Dec 2021 11:09:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232197AbhLHLNA (ORCPT ); Wed, 8 Dec 2021 06:13:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50600 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232100AbhLHLM6 (ORCPT ); Wed, 8 Dec 2021 06:12:58 -0500 Received: from mail-pj1-x102d.google.com (mail-pj1-x102d.google.com [IPv6:2607:f8b0:4864:20::102d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F2350C061746 for ; Wed, 8 Dec 2021 03:09:26 -0800 (PST) Received: by mail-pj1-x102d.google.com with SMTP id cq22-20020a17090af99600b001a9550a17a5so4115372pjb.2 for ; Wed, 08 Dec 2021 03:09:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/Kd9tDbqkXgV+uDS+ZN2PkifPpO4nwWCGziDx4Ai290=; b=DZBRV+VFT9+KfIlW9vI1YNTWjjSwY9UVdpausVhNvxvTeFCAhQr4BL86f1ZVb5wqYx daMm0jsnf1EZop+hPS3Rsl+eU347P7jXAR0P3RT2+WtY7EQ2A2yWN79dg12hx04Hs3DY iR2cUpLa4sLw2jps+ZS2rPZ95nG7mTU3J1k/V23q7IYOzjDvAuCF7BShXIc9S6F75RGg tKsQ5jKl/9h3kXq8Er5D+XxIEwR7FRorkvKKVlKBNuLRh7xuwrknY5JB9iLYWlgPc5Ip brHSmlHdSmwtVh6ZsE/XtTcbGI9NZy6tbBUp6sONJacjKJ0aJI0Wz9IDXReHISQ2qhXx 46UQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/Kd9tDbqkXgV+uDS+ZN2PkifPpO4nwWCGziDx4Ai290=; b=Nl2YFOi4mjCWQPKarI8pfObjn/KvCwv8dAYdmbcGzeNubqH2w66//9ntcylJYnSEiK o3E0dxoXzGF46vF1a0U9aUy0wjmhUBCa5zG6T7Henl8qG2KUEm+wdHRKfeHL2/M3wAdN 43Xe5+lOR/SPMXvrkiWzb9kD55NDa1FoT98Z0/ZzpQDIoqqpXDidjy7oXQ2HhThwsya3 b8SqyJLg75sf6AIsorAhsa7qC4Iu9GkNkUYYJ+d8KlzRY3Dtz3k4c7XHeyec9BDepvcw VA9HTxhwAy2XmCuTWi0r8kzXKViGst1+kCdpxDEqTFOHv1ax89Xo+gg2sswSof3L4JOz /1gA== X-Gm-Message-State: AOAM532WYlf/+duVIvSnLBPUCA1c5l0urJt+bgLBlf7bvBeRWR9pg8Oy Lc1Fhr59bGABVK4ZZwNVErRARMUYRdY= X-Google-Smtp-Source: ABdhPJwtuS/bFQhmnmbhADvj6guciKxV5lk7NvScGFOlc2udq8C8NxlaMojhnh/rnPK9YGSNXl3k8g== X-Received: by 2002:a17:90a:e00c:: with SMTP id u12mr6251592pjy.139.1638961766047; Wed, 08 Dec 2021 03:09:26 -0800 (PST) Received: from localhost ([47.251.3.230]) by smtp.gmail.com with ESMTPSA id e4sm2315444pgi.21.2021.12.08.03.09.25 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 08 Dec 2021 03:09:25 -0800 (PST) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, xen-devel@lists.xenproject.org, Lai Jiangshan , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Juergen Gross , "Peter Zijlstra (Intel)" , "Kirill A. Shutemov" Subject: [PATCH 10/11] x86: Remove the definition of SWAPGS Date: Wed, 8 Dec 2021 19:08:32 +0800 Message-Id: <20211208110833.65366-11-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20211208110833.65366-1-jiangshanlai@gmail.com> References: <20211208110833.65366-1-jiangshanlai@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Lai Jiangshan There is no user of the pv-aware SWAPGS anymore. Signed-off-by: Lai Jiangshan --- arch/x86/include/asm/irqflags.h | 2 -- 1 file changed, 2 deletions(-) diff --git a/arch/x86/include/asm/irqflags.h b/arch/x86/include/asm/irqflag= s.h index 87761396e8cc..ac2e4cc47210 100644 --- a/arch/x86/include/asm/irqflags.h +++ b/arch/x86/include/asm/irqflags.h @@ -140,13 +140,11 @@ static __always_inline void arch_local_irq_restore(un= signed long flags) #else #ifdef CONFIG_X86_64 #ifdef CONFIG_XEN_PV -#define SWAPGS ALTERNATIVE "swapgs", "", X86_FEATURE_XENPV #define INTERRUPT_RETURN \ ANNOTATE_RETPOLINE_SAFE; \ ALTERNATIVE_TERNARY("jmp *paravirt_iret(%rip);", \ X86_FEATURE_XENPV, "jmp xen_iret;", "jmp native_iret;") #else -#define SWAPGS swapgs #define INTERRUPT_RETURN jmp native_iret #endif #endif --=20 2.19.1.6.gb485710b From nobody Fri May 3 04:42:38 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DE6D3C433F5 for ; Wed, 8 Dec 2021 11:09:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232199AbhLHLNH (ORCPT ); Wed, 8 Dec 2021 06:13:07 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50632 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232100AbhLHLNE (ORCPT ); Wed, 8 Dec 2021 06:13:04 -0500 Received: from mail-pf1-x42b.google.com (mail-pf1-x42b.google.com [IPv6:2607:f8b0:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 13852C0617A2 for ; Wed, 8 Dec 2021 03:09:33 -0800 (PST) Received: by mail-pf1-x42b.google.com with SMTP id x5so2233607pfr.0 for ; Wed, 08 Dec 2021 03:09:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2ZAl8F0tOkzqNpxvhUmIBbQ/2XyCnM6LrThKg9k5W8s=; b=XewnByTcwGZ5HuWSJOktdkA8rXnpKBKFuSg0cxOJL7s3BdUvTPdODuZfLIDeepUm4V +UFNS5kKkZ64h2+SLoku6yR1tR/0A+eaTTMV6wH4OAgYBCykpmcbSC84xrwGT8DK5z6I NAwcRUWRE5xi4yleywYOHQ3Z+PAuijS3/ucoUSnX0O8roEEkMd3TMCNQi4/Nee7ynLP8 YtGzd5gKmnV6H5WeuB2jtnHcA/FdVci05P8W9WFHdMj60WaDi0E67yIn6k7XYkXd2N/Q Pkh/hb3+/neA+1fwV3CX98BZZHeeXZYXum/i4O8YUwzfbt+j8AeThm8SubonZeZuOnPu cX4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2ZAl8F0tOkzqNpxvhUmIBbQ/2XyCnM6LrThKg9k5W8s=; b=OKBx8GZpwbq+wBmfMlUbkLDB9GNGHHUkER78ymqG6P5zJEOcsTQnpoWbwkyZucRRt/ lG3b46RoAxpoySkKu7JHaUdql8osnyVHwdIT0cvb0wfloDgFOI8dNzsSVY812aODWM5w YI1YTgvKLFU4XOA5YIOnplCIn+JNl3aVnBXNxeFRNQqusZO+xwaCXLsv7khPkATwFTka xjMa+9RCl0dHozrnYNs6kO4HEbPYV71SvaS9GyuPKTpEBfe6Zxxfe9S+K22S8F18wIwS LCosLZ/M+OuzJA1lSK0qe+0g6u4ygZhk7WHjJNdZ+wgOorNh5undPT6flfeG/YQ8LGs3 BB5Q== X-Gm-Message-State: AOAM533sPPVxvw9vNUg6Y7V89tDIDOgZT2MkAVdOfuwjGb2Unqh+PTbh VG9kGCrP+PozSm90FrOi+wfdATm8les= X-Google-Smtp-Source: ABdhPJw1j8OimGYb+Vk6xtTFU75lvQF/n3cAmCvhA4Lo654aq4frrTPh1GDeEbLLUScPSozJ7D8kIA== X-Received: by 2002:a65:6a0b:: with SMTP id m11mr28632692pgu.372.1638961772554; Wed, 08 Dec 2021 03:09:32 -0800 (PST) Received: from localhost ([47.251.3.230]) by smtp.gmail.com with ESMTPSA id u12sm3355395pfk.71.2021.12.08.03.09.31 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 08 Dec 2021 03:09:32 -0800 (PST) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, xen-devel@lists.xenproject.org, Lai Jiangshan , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Peter Zijlstra , Joerg Roedel , "Chang S. Bae" Subject: [PATCH 11/11] x86/entry: Remove the branch in sync_regs() Date: Wed, 8 Dec 2021 19:08:33 +0800 Message-Id: <20211208110833.65366-12-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20211208110833.65366-1-jiangshanlai@gmail.com> References: <20211208110833.65366-1-jiangshanlai@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Lai Jiangshan In non-xenpv, the sp0 is the trampoline stack, and sync_regs() is called on non-xenpv only since error_entry is not called on xenpv, so the stack must be the trampoline stack or one of the IST stack and the check in sync_regs() is unneeded. Signed-off-by: Lai Jiangshan --- arch/x86/kernel/traps.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index a6c0bc9ee36f..63cdb7aedd67 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -708,13 +708,15 @@ DEFINE_IDTENTRY_RAW(exc_int3) /* * Help handler running on a per-cpu (IST or entry trampoline) stack * to switch to the normal thread stack if the interrupted code was in - * user mode. The actual stack switch is done in entry_64.S + * user mode. The actual stack switch is done in entry_64.S. It is not + * called from XENPV and the caller has ensured it is not on the thread + * stack. */ asmlinkage __visible noinstr struct pt_regs *sync_regs(struct pt_regs *ere= gs) { struct pt_regs *regs =3D (struct pt_regs *)this_cpu_read(cpu_current_top_= of_stack) - 1; - if (regs !=3D eregs) - *regs =3D *eregs; + + *regs =3D *eregs; return regs; } =20 --=20 2.19.1.6.gb485710b