From nobody Fri Sep 5 09:22:44 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B3C2FCDB482 for ; Wed, 11 Oct 2023 22:44:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233646AbjJKWoG (ORCPT ); Wed, 11 Oct 2023 18:44:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59270 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1376503AbjJKWn6 (ORCPT ); Wed, 11 Oct 2023 18:43:58 -0400 Received: from mail-oi1-x22e.google.com (mail-oi1-x22e.google.com [IPv6:2607:f8b0:4864:20::22e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C591DA4 for ; Wed, 11 Oct 2023 15:43:56 -0700 (PDT) Received: by mail-oi1-x22e.google.com with SMTP id 5614622812f47-3af5fcb5e37so199037b6e.0 for ; Wed, 11 Oct 2023 15:43:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1697064236; x=1697669036; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=x3jwsa4qQcq6MjdP7xBGeHtYxSgXnGko/9ZE84LMQJ8=; b=ECh+tPxXyDRic13SZWqVcc9fcrWuVet2skFgr0vWL5hgFDKraU/hGvEtXywm79tIxl 6Hadfai5vE4sUSjL47tZrEwECpgQiNwqXepB43m93rvdmV0ucEVgabCGj1mW8hHVBBDf Rzu6A81yAmV1dpFUlHZsr0H2p1XbG6HqH59moG2xbvudCWoSu2kEqVS5E9lPU9Q7L1NW OWyJVBFAiiUM5y0CWknlz73mHK5Os0lw6EQj5lFc36MzvEJTbSFvOBpr9QviVC0adQ8C mmciu5pstctcEDJshOxP1703PsKOSgU2pKxx2ioAUOKec4LulEESogNfegNZFbA8/DfM YwnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697064236; x=1697669036; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=x3jwsa4qQcq6MjdP7xBGeHtYxSgXnGko/9ZE84LMQJ8=; b=sKQOKghFxcMxl00mvs932as1NUbfYIeGphBik7FocYUxQMUqAvEE6RSC2BKiEU4RPL NNLH3OTi5GIJbUULNSdWP8CIDLgNbXmQMsHYwDmF1ePGdl8VW1GB6396Hp4tREmyJYlt LDvKwHGXo4s2c4nRafiq2XUbXXKpuIzX3HTz5TD2O4x7vmOGkI0d6r5RA3H1P9O53vTo QMSpm0VjfXsKSjh2Wy9x030vpPdvSaZ7oX6vdxDb1CEYd8o4QWPdeqj4HYm3Lh3N5jiP Qrzx4Z4+g/w9y7zBSZ3LWMjBIWUnqAiRfIWzpBNoraIeakqdaC1zvGggukg4HLharZbM tYyA== X-Gm-Message-State: AOJu0YzgoJbDjWn35wk4iodvtchQ4JPzGmXzp0hPzJCMkjo+8d4wbFHI Ex5O9VzATMul6D6r5KYmeU6Wrq1DcA== X-Google-Smtp-Source: AGHT+IHD91chU2bpjpfLNczYCGqe8aLE5Ffk4rfb4b18+Hf/AJPw9ibblIAQZr5UZPrxB1Of2VPnSg== X-Received: by 2002:a05:6808:238d:b0:3b0:d939:298b with SMTP id bp13-20020a056808238d00b003b0d939298bmr11076793oib.47.1697064235608; Wed, 11 Oct 2023 15:43:55 -0700 (PDT) Received: from citadel.lan ([2600:6c4a:4d3f:6d5c::1019]) by smtp.gmail.com with ESMTPSA id o10-20020a0cf4ca000000b0065b129ec0e8sm6132871qvm.57.2023.10.11.15.43.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Oct 2023 15:43:55 -0700 (PDT) From: Brian Gerst To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: Ingo Molnar , Thomas Gleixner , Borislav Petkov , "H . Peter Anvin" , Andy Lutomirski , Brian Gerst Subject: [PATCH v3 1/3] x86/entry/64: Convert SYSRET validation tests to C Date: Wed, 11 Oct 2023 18:43:49 -0400 Message-ID: <20231011224351.130935-2-brgerst@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231011224351.130935-1-brgerst@gmail.com> References: <20231011224351.130935-1-brgerst@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Signed-off-by: Brian Gerst --- arch/x86/entry/common.c | 43 ++++++++++++++++++++++++++- arch/x86/entry/entry_64.S | 53 ++-------------------------------- arch/x86/include/asm/syscall.h | 2 +- 3 files changed, 45 insertions(+), 53 deletions(-) diff --git a/arch/x86/entry/common.c b/arch/x86/entry/common.c index 0551bcb197fb..207149a0a9b3 100644 --- a/arch/x86/entry/common.c +++ b/arch/x86/entry/common.c @@ -71,7 +71,8 @@ static __always_inline bool do_syscall_x32(struct pt_regs= *regs, int nr) return false; } =20 -__visible noinstr void do_syscall_64(struct pt_regs *regs, int nr) +/* Returns true to return using SYSRET, or false to use IRET */ +__visible noinstr bool do_syscall_64(struct pt_regs *regs, int nr) { add_random_kstack_offset(); nr =3D syscall_enter_from_user_mode(regs, nr); @@ -85,6 +86,46 @@ __visible noinstr void do_syscall_64(struct pt_regs *reg= s, int nr) =20 instrumentation_end(); syscall_exit_to_user_mode(regs); + + /* + * Check that the register state is valid for using SYSRET to exit + * to userspace. Otherwise use the slower but fully capable IRET + * exit path. + */ + + /* XEN PV guests always use IRET path */ + if (cpu_feature_enabled(X86_FEATURE_XENPV)) + return false; + + /* SYSRET requires RCX =3D=3D RIP and R11 =3D=3D EFLAGS */ + if (unlikely(regs->cx !=3D regs->ip || regs->r11 !=3D regs->flags)) + return false; + + /* CS and SS must match the values set in MSR_STAR */ + if (unlikely(regs->cs !=3D __USER_CS || regs->ss !=3D __USER_DS)) + return false; + + /* + * On Intel CPUs, SYSRET with non-canonical RCX/RIP will #GP + * in kernel space. This essentially lets the user take over + * the kernel, since userspace controls RSP. + * + * Change top bits to match most significant bit (47th or 56th bit + * depending on paging mode) in the address. + */ + if (unlikely(!__is_canonical_address(regs->ip, __VIRTUAL_MASK_SHIFT + 1))) + return false; + + /* + * SYSRET cannot restore RF. It can restore TF, but unlike IRET, + * restoring TF results in a trap from userspace immediately after + * SYSRET. + */ + if (unlikely(regs->flags & (X86_EFLAGS_RF | X86_EFLAGS_TF))) + return false; + + /* Use SYSRET to exit to userspace */ + return true; } #endif =20 diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index 3bdc22d7e78f..de6469dffe3a 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -126,57 +126,8 @@ SYM_INNER_LABEL(entry_SYSCALL_64_after_hwframe, SYM_L_= GLOBAL) * In the Xen PV case we must use iret anyway. */ =20 - ALTERNATIVE "", "jmp swapgs_restore_regs_and_return_to_usermode", \ - X86_FEATURE_XENPV - - movq RCX(%rsp), %rcx - movq RIP(%rsp), %r11 - - cmpq %rcx, %r11 /* SYSRET requires RCX =3D=3D RIP */ - jne swapgs_restore_regs_and_return_to_usermode - - /* - * On Intel CPUs, SYSRET with non-canonical RCX/RIP will #GP - * in kernel space. This essentially lets the user take over - * the kernel, since userspace controls RSP. - * - * If width of "canonical tail" ever becomes variable, this will need - * to be updated to remain correct on both old and new CPUs. - * - * Change top bits to match most significant bit (47th or 56th bit - * depending on paging mode) in the address. - */ -#ifdef CONFIG_X86_5LEVEL - ALTERNATIVE "shl $(64 - 48), %rcx; sar $(64 - 48), %rcx", \ - "shl $(64 - 57), %rcx; sar $(64 - 57), %rcx", X86_FEATURE_LA57 -#else - shl $(64 - (__VIRTUAL_MASK_SHIFT+1)), %rcx - sar $(64 - (__VIRTUAL_MASK_SHIFT+1)), %rcx -#endif - - /* If this changed %rcx, it was not canonical */ - cmpq %rcx, %r11 - jne swapgs_restore_regs_and_return_to_usermode - - cmpq $__USER_CS, CS(%rsp) /* CS must match SYSRET */ - jne swapgs_restore_regs_and_return_to_usermode - - movq R11(%rsp), %r11 - cmpq %r11, EFLAGS(%rsp) /* R11 =3D=3D RFLAGS */ - jne swapgs_restore_regs_and_return_to_usermode - - /* - * SYSRET cannot restore RF. It can restore TF, but unlike IRET, - * restoring TF results in a trap from userspace immediately after - * SYSRET. - */ - testq $(X86_EFLAGS_RF|X86_EFLAGS_TF), %r11 - jnz swapgs_restore_regs_and_return_to_usermode - - /* nothing to check for RSP */ - - cmpq $__USER_DS, SS(%rsp) /* SS must match SYSRET */ - jne swapgs_restore_regs_and_return_to_usermode + ALTERNATIVE "testb %al, %al; jz swapgs_restore_regs_and_return_to_usermod= e", \ + "jmp swapgs_restore_regs_and_return_to_usermode", X86_FEATURE_XENPV =20 /* * We win! This label is here just for ease of understanding diff --git a/arch/x86/include/asm/syscall.h b/arch/x86/include/asm/syscall.h index c7e25c940f1a..f44e2f9ab65d 100644 --- a/arch/x86/include/asm/syscall.h +++ b/arch/x86/include/asm/syscall.h @@ -126,7 +126,7 @@ static inline int syscall_get_arch(struct task_struct *= task) ? AUDIT_ARCH_I386 : AUDIT_ARCH_X86_64; } =20 -void do_syscall_64(struct pt_regs *regs, int nr); +bool do_syscall_64(struct pt_regs *regs, int nr); =20 #endif /* CONFIG_X86_32 */ =20 --=20 2.41.0 From nobody Fri Sep 5 09:22:44 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64C0BCDB47E for ; Wed, 11 Oct 2023 22:44:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1376575AbjJKWoI (ORCPT ); Wed, 11 Oct 2023 18:44:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59286 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1376550AbjJKWn7 (ORCPT ); Wed, 11 Oct 2023 18:43:59 -0400 Received: from mail-qv1-xf31.google.com (mail-qv1-xf31.google.com [IPv6:2607:f8b0:4864:20::f31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D73E7A9 for ; Wed, 11 Oct 2023 15:43:57 -0700 (PDT) Received: by mail-qv1-xf31.google.com with SMTP id 6a1803df08f44-66d0760cd20so2713416d6.0 for ; Wed, 11 Oct 2023 15:43:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1697064236; x=1697669036; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HZGveJLgwifDwKHvj2/VHQXDCqoTe78KUSxE+lNTdAg=; b=awxtAks/9WVVH+7UeadSsBtLZKhKfCMVMsAsRl3L+7hg0BEKvqf0e0lcM1RPdJvM5Y JR316hc6OGz15ouCEDPgwA7i5lq8MkMKBdv5v94NkEpMQSvzgU9eu6AH2WM/HFzFerr3 5KHXna0S1grvnPNdQZ1BLpiEor3mjjbxiGKP4HrCQFCoFn4Am55c/CbT9kjpMlZpRrM2 Xhanru37wXgbAGIZOqbLlBffUsQR8xhcEEj/2EQZvRKpRvHv4c0N39UJMTMiFvXHl06K lJ5K1ojAAWG/f2fMU0/OjSrcsv4H7NmvrDvpknm9tOXT4ICADSgqr/5LUpt+1lP5x8RV cgKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697064236; x=1697669036; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HZGveJLgwifDwKHvj2/VHQXDCqoTe78KUSxE+lNTdAg=; b=T3ccQmZKmzQxvo7DA/iABj37ITHFYRjf2zp5OSwpFy3ZVzPUEi9rphHKRrvT8Je7UQ +5f/AwGIk0x5+7NsBP6sO+QGrzcMUcfAHg+a+MwNFKgSUZS+EFbGqHveXYVi89Ot4pO2 opq9xXQPb7aJmNYA4LoNzIAh1l6bS1QdNla0dfuT6R8MUlkeMtklqKftI8sZM3Er/yzY psAU5GXaorP+c8wAfv/RQTM2i+MgeA6FQ/qJBrhOmn3Rxul972+uOsTBypE5YkcRvu6N v4pPTHFvmt4gou+IjFse8+U/PLDfYfiz4bYRGTW6enLN1eUoOq7r3xtn2zPccUjHySlG oUag== X-Gm-Message-State: AOJu0Yzlj0TvpJ8PcodkIB6ptoddSZiVGo4PTINGNiNsMhvHqd0EbkMp MTk70oaGAFPOmEhJVqi8EzdJRJQuIA== X-Google-Smtp-Source: AGHT+IHai+Y+y8sZx8W8FhlOxbeoOR1TWhTqWxRutap+GxQJCstB9QWtljWPF63mMXsnsNwCmf3WwA== X-Received: by 2002:a05:6214:21ea:b0:66d:1103:3286 with SMTP id p10-20020a05621421ea00b0066d11033286mr2709612qvj.12.1697064236600; Wed, 11 Oct 2023 15:43:56 -0700 (PDT) Received: from citadel.lan ([2600:6c4a:4d3f:6d5c::1019]) by smtp.gmail.com with ESMTPSA id o10-20020a0cf4ca000000b0065b129ec0e8sm6132871qvm.57.2023.10.11.15.43.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Oct 2023 15:43:56 -0700 (PDT) From: Brian Gerst To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: Ingo Molnar , Thomas Gleixner , Borislav Petkov , "H . Peter Anvin" , Andy Lutomirski , Brian Gerst Subject: [PATCH v3 2/3] x86/entry/64: Use TASK_SIZE_MAX for canonical RIP test Date: Wed, 11 Oct 2023 18:43:50 -0400 Message-ID: <20231011224351.130935-3-brgerst@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231011224351.130935-1-brgerst@gmail.com> References: <20231011224351.130935-1-brgerst@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Using shifts to determine if an address is canonical is difficult for the compiler to optimize when the virtual address width is variable (LA57 feature) without using inline assembly. Instead, compare RIP against TASK_SIZE_MAX. The only user executable address outside of that range is the deprecated vsyscall page, which can fall back to using IRET. Signed-off-by: Brian Gerst --- arch/x86/entry/common.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/arch/x86/entry/common.c b/arch/x86/entry/common.c index 207149a0a9b3..e3d6f255379f 100644 --- a/arch/x86/entry/common.c +++ b/arch/x86/entry/common.c @@ -110,10 +110,10 @@ __visible noinstr bool do_syscall_64(struct pt_regs *= regs, int nr) * in kernel space. This essentially lets the user take over * the kernel, since userspace controls RSP. * - * Change top bits to match most significant bit (47th or 56th bit - * depending on paging mode) in the address. + * TASK_SIZE_MAX covers all user-accessible addresses other than + * the deprecated vsyscall page. */ - if (unlikely(!__is_canonical_address(regs->ip, __VIRTUAL_MASK_SHIFT + 1))) + if (unlikely(regs->ip >=3D TASK_SIZE_MAX)) return false; =20 /* --=20 2.41.0 From nobody Fri Sep 5 09:22:44 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0240BCDB482 for ; Wed, 11 Oct 2023 22:44:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1376602AbjJKWoK (ORCPT ); Wed, 11 Oct 2023 18:44:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59298 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1376578AbjJKWoC (ORCPT ); Wed, 11 Oct 2023 18:44:02 -0400 Received: from mail-qv1-xf2a.google.com (mail-qv1-xf2a.google.com [IPv6:2607:f8b0:4864:20::f2a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C6848A4 for ; Wed, 11 Oct 2023 15:43:58 -0700 (PDT) Received: by mail-qv1-xf2a.google.com with SMTP id 6a1803df08f44-66d0ceba445so2079146d6.0 for ; Wed, 11 Oct 2023 15:43:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1697064237; x=1697669037; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=s/e/e5drqChWwuXG6soyxRsu9kSizf72f1McaizEqmY=; b=KfG+5kXFwbi8s9e9C9i0A+an/sTrACwJmvaBTGPuzmgCRThBHRdKqutY6cFPvfzVKt W2XXslbtzICFNBiY3WwknRYczssYoechEXEDlVgUNazZi5/EYY8yuB+GmyEy6hKd901+ AL93sIQki1iHyh5WhdecSppakjsHlA806b3PE5WvRVi4bFfP1yuwqlPcmME02kgZU97h rG6/WcTmP86s+ETvl2ddh66tXCSSm89NQFyR+KMHmfSToP81Gp3eWpZGIZ30yTyET+07 zKYuPuluhWl5dhpd4olFLl5+ROmVX/9mHPzXNlFDdku5Uhs2F56xYaILGe7sP8ZRCpJR bq7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697064237; x=1697669037; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=s/e/e5drqChWwuXG6soyxRsu9kSizf72f1McaizEqmY=; b=Yy/xSnYX1SwyR/prMfy/P6xchiq0TjAbzArGEP5MzyPRoL2li/NzkBsvfoj/hELF32 4tvHdSn7YjvV74N2VPy5MKkGKy3JcE8pTOes2uvvVCFxCS9ZF7z/XENnQMqzVyi5RSlh tcP6wSWwxLQu2dQF0/dv6Ivhve/oCoGEOCU+/yPsfX4H4tCu5u7Aj3MvW+ob/TeYUOO1 byM6b4EMdTTAwbJTX+90WKVxCfsOZWR9ABBzDHDVz2UY3ilb3LCv04nyBacWv5iSCGJh 0fWI4lmJXA/qRGzxYM5v9G+P4hZH3qW/F2gu/CqFaU6HdU4Wv8+baLUGGthO/W2e6yy9 Lvvg== X-Gm-Message-State: AOJu0YyKRRXEmHeSrQPDrtqIliwsNc08BNZG2wyAmJ6Kf1EKbwVPhkgz Ti8QjjadDyHWMnMtHRGma8ogOcMzDQ== X-Google-Smtp-Source: AGHT+IFrc4I5n58/NTdAASVqcxX266gphMt+GTGmOWfYfo5XhT+GjQADGrhfsGOVzMNmmBw/eD88ew== X-Received: by 2002:a0c:efd4:0:b0:65c:ffb6:11ea with SMTP id a20-20020a0cefd4000000b0065cffb611eamr21865019qvt.33.1697064237533; Wed, 11 Oct 2023 15:43:57 -0700 (PDT) Received: from citadel.lan ([2600:6c4a:4d3f:6d5c::1019]) by smtp.gmail.com with ESMTPSA id o10-20020a0cf4ca000000b0065b129ec0e8sm6132871qvm.57.2023.10.11.15.43.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Oct 2023 15:43:57 -0700 (PDT) From: Brian Gerst To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: Ingo Molnar , Thomas Gleixner , Borislav Petkov , "H . Peter Anvin" , Andy Lutomirski , Brian Gerst Subject: [PATCH v3 3/3] x86/entry/32: Clean up syscall fast exit tests Date: Wed, 11 Oct 2023 18:43:51 -0400 Message-ID: <20231011224351.130935-4-brgerst@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231011224351.130935-1-brgerst@gmail.com> References: <20231011224351.130935-1-brgerst@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Merge compat and native code and clarify comments. Signed-off-by: Brian Gerst --- arch/x86/entry/common.c | 48 +++++++++++++++++++---------------------- 1 file changed, 22 insertions(+), 26 deletions(-) diff --git a/arch/x86/entry/common.c b/arch/x86/entry/common.c index e3d6f255379f..0acf35d7fe55 100644 --- a/arch/x86/entry/common.c +++ b/arch/x86/entry/common.c @@ -255,34 +255,30 @@ __visible noinstr bool do_fast_syscall_32(struct pt_r= egs *regs) if (!__do_fast_syscall_32(regs)) return false; =20 -#ifdef CONFIG_X86_64 /* - * Opportunistic SYSRETL: if possible, try to return using SYSRETL. - * SYSRETL is available on all 64-bit CPUs, so we don't need to - * bother with SYSEXIT. - * - * Unlike 64-bit opportunistic SYSRET, we can't check that CX =3D=3D IP, - * because the ECX fixup above will ensure that this is essentially - * never the case. + * Check that the register state is valid for using SYSRETL/SYSEXIT + * to exit to userspace. Otherwise use the slower but fully capable + * IRET exit path. */ - return regs->cs =3D=3D __USER32_CS && regs->ss =3D=3D __USER_DS && - regs->ip =3D=3D landing_pad && - (regs->flags & (X86_EFLAGS_RF | X86_EFLAGS_TF)) =3D=3D 0; -#else - /* - * Opportunistic SYSEXIT: if possible, try to return using SYSEXIT. - * - * Unlike 64-bit opportunistic SYSRET, we can't check that CX =3D=3D IP, - * because the ECX fixup above will ensure that this is essentially - * never the case. - * - * We don't allow syscalls at all from VM86 mode, but we still - * need to check VM, because we might be returning from sys_vm86. - */ - return regs->cs =3D=3D __USER_CS && regs->ss =3D=3D __USER_DS && - regs->ip =3D=3D landing_pad && - (regs->flags & (X86_EFLAGS_RF | X86_EFLAGS_TF | X86_EFLAGS_VM)) =3D=3D 0; -#endif + + /* XEN PV guests always use IRET path */ + if (cpu_feature_enabled(X86_FEATURE_XENPV)) + return false; + + /* EIP must point to the VDSO landing pad */ + if (unlikely(regs->ip !=3D landing_pad)) + return false; + + /* CS and SS must match the values set in MSR_STAR */ + if (unlikely(regs->cs !=3D __USER32_CS || regs->ss !=3D __USER_DS)) + return false; + + /* If the TF, RF, or VM flags are set, use IRET */ + if (unlikely(regs->flags & (X86_EFLAGS_RF | X86_EFLAGS_TF | X86_EFLAGS_VM= ))) + return false; + + /* Use SYSRETL/SYSEXIT to exit to userspace */ + return true; } =20 /* Returns true to return using SYSEXIT/SYSRETL, or false to use IRET */ --=20 2.41.0