From nobody Mon Jun 29 15:59:28 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1911C46467 for ; Tue, 8 Feb 2022 01:05:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344536AbiBHBDk (ORCPT ); Mon, 7 Feb 2022 20:03:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35594 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236116AbiBGXD1 (ORCPT ); Mon, 7 Feb 2022 18:03:27 -0500 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D6409C061355 for ; Mon, 7 Feb 2022 15:03:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1644275006; x=1675811006; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Zzuc+4BUhDw8AYLm9O80nUHIQ9qam3U/ceY5Tn4nhdA=; b=Z1NV8vEue4+XvJgpq8m+LgU34fJB88t9X20q2iYroyVUAqscCAbr3cQG uPBQm8eQvG/DK6IEOs9oEHGCjq0fWfU/TSqtl5iX4EIp7S7OVnt4klLBQ rCVgxgPcPg+ZmSleAwE5n8KxCy6l7tfw4Gi9Ns5t4XENigsP0E0d3a/60 W5m+IemVXS1Hvpc6xBZHK6P70QuZoZq06nlp/TDTFkV7yRTzO7GdGOXoG p69mOMjy7dmpalz3gIv2KjwDg9zQHzsz967C73URqIlJqD2noOKTpyaYt HL3+C2O9ZYL26C/9u127wNsjuAEO9N6Pijsai5pGW0xeLsxhYu4p0Evmt g==; X-IronPort-AV: E=McAfee;i="6200,9189,10251"; a="229475001" X-IronPort-AV: E=Sophos;i="5.88,351,1635231600"; d="scan'208";a="229475001" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Feb 2022 15:03:02 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,351,1635231600"; d="scan'208";a="540324005" Received: from otcwcpicx3.sc.intel.com ([172.25.55.73]) by orsmga008.jf.intel.com with ESMTP; 07 Feb 2022 15:03:01 -0800 From: Fenghua Yu To: "Thomas Gleixner" , "Dave Hansen" , "Ingo Molnar" , "Borislav Petkov" , "Peter Zijlstra" , "Andy Lutomirski" , "Tony Luck" , "Lu Baolu" , "Joerg Roedel" , Josh Poimboeuf , "Jacob Pan" , "Ashok Raj" , "Ravi V Shankar" Cc: iommu@lists.linux-foundation.org, "x86" , "linux-kernel" , Fenghua Yu Subject: [PATCH v4 01/11] iommu/sva: Rename CONFIG_IOMMU_SVA_LIB to CONFIG_IOMMU_SVA Date: Mon, 7 Feb 2022 15:02:44 -0800 Message-Id: <20220207230254.3342514-2-fenghua.yu@intel.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220207230254.3342514-1-fenghua.yu@intel.com> References: <20220207230254.3342514-1-fenghua.yu@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This CONFIG option originally only referred to the Shared Virtual Address (SVA) library. But it is now also used for non-library portions of code. Drop the "_LIB" suffix so that there is just one configuration options for all code relating to SVA. Signed-off-by: Fenghua Yu Reviewed-by: Tony Luck Reviewed-by: Thomas Gleixner Reviewed-by: Lu Baolu --- v4: - Add "Reviewed-by: Thomas Gleixner " (Thomas). v2: - Add this patch for more meaningful name CONFIG_IOMMU_SVA drivers/iommu/Kconfig | 6 +++--- drivers/iommu/Makefile | 2 +- drivers/iommu/intel/Kconfig | 2 +- drivers/iommu/iommu-sva-lib.h | 6 +++--- 4 files changed, 8 insertions(+), 8 deletions(-) diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig index 3eb68fa1b8cc..c79a0df090c0 100644 --- a/drivers/iommu/Kconfig +++ b/drivers/iommu/Kconfig @@ -144,8 +144,8 @@ config IOMMU_DMA select IRQ_MSI_IOMMU select NEED_SG_DMA_LENGTH =20 -# Shared Virtual Addressing library -config IOMMU_SVA_LIB +# Shared Virtual Addressing +config IOMMU_SVA bool select IOASID =20 @@ -379,7 +379,7 @@ config ARM_SMMU_V3 config ARM_SMMU_V3_SVA bool "Shared Virtual Addressing support for the ARM SMMUv3" depends on ARM_SMMU_V3 - select IOMMU_SVA_LIB + select IOMMU_SVA select MMU_NOTIFIER help Support for sharing process address spaces with devices using the diff --git a/drivers/iommu/Makefile b/drivers/iommu/Makefile index bc7f730edbb0..44475a9b3eea 100644 --- a/drivers/iommu/Makefile +++ b/drivers/iommu/Makefile @@ -27,6 +27,6 @@ obj-$(CONFIG_FSL_PAMU) +=3D fsl_pamu.o fsl_pamu_domain.o obj-$(CONFIG_S390_IOMMU) +=3D s390-iommu.o obj-$(CONFIG_HYPERV_IOMMU) +=3D hyperv-iommu.o obj-$(CONFIG_VIRTIO_IOMMU) +=3D virtio-iommu.o -obj-$(CONFIG_IOMMU_SVA_LIB) +=3D iommu-sva-lib.o io-pgfault.o +obj-$(CONFIG_IOMMU_SVA) +=3D iommu-sva-lib.o io-pgfault.o obj-$(CONFIG_SPRD_IOMMU) +=3D sprd-iommu.o obj-$(CONFIG_APPLE_DART) +=3D apple-dart.o diff --git a/drivers/iommu/intel/Kconfig b/drivers/iommu/intel/Kconfig index 247d0f2d5fdf..39a06d245f12 100644 --- a/drivers/iommu/intel/Kconfig +++ b/drivers/iommu/intel/Kconfig @@ -52,7 +52,7 @@ config INTEL_IOMMU_SVM select PCI_PRI select MMU_NOTIFIER select IOASID - select IOMMU_SVA_LIB + select IOMMU_SVA help Shared Virtual Memory (SVM) provides a facility for devices to access DMA resources through process address space by diff --git a/drivers/iommu/iommu-sva-lib.h b/drivers/iommu/iommu-sva-lib.h index 031155010ca8..95dc3ebc1928 100644 --- a/drivers/iommu/iommu-sva-lib.h +++ b/drivers/iommu/iommu-sva-lib.h @@ -17,7 +17,7 @@ struct device; struct iommu_fault; struct iopf_queue; =20 -#ifdef CONFIG_IOMMU_SVA_LIB +#ifdef CONFIG_IOMMU_SVA int iommu_queue_iopf(struct iommu_fault *fault, void *cookie); =20 int iopf_queue_add_device(struct iopf_queue *queue, struct device *dev); @@ -28,7 +28,7 @@ struct iopf_queue *iopf_queue_alloc(const char *name); void iopf_queue_free(struct iopf_queue *queue); int iopf_queue_discard_partial(struct iopf_queue *queue); =20 -#else /* CONFIG_IOMMU_SVA_LIB */ +#else /* CONFIG_IOMMU_SVA */ static inline int iommu_queue_iopf(struct iommu_fault *fault, void *cookie) { return -ENODEV; @@ -64,5 +64,5 @@ static inline int iopf_queue_discard_partial(struct iopf_= queue *queue) { return -ENODEV; } -#endif /* CONFIG_IOMMU_SVA_LIB */ +#endif /* CONFIG_IOMMU_SVA */ #endif /* _IOMMU_SVA_LIB_H */ --=20 2.35.1 From nobody Mon Jun 29 15:59:28 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8F4B6C35275 for ; Tue, 8 Feb 2022 01:05:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344580AbiBHBDo (ORCPT ); Mon, 7 Feb 2022 20:03:44 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35610 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245429AbiBGXD2 (ORCPT ); Mon, 7 Feb 2022 18:03:28 -0500 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5B981C061355 for ; Mon, 7 Feb 2022 15:03:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1644275008; x=1675811008; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ZQuX/1pAiBPvBMAccFCq7xa5FLVExUm2Q9ePeODXOtk=; b=SKs62tmIVcjwb0eMnIX0peBq2WYQNZIG5/PdO/LsRYwJcowtHffnn8+R sL9AN/FyR33ic6NrNn/b9qho9Vq/rX73dbR50cpfiaIDRNm1x01By4mYA ilSl2z+kI0PYM05FIXZ079yOmLQJpCvIwmQBr+/rCGeTpDobtu72o+/R2 6fqRcL+U88kLO12SWVybyvoW4WdzD5vnyrU6/cvmhWdXghzdBbs+q3eXS 1TAZ/5Ta4h8I9ufU672xR8wHz0a3fYgwH3db9JGRPh9oi1xiDySzkxRtP 8L5TfLuodKcqqneGCNt5U4OhvUpRETKocyya/rDd8B7lOHsDzbTgcA2e5 A==; X-IronPort-AV: E=McAfee;i="6200,9189,10251"; a="229475005" X-IronPort-AV: E=Sophos;i="5.88,351,1635231600"; d="scan'208";a="229475005" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Feb 2022 15:03:02 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,351,1635231600"; d="scan'208";a="540324016" Received: from otcwcpicx3.sc.intel.com ([172.25.55.73]) by orsmga008.jf.intel.com with ESMTP; 07 Feb 2022 15:03:02 -0800 From: Fenghua Yu To: "Thomas Gleixner" , "Dave Hansen" , "Ingo Molnar" , "Borislav Petkov" , "Peter Zijlstra" , "Andy Lutomirski" , "Tony Luck" , "Lu Baolu" , "Joerg Roedel" , Josh Poimboeuf , "Jacob Pan" , "Ashok Raj" , "Ravi V Shankar" Cc: iommu@lists.linux-foundation.org, "x86" , "linux-kernel" , Fenghua Yu Subject: [PATCH v4 02/11] mm: Change CONFIG option for mm->pasid field Date: Mon, 7 Feb 2022 15:02:45 -0800 Message-Id: <20220207230254.3342514-3-fenghua.yu@intel.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220207230254.3342514-1-fenghua.yu@intel.com> References: <20220207230254.3342514-1-fenghua.yu@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This currently depends on CONFIG_IOMMU_SUPPORT. But it is only needed when CONFIG_IOMMU_SVA option is enabled. Change the CONFIG guards around definition and initialization of mm->pasid field. Suggested-by: Jacob Pan Signed-off-by: Fenghua Yu Reviewed-by: Tony Luck Reviewed-by: Thomas Gleixner Reviewed-by: Lu Baolu --- v4: - Add "Reviewed-by: Thomas Gleixner " (Thomas). v2: - Change condition to more accurate CONFIG_IOMMU_SVA (Jacob and Tony) include/linux/mm_types.h | 2 +- kernel/fork.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 5140e5feb486..c5cbfd7915ad 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -631,7 +631,7 @@ struct mm_struct { #endif struct work_struct async_put_work; =20 -#ifdef CONFIG_IOMMU_SUPPORT +#ifdef CONFIG_IOMMU_SVA u32 pasid; #endif } __randomize_layout; diff --git a/kernel/fork.c b/kernel/fork.c index d75a528f7b21..6ee7551d3bd2 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -1021,7 +1021,7 @@ static void mm_init_owner(struct mm_struct *mm, struc= t task_struct *p) =20 static void mm_init_pasid(struct mm_struct *mm) { -#ifdef CONFIG_IOMMU_SUPPORT +#ifdef CONFIG_IOMMU_SVA mm->pasid =3D INIT_PASID; #endif } --=20 2.35.1 From nobody Mon Jun 29 15:59:28 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE4AAC43217 for ; Tue, 8 Feb 2022 01:04:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344559AbiBHBDl (ORCPT ); Mon, 7 Feb 2022 20:03:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35626 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245439AbiBGXD3 (ORCPT ); Mon, 7 Feb 2022 18:03:29 -0500 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D53E7C06109E for ; Mon, 7 Feb 2022 15:03:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1644275008; x=1675811008; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=plQnU2Dc8OE22hr3S5V59pRVLYmGY5GOp2NaA2zkwW8=; b=Z4S+LZSxcLCwQ72ADp+UrqTAn4RsTuAKdhTw9/qEaGf4OGTFeNg50J4g 5H02OEdZcJqoWTA0jYjtm2vLZtMr9IpPXUuo9DVHYKLuevRbYR9Ekiy7X 7fUL1UjwFmF8eARNVMap4hBDYxddIjHoBMFSLwrTfBEpw6RXjXNH2vZXj AP4y2Ns2GN4BYRsncxBxdA9Dqz3OBVyLXn1MVkQR+vLNKznvMH2qQCU0L RKXTbOnTkYYiB+NOd6EhvAupUfbfFytD7XcTeCn8cQWLmFhjx3j7BHIeD fbfeZSE+bIeuXdlTOVO4EWyawg2QLFbqzCfFwf8vd+hjH7aWYJhOV/SmJ w==; X-IronPort-AV: E=McAfee;i="6200,9189,10251"; a="229475009" X-IronPort-AV: E=Sophos;i="5.88,351,1635231600"; d="scan'208";a="229475009" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Feb 2022 15:03:03 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,351,1635231600"; d="scan'208";a="540324030" Received: from otcwcpicx3.sc.intel.com ([172.25.55.73]) by orsmga008.jf.intel.com with ESMTP; 07 Feb 2022 15:03:02 -0800 From: Fenghua Yu To: "Thomas Gleixner" , "Dave Hansen" , "Ingo Molnar" , "Borislav Petkov" , "Peter Zijlstra" , "Andy Lutomirski" , "Tony Luck" , "Lu Baolu" , "Joerg Roedel" , Josh Poimboeuf , "Jacob Pan" , "Ashok Raj" , "Ravi V Shankar" Cc: iommu@lists.linux-foundation.org, "x86" , "linux-kernel" , Fenghua Yu Subject: [PATCH v4 03/11] iommu/ioasid: Introduce a helper to check for valid PASIDs Date: Mon, 7 Feb 2022 15:02:46 -0800 Message-Id: <20220207230254.3342514-4-fenghua.yu@intel.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220207230254.3342514-1-fenghua.yu@intel.com> References: <20220207230254.3342514-1-fenghua.yu@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" pasid_valid() is defined to check if a given PASID is valid. Suggested-by: Ashok Raj Suggested-by: Jacob Pan Signed-off-by: Fenghua Yu Reviewed-by: Tony Luck Reviewed-by: Thomas Gleixner Reviewed-by: Lu Baolu --- v4: - Add "Reviewed-by: Thomas Gleixner " (Thomas). v2: - Define a helper pasid_valid() (Ashok, Jacob, Thomas, Tony) include/linux/ioasid.h | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/include/linux/ioasid.h b/include/linux/ioasid.h index e9dacd4b9f6b..2237f64dbaae 100644 --- a/include/linux/ioasid.h +++ b/include/linux/ioasid.h @@ -41,6 +41,10 @@ void *ioasid_find(struct ioasid_set *set, ioasid_t ioasi= d, int ioasid_register_allocator(struct ioasid_allocator_ops *allocator); void ioasid_unregister_allocator(struct ioasid_allocator_ops *allocator); int ioasid_set_data(ioasid_t ioasid, void *data); +static inline bool pasid_valid(ioasid_t ioasid) +{ + return ioasid !=3D INVALID_IOASID; +} =20 #else /* !CONFIG_IOASID */ static inline ioasid_t ioasid_alloc(struct ioasid_set *set, ioasid_t min, @@ -78,5 +82,10 @@ static inline int ioasid_set_data(ioasid_t ioasid, void = *data) return -ENOTSUPP; } =20 +static inline bool pasid_valid(ioasid_t ioasid) +{ + return false; +} + #endif /* CONFIG_IOASID */ #endif /* __LINUX_IOASID_H */ --=20 2.35.1 From nobody Mon Jun 29 15:59:28 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42F28C433F5 for ; Tue, 8 Feb 2022 01:05:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344562AbiBHBDm (ORCPT ); Mon, 7 Feb 2022 20:03:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35624 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245434AbiBGXD3 (ORCPT ); Mon, 7 Feb 2022 18:03:29 -0500 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6ABCEC061A73 for ; Mon, 7 Feb 2022 15:03:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1644275008; x=1675811008; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=905JQ3YZKvuZYu7D1FVJXac593yX75ihm3basE87bLk=; b=RxPGZAVnlTuKDuKMWh4piYMqWy32Ri5PMwox5lQcXvVApO1qKBb1M62d Uf0K3DJlwc5OxdXiUmD9ZhvLie75BiKMN74lWEe8Nfmv9XFb1eb0pgFhA vRTYAlBWKjkR91uMNwMZXI5PeoxxsMGpt7ACpb8ioX1abINi/JBBeMaVc Muq99MJnkwsgo0AgR44fJZDpAqr9kpsXuoA1ZKQY0XWA4RlEqAcIeQXKx Xs8HoR2kXpZsLTUUfZLLilnLAJDxZzqAKq0gK+JDNIV1/dsx4O+V2tN8E jvVFahVyNMay5g4bHiRdo8iZhAhE2wN7e9tZgJ/s5cKXxricvACCFMK6b g==; X-IronPort-AV: E=McAfee;i="6200,9189,10251"; a="229475012" X-IronPort-AV: E=Sophos;i="5.88,351,1635231600"; d="scan'208";a="229475012" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Feb 2022 15:03:04 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,351,1635231600"; d="scan'208";a="540324041" Received: from otcwcpicx3.sc.intel.com ([172.25.55.73]) by orsmga008.jf.intel.com with ESMTP; 07 Feb 2022 15:03:03 -0800 From: Fenghua Yu To: "Thomas Gleixner" , "Dave Hansen" , "Ingo Molnar" , "Borislav Petkov" , "Peter Zijlstra" , "Andy Lutomirski" , "Tony Luck" , "Lu Baolu" , "Joerg Roedel" , Josh Poimboeuf , "Jacob Pan" , "Ashok Raj" , "Ravi V Shankar" Cc: iommu@lists.linux-foundation.org, "x86" , "linux-kernel" , Fenghua Yu Subject: [PATCH v4 04/11] kernel/fork: Initialize mm's PASID Date: Mon, 7 Feb 2022 15:02:47 -0800 Message-Id: <20220207230254.3342514-5-fenghua.yu@intel.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220207230254.3342514-1-fenghua.yu@intel.com> References: <20220207230254.3342514-1-fenghua.yu@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" A new mm doesn't have a PASID yet when it's created. Initialize the mm's PASID on fork() or for init_mm to INVALID_IOASID (-1). INIT_PASID (0) is reserved for kernel legacy DMA PASID. It cannot be allocated to a user process. Initializing the process's PASID to 0 may cause confusion that why the process uses the reserved kernel legacy DMA PASID. Initializing the PASID to INVALID_IOASID (-1) explicitly tells the process doesn't have a valid PASID yet. Even though the only user of mm_pasid_init() is in fork.c, define it in as the first of three mm/pasid life cycle functions (init/set/drop) to keep these all together. Suggested-by: Dave Hansen Signed-off-by: Fenghua Yu Reviewed-by: Tony Luck Reviewed-by: Thomas Gleixner --- v4: - Update the commit message to explain why init pasid to -1 and why define mm_pasid_init() (Thomas) v2: - Change condition to more accurate CONFIG_IOMMU_SVA (Jacob) include/linux/sched/mm.h | 10 ++++++++++ kernel/fork.c | 10 ++-------- mm/init-mm.c | 4 ++++ 3 files changed, 16 insertions(+), 8 deletions(-) diff --git a/include/linux/sched/mm.h b/include/linux/sched/mm.h index aa5f09ca5bcf..c74d1edbac2f 100644 --- a/include/linux/sched/mm.h +++ b/include/linux/sched/mm.h @@ -8,6 +8,7 @@ #include #include #include +#include =20 /* * Routines for handling mm_structs @@ -433,4 +434,13 @@ static inline void membarrier_update_current_mm(struct= mm_struct *next_mm) } #endif =20 +#ifdef CONFIG_IOMMU_SVA +static inline void mm_pasid_init(struct mm_struct *mm) +{ + mm->pasid =3D INVALID_IOASID; +} +#else +static inline void mm_pasid_init(struct mm_struct *mm) {} +#endif + #endif /* _LINUX_SCHED_MM_H */ diff --git a/kernel/fork.c b/kernel/fork.c index 6ee7551d3bd2..deacd2c17a7f 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -97,6 +97,7 @@ #include #include #include +#include =20 #include #include @@ -1019,13 +1020,6 @@ static void mm_init_owner(struct mm_struct *mm, stru= ct task_struct *p) #endif } =20 -static void mm_init_pasid(struct mm_struct *mm) -{ -#ifdef CONFIG_IOMMU_SVA - mm->pasid =3D INIT_PASID; -#endif -} - static void mm_init_uprobes_state(struct mm_struct *mm) { #ifdef CONFIG_UPROBES @@ -1054,7 +1048,7 @@ static struct mm_struct *mm_init(struct mm_struct *mm= , struct task_struct *p, mm_init_cpumask(mm); mm_init_aio(mm); mm_init_owner(mm, p); - mm_init_pasid(mm); + mm_pasid_init(mm); RCU_INIT_POINTER(mm->exe_file, NULL); mmu_notifier_subscriptions_init(mm); init_tlb_flush_pending(mm); diff --git a/mm/init-mm.c b/mm/init-mm.c index b4a6f38fb51d..fbe7844d0912 100644 --- a/mm/init-mm.c +++ b/mm/init-mm.c @@ -10,6 +10,7 @@ =20 #include #include +#include #include =20 #ifndef INIT_MM_CONTEXT @@ -38,6 +39,9 @@ struct mm_struct init_mm =3D { .mmlist =3D LIST_HEAD_INIT(init_mm.mmlist), .user_ns =3D &init_user_ns, .cpu_bitmap =3D CPU_BITS_NONE, +#ifdef CONFIG_IOMMU_SVA + .pasid =3D INVALID_IOASID, +#endif INIT_MM_CONTEXT(init_mm) }; =20 --=20 2.35.1 From nobody Mon Jun 29 15:59:28 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40122C3527A for ; Tue, 8 Feb 2022 01:05:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344751AbiBHBD4 (ORCPT ); Mon, 7 Feb 2022 20:03:56 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35672 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245469AbiBGXDb (ORCPT ); Mon, 7 Feb 2022 18:03:31 -0500 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 34C5DC061355 for ; Mon, 7 Feb 2022 15:03:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1644275010; x=1675811010; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Zl/xFzvxxZzaswNfksOIBzdKah3PcR77VUXQ0r6gtnw=; b=HFTSAXdr47FlTCim/YO5mSRvBircEssI0GB+3YJ5mnooklYl8mgeNwRm xnrO/Y1HJM87HSNasKs/c8qSp9wSZ+OJQBhOJ/DXzQE/X2J44Qb0lcEG2 Ud7oVkDOVoBWYLxO8j24VEi+u+flHIC5KPcud6Nc8+evlTY00GYUJ0og4 Kb/6kstLMSHMwfFH2KJT3BjqnSRAXs+Bn3fDJQn4VgS0ieAn+AfVyUjFB oDgHQx+au+hEzLPThrfCusND1sNw/zZSr4ydPLOr/uy+rpnzuyIi5i/aX tC6eZ1CO5B4x0JWjQcIvtyDRHGLeWHBA/2oXXYECqo10kxwkRylYGRZZg Q==; X-IronPort-AV: E=McAfee;i="6200,9189,10251"; a="229475013" X-IronPort-AV: E=Sophos;i="5.88,351,1635231600"; d="scan'208";a="229475013" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Feb 2022 15:03:04 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,351,1635231600"; d="scan'208";a="540324051" Received: from otcwcpicx3.sc.intel.com ([172.25.55.73]) by orsmga008.jf.intel.com with ESMTP; 07 Feb 2022 15:03:03 -0800 From: Fenghua Yu To: "Thomas Gleixner" , "Dave Hansen" , "Ingo Molnar" , "Borislav Petkov" , "Peter Zijlstra" , "Andy Lutomirski" , "Tony Luck" , "Lu Baolu" , "Joerg Roedel" , Josh Poimboeuf , "Jacob Pan" , "Ashok Raj" , "Ravi V Shankar" Cc: iommu@lists.linux-foundation.org, "x86" , "linux-kernel" , Fenghua Yu Subject: [PATCH v4 05/11] iommu/sva: Assign a PASID to mm on PASID allocation and free it on mm exit Date: Mon, 7 Feb 2022 15:02:48 -0800 Message-Id: <20220207230254.3342514-6-fenghua.yu@intel.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220207230254.3342514-1-fenghua.yu@intel.com> References: <20220207230254.3342514-1-fenghua.yu@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" PASIDs are process wide. It was attempted to use refcounted PASIDs to free them when the last thread drops the refcount. This turned out to be complex and error prone. Given the fact that the PASID space is 20 bits, which allows up to 1M processes to have a PASID associated concurrently, PASID resource exhaustion is not a realistic concern. Therefore it was decided to simplify the approach and stick with lazy on demand PASID allocation, but drop the eager free approach and make a allocated PASID lifetime bound to the life time of the process. Get rid of the refcounting mechanisms and replace/rename the interfaces to reflect this new approach. Suggested-by: Dave Hansen Signed-off-by: Fenghua Yu Reviewed-by: Tony Luck Acked-by: Joerg Roedel Reviewed-by: Jacob Pan Reviewed-by: Lu Baolu Reviewed-by: Thomas Gleixner --- v4: - Update the commit message (Thomas). v3: - Rename mm_pasid_get() to mm_pasid_set() (Thomas). - Remove ioasid_get() because it's not used any more when the IOASID is freed on mm exit (Thomas). - Remove PASID's refcount exercise in ioasid_put() and rename ioasid_put() to ioasid_free() (Thomas). v2: - Free PASID on mm exit instead of in exit(2) or unbind() (Thomas, AndyL, PeterZ) - Add mm_pasid_init(), mm_pasid_get(), and mm_pasid_drop() functions in mm. So the mm's PASID operations are generic for both X86 and ARM (Dave Hansen) .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 5 +-- drivers/iommu/intel/iommu.c | 4 +- drivers/iommu/intel/svm.c | 9 ----- drivers/iommu/ioasid.c | 39 ++----------------- drivers/iommu/iommu-sva-lib.c | 39 ++++++------------- drivers/iommu/iommu-sva-lib.h | 1 - include/linux/ioasid.h | 12 +----- include/linux/sched/mm.h | 16 ++++++++ kernel/fork.c | 1 + 9 files changed, 38 insertions(+), 88 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iomm= u/arm/arm-smmu-v3/arm-smmu-v3-sva.c index a737ba5f727e..22ddd05bbdcd 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -340,14 +340,12 @@ __arm_smmu_sva_bind(struct device *dev, struct mm_str= uct *mm) bond->smmu_mn =3D arm_smmu_mmu_notifier_get(smmu_domain, mm); if (IS_ERR(bond->smmu_mn)) { ret =3D PTR_ERR(bond->smmu_mn); - goto err_free_pasid; + goto err_free_bond; } =20 list_add(&bond->list, &master->bonds); return &bond->sva; =20 -err_free_pasid: - iommu_sva_free_pasid(mm); err_free_bond: kfree(bond); return ERR_PTR(ret); @@ -377,7 +375,6 @@ void arm_smmu_sva_unbind(struct iommu_sva *handle) if (refcount_dec_and_test(&bond->refs)) { list_del(&bond->list); arm_smmu_mmu_notifier_put(bond->smmu_mn); - iommu_sva_free_pasid(bond->mm); kfree(bond); } mutex_unlock(&sva_lock); diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 92fea3fbbb11..ef03b2176bbd 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -4781,7 +4781,7 @@ static int aux_domain_add_dev(struct dmar_domain *dom= ain, link_failed: spin_unlock_irqrestore(&device_domain_lock, flags); if (list_empty(&domain->subdevices) && domain->default_pasid > 0) - ioasid_put(domain->default_pasid); + ioasid_free(domain->default_pasid); =20 return ret; } @@ -4811,7 +4811,7 @@ static void aux_domain_remove_dev(struct dmar_domain = *domain, spin_unlock_irqrestore(&device_domain_lock, flags); =20 if (list_empty(&domain->subdevices) && domain->default_pasid > 0) - ioasid_put(domain->default_pasid); + ioasid_free(domain->default_pasid); } =20 static int prepare_domain_attach_device(struct iommu_domain *domain, diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index 5b5d69b04fcc..51ac2096b3da 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -514,11 +514,6 @@ static int intel_svm_alloc_pasid(struct device *dev, s= truct mm_struct *mm, return iommu_sva_alloc_pasid(mm, PASID_MIN, max_pasid - 1); } =20 -static void intel_svm_free_pasid(struct mm_struct *mm) -{ - iommu_sva_free_pasid(mm); -} - static struct iommu_sva *intel_svm_bind_mm(struct intel_iommu *iommu, struct device *dev, struct mm_struct *mm, @@ -662,8 +657,6 @@ static int intel_svm_unbind_mm(struct device *dev, u32 = pasid) kfree(svm); } } - /* Drop a PASID reference and free it if no reference. */ - intel_svm_free_pasid(mm); } out: return ret; @@ -1047,8 +1040,6 @@ struct iommu_sva *intel_svm_bind(struct device *dev, = struct mm_struct *mm, void } =20 sva =3D intel_svm_bind_mm(iommu, dev, mm, flags); - if (IS_ERR_OR_NULL(sva)) - intel_svm_free_pasid(mm); mutex_unlock(&pasid_mutex); =20 return sva; diff --git a/drivers/iommu/ioasid.c b/drivers/iommu/ioasid.c index 06fee7416816..a786c034907c 100644 --- a/drivers/iommu/ioasid.c +++ b/drivers/iommu/ioasid.c @@ -2,7 +2,7 @@ /* * I/O Address Space ID allocator. There is one global IOASID space, split= into * subsets. Users create a subset with DECLARE_IOASID_SET, then allocate a= nd - * free IOASIDs with ioasid_alloc and ioasid_put. + * free IOASIDs with ioasid_alloc() and ioasid_free(). */ #include #include @@ -15,7 +15,6 @@ struct ioasid_data { struct ioasid_set *set; void *private; struct rcu_head rcu; - refcount_t refs; }; =20 /* @@ -315,7 +314,6 @@ ioasid_t ioasid_alloc(struct ioasid_set *set, ioasid_t = min, ioasid_t max, =20 data->set =3D set; data->private =3D private; - refcount_set(&data->refs, 1); =20 /* * Custom allocator needs allocator data to perform platform specific @@ -348,35 +346,11 @@ ioasid_t ioasid_alloc(struct ioasid_set *set, ioasid_= t min, ioasid_t max, EXPORT_SYMBOL_GPL(ioasid_alloc); =20 /** - * ioasid_get - obtain a reference to the IOASID - * @ioasid: the ID to get - */ -void ioasid_get(ioasid_t ioasid) -{ - struct ioasid_data *ioasid_data; - - spin_lock(&ioasid_allocator_lock); - ioasid_data =3D xa_load(&active_allocator->xa, ioasid); - if (ioasid_data) - refcount_inc(&ioasid_data->refs); - else - WARN_ON(1); - spin_unlock(&ioasid_allocator_lock); -} -EXPORT_SYMBOL_GPL(ioasid_get); - -/** - * ioasid_put - Release a reference to an ioasid + * ioasid_free - Free an ioasid * @ioasid: the ID to remove - * - * Put a reference to the IOASID, free it when the number of references dr= ops to - * zero. - * - * Return: %true if the IOASID was freed, %false otherwise. */ -bool ioasid_put(ioasid_t ioasid) +void ioasid_free(ioasid_t ioasid) { - bool free =3D false; struct ioasid_data *ioasid_data; =20 spin_lock(&ioasid_allocator_lock); @@ -386,10 +360,6 @@ bool ioasid_put(ioasid_t ioasid) goto exit_unlock; } =20 - free =3D refcount_dec_and_test(&ioasid_data->refs); - if (!free) - goto exit_unlock; - active_allocator->ops->free(ioasid, active_allocator->ops->pdata); /* Custom allocator needs additional steps to free the xa element */ if (active_allocator->flags & IOASID_ALLOCATOR_CUSTOM) { @@ -399,9 +369,8 @@ bool ioasid_put(ioasid_t ioasid) =20 exit_unlock: spin_unlock(&ioasid_allocator_lock); - return free; } -EXPORT_SYMBOL_GPL(ioasid_put); +EXPORT_SYMBOL_GPL(ioasid_free); =20 /** * ioasid_find - Find IOASID data diff --git a/drivers/iommu/iommu-sva-lib.c b/drivers/iommu/iommu-sva-lib.c index bd41405d34e9..106506143896 100644 --- a/drivers/iommu/iommu-sva-lib.c +++ b/drivers/iommu/iommu-sva-lib.c @@ -18,8 +18,7 @@ static DECLARE_IOASID_SET(iommu_sva_pasid); * * Try to allocate a PASID for this mm, or take a reference to the existin= g one * provided it fits within the [@min, @max] range. On success the PASID is - * available in mm->pasid, and must be released with iommu_sva_free_pasid(= ). - * @min must be greater than 0, because 0 indicates an unused mm->pasid. + * available in mm->pasid and will be available for the lifetime of the mm. * * Returns 0 on success and < 0 on error. */ @@ -33,38 +32,24 @@ int iommu_sva_alloc_pasid(struct mm_struct *mm, ioasid_= t min, ioasid_t max) return -EINVAL; =20 mutex_lock(&iommu_sva_lock); - if (mm->pasid) { - if (mm->pasid >=3D min && mm->pasid <=3D max) - ioasid_get(mm->pasid); - else + /* Is a PASID already associated with this mm? */ + if (pasid_valid(mm->pasid)) { + if (mm->pasid < min || mm->pasid >=3D max) ret =3D -EOVERFLOW; - } else { - pasid =3D ioasid_alloc(&iommu_sva_pasid, min, max, mm); - if (pasid =3D=3D INVALID_IOASID) - ret =3D -ENOMEM; - else - mm->pasid =3D pasid; + goto out; } + + pasid =3D ioasid_alloc(&iommu_sva_pasid, min, max, mm); + if (!pasid_valid(pasid)) + ret =3D -ENOMEM; + else + mm_pasid_set(mm, pasid); +out: mutex_unlock(&iommu_sva_lock); return ret; } EXPORT_SYMBOL_GPL(iommu_sva_alloc_pasid); =20 -/** - * iommu_sva_free_pasid - Release the mm's PASID - * @mm: the mm - * - * Drop one reference to a PASID allocated with iommu_sva_alloc_pasid() - */ -void iommu_sva_free_pasid(struct mm_struct *mm) -{ - mutex_lock(&iommu_sva_lock); - if (ioasid_put(mm->pasid)) - mm->pasid =3D 0; - mutex_unlock(&iommu_sva_lock); -} -EXPORT_SYMBOL_GPL(iommu_sva_free_pasid); - /* ioasid_find getter() requires a void * argument */ static bool __mmget_not_zero(void *mm) { diff --git a/drivers/iommu/iommu-sva-lib.h b/drivers/iommu/iommu-sva-lib.h index 95dc3ebc1928..8909ea1094e3 100644 --- a/drivers/iommu/iommu-sva-lib.h +++ b/drivers/iommu/iommu-sva-lib.h @@ -9,7 +9,6 @@ #include =20 int iommu_sva_alloc_pasid(struct mm_struct *mm, ioasid_t min, ioasid_t max= ); -void iommu_sva_free_pasid(struct mm_struct *mm); struct mm_struct *iommu_sva_find(ioasid_t pasid); =20 /* I/O Page fault */ diff --git a/include/linux/ioasid.h b/include/linux/ioasid.h index 2237f64dbaae..af1c9d62e642 100644 --- a/include/linux/ioasid.h +++ b/include/linux/ioasid.h @@ -34,8 +34,7 @@ struct ioasid_allocator_ops { #if IS_ENABLED(CONFIG_IOASID) ioasid_t ioasid_alloc(struct ioasid_set *set, ioasid_t min, ioasid_t max, void *private); -void ioasid_get(ioasid_t ioasid); -bool ioasid_put(ioasid_t ioasid); +void ioasid_free(ioasid_t ioasid); void *ioasid_find(struct ioasid_set *set, ioasid_t ioasid, bool (*getter)(void *)); int ioasid_register_allocator(struct ioasid_allocator_ops *allocator); @@ -53,14 +52,7 @@ static inline ioasid_t ioasid_alloc(struct ioasid_set *s= et, ioasid_t min, return INVALID_IOASID; } =20 -static inline void ioasid_get(ioasid_t ioasid) -{ -} - -static inline bool ioasid_put(ioasid_t ioasid) -{ - return false; -} +static inline void ioasid_free(ioasid_t ioasid) { } =20 static inline void *ioasid_find(struct ioasid_set *set, ioasid_t ioasid, bool (*getter)(void *)) diff --git a/include/linux/sched/mm.h b/include/linux/sched/mm.h index c74d1edbac2f..a80356e9dc69 100644 --- a/include/linux/sched/mm.h +++ b/include/linux/sched/mm.h @@ -439,8 +439,24 @@ static inline void mm_pasid_init(struct mm_struct *mm) { mm->pasid =3D INVALID_IOASID; } + +/* Associate a PASID with an mm_struct: */ +static inline void mm_pasid_set(struct mm_struct *mm, u32 pasid) +{ + mm->pasid =3D pasid; +} + +static inline void mm_pasid_drop(struct mm_struct *mm) +{ + if (pasid_valid(mm->pasid)) { + ioasid_free(mm->pasid); + mm->pasid =3D INVALID_IOASID; + } +} #else static inline void mm_pasid_init(struct mm_struct *mm) {} +static inline void mm_pasid_set(struct mm_struct *mm, u32 pasid) {} +static inline void mm_pasid_drop(struct mm_struct *mm) {} #endif =20 #endif /* _LINUX_SCHED_MM_H */ diff --git a/kernel/fork.c b/kernel/fork.c index deacd2c17a7f..c03c6682464c 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -1115,6 +1115,7 @@ static inline void __mmput(struct mm_struct *mm) } if (mm->binfmt) module_put(mm->binfmt->module); + mm_pasid_drop(mm); mmdrop(mm); } =20 --=20 2.35.1 From nobody Mon Jun 29 15:59:28 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ECCE8C35278 for ; Tue, 8 Feb 2022 01:05:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344668AbiBHBDu (ORCPT ); Mon, 7 Feb 2022 20:03:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35640 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245457AbiBGXD3 (ORCPT ); Mon, 7 Feb 2022 18:03:29 -0500 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 47D28C043180 for ; Mon, 7 Feb 2022 15:03:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1644275009; x=1675811009; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=pSiI5bRDWKZ8wcmfvSwFj0dyl6gbbkbtffSYluVkHLo=; b=F1vjZWnddLAVvyvUSU4871QEz5idrLoEk5yShnWA8YgS9bzzOdQZrC1t R6iMD68wvheMDsMSUC0hdGOHV9Lt/o3NhhvTLdJ0wya7dSo+vEmV6hdYt vjr9uHzO9Uj1piB3DcDChDOTgw5R97fUuGdoN2ooK/X2l//xKaHHrmCB4 UaplCBBQoX/lVujeeGnfDJYxxpuw9SGRqbJgaBjIHe/2geUj7n+cwyGs6 xxDhUDxACKxlUIGc7dVi0dMrDKigfWHLoFTburJkokjYJhku3EcjB/J69 IwMLb16Z3dL8EKTGwC66WMgzHbLgvMYeX83Qxx0dUeXU5Y11yj+0sPsIN w==; X-IronPort-AV: E=McAfee;i="6200,9189,10251"; a="229475014" X-IronPort-AV: E=Sophos;i="5.88,351,1635231600"; d="scan'208";a="229475014" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Feb 2022 15:03:05 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,351,1635231600"; d="scan'208";a="540324061" Received: from otcwcpicx3.sc.intel.com ([172.25.55.73]) by orsmga008.jf.intel.com with ESMTP; 07 Feb 2022 15:03:04 -0800 From: Fenghua Yu To: "Thomas Gleixner" , "Dave Hansen" , "Ingo Molnar" , "Borislav Petkov" , "Peter Zijlstra" , "Andy Lutomirski" , "Tony Luck" , "Lu Baolu" , "Joerg Roedel" , Josh Poimboeuf , "Jacob Pan" , "Ashok Raj" , "Ravi V Shankar" Cc: iommu@lists.linux-foundation.org, "x86" , "linux-kernel" , Fenghua Yu Subject: [PATCH v4 06/11] x86/fpu: Clear PASID when copying fpstate Date: Mon, 7 Feb 2022 15:02:49 -0800 Message-Id: <20220207230254.3342514-7-fenghua.yu@intel.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220207230254.3342514-1-fenghua.yu@intel.com> References: <20220207230254.3342514-1-fenghua.yu@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The kernel must allocate a Process Address Space ID (PASID) on behalf of each process which will use ENQCMD and program it into the new MSR to communicate the process identity to platform hardware. ENQCMD uses the PASID stored in this MSR to tag requests from this process. The PASID state must be cleared on fork() since fork creates a new address space. For clone(), it would be functionally OK to copy the PASID. However, clearing it is _also_ functionally OK since any PASID use will trigger the #GP handler to populate the MSR. Copying the PASID state has two main downsides: * It requires differentiating fork() and clone() in the code, both in the FPU code and keeping tsk->pasid_activated consistent. * It guarantees that the PASID is out of its init state, which incurs small but non-zero cost on every XSAVE/XRSTOR. The main downside of clearing the PASID at fpstate copy is the future, one-time #GP for the thread. Use the simplest approach: clear the PASID state both on clone() and fork(). Rely on the #GP handler for MSR population in children. Also, just clear the PASID bit from xfeatures if XSAVE is supported. This will have no effect on systems that do not have PASID support. It is virtually zero overhead because 'dst_fpu' was just written and the whole thing is cache hot. Signed-off-by: Fenghua Yu Reviewed-by: Tony Luck Reviewed-by: Thomas Gleixner --- v4: - Add "Reviewed-by: Thomas Gleixner " (Thomas). v2: - Rewrite changelog (Dave Hansen). - Move xfeature tweaking into fpu_clone() and make it unconditional if XSAVE is supported (Dave Hansen). arch/x86/kernel/fpu/core.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/arch/x86/kernel/fpu/core.c b/arch/x86/kernel/fpu/core.c index 8dea01ffc5c1..19821f027cb3 100644 --- a/arch/x86/kernel/fpu/core.c +++ b/arch/x86/kernel/fpu/core.c @@ -612,6 +612,13 @@ int fpu_clone(struct task_struct *dst, unsigned long c= lone_flags) fpu_inherit_perms(dst_fpu); fpregs_unlock(); =20 + /* + * Children never inherit PASID state. + * Force it to have its init value: + */ + if (use_xsave()) + dst_fpu->fpstate->regs.xsave.header.xfeatures &=3D ~XFEATURE_MASK_PASID; + trace_x86_fpu_copy_src(src_fpu); trace_x86_fpu_copy_dst(dst_fpu); =20 --=20 2.35.1 From nobody Mon Jun 29 15:59:28 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0A073C4707F for ; Tue, 8 Feb 2022 01:05:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344699AbiBHBDv (ORCPT ); Mon, 7 Feb 2022 20:03:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35642 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245441AbiBGXD3 (ORCPT ); Mon, 7 Feb 2022 18:03:29 -0500 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4FAE8C043181 for ; Mon, 7 Feb 2022 15:03:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1644275009; x=1675811009; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=qAN6i2tcGpKHAdLmfykkJrrQYTFoSpMh8/ZHA7l/RSs=; b=GpP7HVsV0ZT44Q5jMpLdlHIl/lreOIp502LymK7yz2L5lx8uTeLcNobf FE2YGwq+yLLs7g0Y7mXF4QBSCF7whIi4u4tkwhOxn3LO7S0EtV73tq27k p9UMKLvR56USnH13L286N7eqdmDqxMTh+aMPqSXOKksd5t3tinXW1LSNO p4DJAhCWqLvYpziUD2R8XCrOy4cI/mQxnK2JJU044eq2mG48CLtLHzxrJ 4Pg4L8W7CUZXcO7eSZW62csizyxwZYQpFe9pO3sw39vkYFPSQDZXkhlrU FiQuU4uwudCFlEnNQDa5sxsJjm7BBVpdNYzwQ2+/yPsvvXKGFKOAEFqk0 g==; X-IronPort-AV: E=McAfee;i="6200,9189,10251"; a="229475017" X-IronPort-AV: E=Sophos;i="5.88,351,1635231600"; d="scan'208";a="229475017" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Feb 2022 15:03:06 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,351,1635231600"; d="scan'208";a="540324076" Received: from otcwcpicx3.sc.intel.com ([172.25.55.73]) by orsmga008.jf.intel.com with ESMTP; 07 Feb 2022 15:03:05 -0800 From: Fenghua Yu To: "Thomas Gleixner" , "Dave Hansen" , "Ingo Molnar" , "Borislav Petkov" , "Peter Zijlstra" , "Andy Lutomirski" , "Tony Luck" , "Lu Baolu" , "Joerg Roedel" , Josh Poimboeuf , "Jacob Pan" , "Ashok Raj" , "Ravi V Shankar" Cc: iommu@lists.linux-foundation.org, "x86" , "linux-kernel" , Fenghua Yu Subject: [PATCH v4 07/11] sched: Define and initialize a flag to identify valid PASID in the task Date: Mon, 7 Feb 2022 15:02:50 -0800 Message-Id: <20220207230254.3342514-8-fenghua.yu@intel.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220207230254.3342514-1-fenghua.yu@intel.com> References: <20220207230254.3342514-1-fenghua.yu@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Peter Zijlstra Add a new single bit field to the task structure to track whether this task has initialized the IA32_PASID MSR to the mm's PASID. Initialize the field to zero when creating a new task with fork/clone. Signed-off-by: Peter Zijlstra Co-developed-by: Fenghua Yu Signed-off-by: Fenghua Yu Reviewed-by: Tony Luck Reviewed-by: Thomas Gleixner --- v4: - Add "Reviewed-by: Thomas Gleixner " (Thomas). v2: - Change condition to more accurate CONFIG_IOMMU_SVA (Jacob) include/linux/sched.h | 3 +++ kernel/fork.c | 4 ++++ 2 files changed, 7 insertions(+) diff --git a/include/linux/sched.h b/include/linux/sched.h index 75ba8aa60248..4e5de3aed410 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -938,6 +938,9 @@ struct task_struct { /* Recursion prevention for eventfd_signal() */ unsigned in_eventfd_signal:1; #endif +#ifdef CONFIG_IOMMU_SVA + unsigned pasid_activated:1; +#endif =20 unsigned long atomic_flags; /* Flags requiring atomic access. */ =20 diff --git a/kernel/fork.c b/kernel/fork.c index c03c6682464c..51fd1df994b7 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -968,6 +968,10 @@ static struct task_struct *dup_task_struct(struct task= _struct *orig, int node) tsk->use_memdelay =3D 0; #endif =20 +#ifdef CONFIG_IOMMU_SVA + tsk->pasid_activated =3D 0; +#endif + #ifdef CONFIG_MEMCG tsk->active_memcg =3D NULL; #endif --=20 2.35.1 From nobody Mon Jun 29 15:59:28 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8572C35276 for ; Tue, 8 Feb 2022 01:05:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237090AbiBHBDq (ORCPT ); Mon, 7 Feb 2022 20:03:46 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35656 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245459AbiBGXDa (ORCPT ); Mon, 7 Feb 2022 18:03:30 -0500 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 36AF2C0612A4 for ; Mon, 7 Feb 2022 15:03:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1644275010; x=1675811010; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=G2ko2lJpyoW8LzCkDhi+2nZvr53q38fpGk6sB2Ue41w=; b=lJIboQtle/aq9nhdr4qja4xr8ksiEiC18tskTCVi3qQDGfzQN4+ljJMS mptSYo/41KEZbS4E5pk5KKW6WFMDpL6Kd7buBHX5skx+DQLb6IT1dTY5M Sja6qBCHHf3LTDQFsEn3TrZQdzC8tANx7YCa2YQTa3rwQLQiiYliIltnq j/wAxMP4ldLYg7HX4gflDAKFyxMqkBELeX3J5igCDhIWYRwrF1RnpR6gf TdiZWaIlASQBACnlSBTO1/adbz9/YA8Dbc72quHRlbk+tGDouY2Enl4WG wyqTQTBoMUcsMU4hwaVN5E0ZU1LUUJa0URoNNEJG3Hrrx6bYUWofJVaME Q==; X-IronPort-AV: E=McAfee;i="6200,9189,10251"; a="229475019" X-IronPort-AV: E=Sophos;i="5.88,351,1635231600"; d="scan'208";a="229475019" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Feb 2022 15:03:06 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,351,1635231600"; d="scan'208";a="540324084" Received: from otcwcpicx3.sc.intel.com ([172.25.55.73]) by orsmga008.jf.intel.com with ESMTP; 07 Feb 2022 15:03:05 -0800 From: Fenghua Yu To: "Thomas Gleixner" , "Dave Hansen" , "Ingo Molnar" , "Borislav Petkov" , "Peter Zijlstra" , "Andy Lutomirski" , "Tony Luck" , "Lu Baolu" , "Joerg Roedel" , Josh Poimboeuf , "Jacob Pan" , "Ashok Raj" , "Ravi V Shankar" Cc: iommu@lists.linux-foundation.org, "x86" , "linux-kernel" , Fenghua Yu Subject: [PATCH v4 08/11] x86/traps: Demand-populate PASID MSR via #GP Date: Mon, 7 Feb 2022 15:02:51 -0800 Message-Id: <20220207230254.3342514-9-fenghua.yu@intel.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220207230254.3342514-1-fenghua.yu@intel.com> References: <20220207230254.3342514-1-fenghua.yu@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" All tasks start with PASID state disabled. This means that the first time they execute an ENQCMD instruction they will take a #GP fault. Modify the #GP fault handler to check if the "mm" for the task has already been allocated a PASID. If so, try to fix the #GP fault by loading the IA32_PASID MSR. Signed-off-by: Fenghua Yu Reviewed-by: Tony Luck Reviewed-by: Thomas Gleixner --- v4: - Add "Reviewed-by: Thomas Gleixner " (Thomas). v2: - Directly write IA32_PASID MSR in fixup while local IRQ is still disabled (Thomas) - Move #ifdef over to CONFIG_IOMMU_SVA since it is what defines mm->pasid and ->pasid_activated (Dave Hansen). - Rename try_fixup_pasid() -> try_fixup_enqcmd_gp(). This code really is highly specific to ENQCMD, not PASIDs (Dave Hansen). - Add lockdep assert and comment about context (Dave Hansen). - Re-flow the if() mess (Dave Hansen). arch/x86/kernel/traps.c | 55 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 55 insertions(+) diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index c9d566dcf89a..7ef00dee35be 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -39,6 +39,7 @@ #include #include #include +#include =20 #include #include @@ -559,6 +560,57 @@ static bool fixup_iopl_exception(struct pt_regs *regs) return true; } =20 +/* + * The unprivileged ENQCMD instruction generates #GPs if the + * IA32_PASID MSR has not been populated. If possible, populate + * the MSR from a PASID previously allocated to the mm. + */ +static bool try_fixup_enqcmd_gp(void) +{ +#ifdef CONFIG_IOMMU_SVA + u32 pasid; + + /* + * MSR_IA32_PASID is managed using XSAVE. Directly + * writing to the MSR is only possible when fpregs + * are valid and the fpstate is not. This is + * guaranteed when handling a userspace exception + * in *before* interrupts are re-enabled. + */ + lockdep_assert_irqs_disabled(); + + /* + * Hardware without ENQCMD will not generate + * #GPs that can be fixed up here. + */ + if (!cpu_feature_enabled(X86_FEATURE_ENQCMD)) + return false; + + pasid =3D current->mm->pasid; + + /* + * If the mm has not been allocated a + * PASID, the #GP can not be fixed up. + */ + if (!pasid_valid(pasid)) + return false; + + /* + * Did this thread already have its PASID activated? + * If so, the #GP must be from something else. + */ + if (current->pasid_activated) + return false; + + wrmsrl(MSR_IA32_PASID, pasid | MSR_IA32_PASID_VALID); + current->pasid_activated =3D 1; + + return true; +#else + return false; +#endif +} + DEFINE_IDTENTRY_ERRORCODE(exc_general_protection) { char desc[sizeof(GPFSTR) + 50 + 2*sizeof(unsigned long) + 1] =3D GPFSTR; @@ -567,6 +619,9 @@ DEFINE_IDTENTRY_ERRORCODE(exc_general_protection) unsigned long gp_addr; int ret; =20 + if (user_mode(regs) && try_fixup_enqcmd_gp()) + return; + cond_local_irq_enable(regs); =20 if (static_cpu_has(X86_FEATURE_UMIP)) { --=20 2.35.1 From nobody Mon Jun 29 15:59:28 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56D20C3527B for ; Tue, 8 Feb 2022 01:05:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344772AbiBHBD7 (ORCPT ); Mon, 7 Feb 2022 20:03:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35670 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245462AbiBGXDb (ORCPT ); Mon, 7 Feb 2022 18:03:31 -0500 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4ED64C061A73 for ; Mon, 7 Feb 2022 15:03:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1644275010; x=1675811010; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=dcqYJ5g1xNlLeHNxm7HxWP+rdA8fELICnTfN3W2xdQk=; b=nmxgbLL7NVf4C7n7n+dmL/SvxJCdR3DK3O9e+nGiQHuRduZHji/eblaF 8tFqFMwC3IaAN8mYQAqOQr65Uq5WuUndZKkhgoVBilU9GmsePCtNZBfxi zzgBgwSV1ZZ0GpuspS3yPZHr2ZndL+OQcsyx+KF+fRdhhWPvBO9lN2+GP p2Bti4keTRuSZw1/F96IpCg5hgUAwB4bSfCBX//rZNufdTzCArDHH6kHN JNTOit6n6qgFyLRxpPs1CNzoSQ6RK7ZscKWEZijzluba6XaxMyYlAouQh cc6MG+nb23kZOWY4fRfHwPL+eWCHJjdcej/G0dgLNZSLgjCDh4cWSUPzt g==; X-IronPort-AV: E=McAfee;i="6200,9189,10251"; a="229475020" X-IronPort-AV: E=Sophos;i="5.88,351,1635231600"; d="scan'208";a="229475020" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Feb 2022 15:03:06 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,351,1635231600"; d="scan'208";a="540324091" Received: from otcwcpicx3.sc.intel.com ([172.25.55.73]) by orsmga008.jf.intel.com with ESMTP; 07 Feb 2022 15:03:06 -0800 From: Fenghua Yu To: "Thomas Gleixner" , "Dave Hansen" , "Ingo Molnar" , "Borislav Petkov" , "Peter Zijlstra" , "Andy Lutomirski" , "Tony Luck" , "Lu Baolu" , "Joerg Roedel" , Josh Poimboeuf , "Jacob Pan" , "Ashok Raj" , "Ravi V Shankar" Cc: iommu@lists.linux-foundation.org, "x86" , "linux-kernel" , Fenghua Yu Subject: [PATCH v4 09/11] x86/cpufeatures: Re-enable ENQCMD Date: Mon, 7 Feb 2022 15:02:52 -0800 Message-Id: <20220207230254.3342514-10-fenghua.yu@intel.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220207230254.3342514-1-fenghua.yu@intel.com> References: <20220207230254.3342514-1-fenghua.yu@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Since ENQCMD is handled by #GP fix up, it can be re-enabled. The ENQCMD feature can only be used if CONFIG_INTEL_IOMMU_SVM is set. Add X86_FEATURE_ENQCMD to the disabled features mask as appropriate so that cpu_feature_enabled() can be used to check the feature. Signed-off-by: Fenghua Yu Reviewed-by: Tony Luck Reviewed-by: Thomas Gleixner --- v4: - Add "Reviewed-by: Thomas Gleixner " (Thomas). v2: - Update the commit message (Tony). arch/x86/include/asm/disabled-features.h | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/disabled-features.h b/arch/x86/include/as= m/disabled-features.h index 8f28fafa98b3..1231d63f836d 100644 --- a/arch/x86/include/asm/disabled-features.h +++ b/arch/x86/include/asm/disabled-features.h @@ -56,8 +56,11 @@ # define DISABLE_PTI (1 << (X86_FEATURE_PTI & 31)) #endif =20 -/* Force disable because it's broken beyond repair */ -#define DISABLE_ENQCMD (1 << (X86_FEATURE_ENQCMD & 31)) +#ifdef CONFIG_INTEL_IOMMU_SVM +# define DISABLE_ENQCMD 0 +#else +# define DISABLE_ENQCMD (1 << (X86_FEATURE_ENQCMD & 31)) +#endif =20 #ifdef CONFIG_X86_SGX # define DISABLE_SGX 0 --=20 2.35.1 From nobody Mon Jun 29 15:59:28 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE80DC4707E for ; Tue, 8 Feb 2022 01:05:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344649AbiBHBDs (ORCPT ); Mon, 7 Feb 2022 20:03:48 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35644 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245458AbiBGXDa (ORCPT ); Mon, 7 Feb 2022 18:03:30 -0500 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 65761C043188 for ; Mon, 7 Feb 2022 15:03:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1644275009; x=1675811009; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=exni2nSl44fcOYxYrJayxLKBv/TSBnKt2khR1kpp+yw=; b=X4cSqdmE7Mae65zrv5Q39T37YcZ/FBfGhBCYVSLKx0Du8Kwq+lwMWgM9 z0Vx74FS2jBYMLig/TI1jIwSSKvDpbDdXrEXUBsavNLSiTCVFtBtT9sit RPvUjUQfusXbVTYJkfd9bZxh1H3Y8BMzkyGNRksN7IqoSVQOYF6OTE+4u 5x2MuxLYl8SyCBNNHNwLYrvMJeNkb4/6hRk93mQCwH0+iBx55oiva86I/ Ijgwm9uZsAjdOzHUJb2mtGaoibaxD5kwM03n+ccQDQxOCc5pambyATuph e5eSTvZHn2dHmyCP1L1z/fuhRvcwRj3MAPNm4y2WF0s19xJerNBOBy4DN w==; X-IronPort-AV: E=McAfee;i="6200,9189,10251"; a="229475023" X-IronPort-AV: E=Sophos;i="5.88,351,1635231600"; d="scan'208";a="229475023" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Feb 2022 15:03:07 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,351,1635231600"; d="scan'208";a="540324097" Received: from otcwcpicx3.sc.intel.com ([172.25.55.73]) by orsmga008.jf.intel.com with ESMTP; 07 Feb 2022 15:03:06 -0800 From: Fenghua Yu To: "Thomas Gleixner" , "Dave Hansen" , "Ingo Molnar" , "Borislav Petkov" , "Peter Zijlstra" , "Andy Lutomirski" , "Tony Luck" , "Lu Baolu" , "Joerg Roedel" , Josh Poimboeuf , "Jacob Pan" , "Ashok Raj" , "Ravi V Shankar" Cc: iommu@lists.linux-foundation.org, "x86" , "linux-kernel" , Fenghua Yu Subject: [PATCH v4 10/11] tools/objtool: Check for use of the ENQCMD instruction in the kernel Date: Mon, 7 Feb 2022 15:02:53 -0800 Message-Id: <20220207230254.3342514-11-fenghua.yu@intel.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220207230254.3342514-1-fenghua.yu@intel.com> References: <20220207230254.3342514-1-fenghua.yu@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The ENQCMD implicitly accesses the PASID_MSR to fill in the pasid field of the descriptor being submitted to an accelerator. But there is no precise (and stable across kernel changes) point at which the PASID_MSR is updated from the value for one task to the next. Kernel code that uses accelerators must always use the ENQCMDS instruction which does not access the PASID_MSR. Check for use of the ENQCMD instruction in the kernel and warn on its usage. Signed-off-by: Fenghua Yu Reviewed-by: Tony Luck Acked-by: Josh Poimboeuf --- v3: - Add Acked-by: Josh Poimboeuf v2: - Simplify handling ENQCMD (PeterZ and Josh) tools/objtool/arch/x86/decode.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/tools/objtool/arch/x86/decode.c b/tools/objtool/arch/x86/decod= e.c index c10ef78df050..479e769ca324 100644 --- a/tools/objtool/arch/x86/decode.c +++ b/tools/objtool/arch/x86/decode.c @@ -112,7 +112,7 @@ int arch_decode_instruction(struct objtool_file *file, = const struct section *sec const struct elf *elf =3D file->elf; struct insn insn; int x86_64, ret; - unsigned char op1, op2, + unsigned char op1, op2, op3, rex =3D 0, rex_b =3D 0, rex_r =3D 0, rex_w =3D 0, rex_x =3D 0, modrm =3D 0, modrm_mod =3D 0, modrm_rm =3D 0, modrm_reg =3D 0, sib =3D 0, /* sib_scale =3D 0, */ sib_index =3D 0, sib_base =3D 0; @@ -139,6 +139,7 @@ int arch_decode_instruction(struct objtool_file *file, = const struct section *sec =20 op1 =3D insn.opcode.bytes[0]; op2 =3D insn.opcode.bytes[1]; + op3 =3D insn.opcode.bytes[2]; =20 if (insn.rex_prefix.nbytes) { rex =3D insn.rex_prefix.bytes[0]; @@ -491,6 +492,14 @@ int arch_decode_instruction(struct objtool_file *file,= const struct section *sec /* nopl/nopw */ *type =3D INSN_NOP; =20 + } else if (op2 =3D=3D 0x38 && op3 =3D=3D 0xf8) { + if (insn.prefixes.nbytes =3D=3D 1 && + insn.prefixes.bytes[0] =3D=3D 0xf2) { + /* ENQCMD cannot be used in the kernel. */ + WARN("ENQCMD instruction at %s:%lx", sec->name, + offset); + } + } else if (op2 =3D=3D 0xa0 || op2 =3D=3D 0xa8) { =20 /* push fs/gs */ --=20 2.35.1 From nobody Mon Jun 29 15:59:28 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 20242C47080 for ; Tue, 8 Feb 2022 01:05:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344723AbiBHBDw (ORCPT ); Mon, 7 Feb 2022 20:03:52 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35674 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245472AbiBGXDb (ORCPT ); Mon, 7 Feb 2022 18:03:31 -0500 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D0254C06109E for ; Mon, 7 Feb 2022 15:03:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1644275010; x=1675811010; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ncFZ3GgiFenQLT9iFwTMGx9ZJmzYJBlebX18EhhOoaA=; b=k5/Xj5dckNbdlxHhvR554H8HV0oJMNJo4BCJSVyWKuKyv6v7uZQr21vw QIiNtk9O+gkSj59NS/guahtkY6U2FNaKWmsezBQxw93scfV9G1/dsK9Wh AJXnMp6arGuMy4K4B/C/wTTwv3QSseaTb9h7BvtdYxiw9JMYEKg0/fTQO cx4ZxI0yyXN00PYZv1xoSubrRlB/CcabQ81zQfLLeAnFHvhIx5Xh1BdSc SEPKHaHHGp75v/KPl1VPkAq4OlnfCIHYKYLXZ5dqof952DF5dd2QELr/E fKFs9oxj5GSjYbNlcMB3upR8Tj17j16PnsxaIjrfkoUj9P4gyHbyZ+K3p A==; X-IronPort-AV: E=McAfee;i="6200,9189,10251"; a="229475024" X-IronPort-AV: E=Sophos;i="5.88,351,1635231600"; d="scan'208";a="229475024" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Feb 2022 15:03:07 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,351,1635231600"; d="scan'208";a="540324104" Received: from otcwcpicx3.sc.intel.com ([172.25.55.73]) by orsmga008.jf.intel.com with ESMTP; 07 Feb 2022 15:03:07 -0800 From: Fenghua Yu To: "Thomas Gleixner" , "Dave Hansen" , "Ingo Molnar" , "Borislav Petkov" , "Peter Zijlstra" , "Andy Lutomirski" , "Tony Luck" , "Lu Baolu" , "Joerg Roedel" , Josh Poimboeuf , "Jacob Pan" , "Ashok Raj" , "Ravi V Shankar" Cc: iommu@lists.linux-foundation.org, "x86" , "linux-kernel" , Fenghua Yu Subject: [PATCH v4 11/11] docs: x86: Change documentation for SVA (Shared Virtual Addressing) Date: Mon, 7 Feb 2022 15:02:54 -0800 Message-Id: <20220207230254.3342514-12-fenghua.yu@intel.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220207230254.3342514-1-fenghua.yu@intel.com> References: <20220207230254.3342514-1-fenghua.yu@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Since allocating, freeing and fixing up PASID are changed, the documentation is updated to reflect the changes. Originally-by: Ashok Raj Signed-off-by: Fenghua Yu Reviewed-by: Tony Luck --- v3: - Remove PASID refcount description (Thomas). v2: - Update life cycle info (Tony and Thomas). - Update initial PASID value to INVALID_IOASID on fork(). Documentation/x86/sva.rst | 53 ++++++++++++++++++++++++++++++--------- 1 file changed, 41 insertions(+), 12 deletions(-) diff --git a/Documentation/x86/sva.rst b/Documentation/x86/sva.rst index 076efd51ef1f..1a22020735a3 100644 --- a/Documentation/x86/sva.rst +++ b/Documentation/x86/sva.rst @@ -104,18 +104,47 @@ The MSR must be configured on each logical CPU before= any application thread can interact with a device. Threads that belong to the same process share the same page tables, thus the same MSR value. =20 -PASID is cleared when a process is created. The PASID allocation and MSR -programming may occur long after a process and its threads have been creat= ed. -One thread must call iommu_sva_bind_device() to allocate the PASID for the -process. If a thread uses ENQCMD without the MSR first being populated, a = #GP -will be raised. The kernel will update the PASID MSR with the PASID for all -threads in the process. A single process PASID can be used simultaneously -with multiple devices since they all share the same address space. - -One thread can call iommu_sva_unbind_device() to free the allocated PASID. -The kernel will clear the PASID MSR for all threads belonging to the proce= ss. - -New threads inherit the MSR value from the parent. +PASID Life Cycle Management +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D + +PASID is initialized as INVALID_IOASID (-1) when a process is created. + +Only processes that access SVA-capable devices need to have a PASID +allocated. This allocation happens when a process opens/binds an SVA-capab= le +device but finds no PASID for this process. Subsequent binds of the same, = or +other devices will share the same PASID. + +Although the PASID is allocated to the process by opening a device, +it is not active in any of the threads of that process. It's loaded to the +IA32_PASID MSR lazily when a thread tries to submit a work descriptor +to a device using the ENQCMD. + +That first access will trigger a #GP fault because the IA32_PASID MSR +has not been initialized with the PASID value assigned to the process +when the device was opened. The Linux #GP handler notes that a PASID has +been allocated for the process, and so initializes the IA32_PASID MSR +and returns so that the ENQCMD instruction is re-executed. + +On fork(2) or exec(2) the PASID is removed from the process as it no +longer has the same address space that it had when the device was opened. + +On clone(2) the new task shares the same address space, so will be +able to use the PASID allocated to the process. The IA32_PASID is not +preemptively initialized as the PASID value might not be allocated yet or +the kernel does not know whether this thread is going to access the device +and the cleared IA32_PASID MSR reduces context switch overhead by xstate +init optimization. Since #GP faults have to be handled on any threads that +were created before the PASID was assigned to the mm of the process, newly +created threads might as well be treated in a consistent way. + +Due to complexity of freeing the PASID and clearing all IA32_PASID MSRs in +all threads in unbind, free the PASID lazily only on mm exit. + +If a process does a close(2) of the device file descriptor and munmap(2) +of the device MMIO portal, then the driver will unbind the device. The +PASID is still marked VALID in the PASID_MSR for any threads in the +process that accessed the device. But this is harmless as without the +MMIO portal they cannot submit new work to the device. =20 Relationships =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --=20 2.35.1