From nobody Sun Sep 14 04:56:24 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ECA5DC636CC for ; Tue, 31 Jan 2023 07:46:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230124AbjAaHqP (ORCPT ); Tue, 31 Jan 2023 02:46:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47726 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229661AbjAaHqL (ORCPT ); Tue, 31 Jan 2023 02:46:11 -0500 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 849C93B3DE for ; Mon, 30 Jan 2023 23:46:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675151166; x=1706687166; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=s1MxTLdDeLeeouG952McaeDlNjDKN6G16uZN42WANIs=; b=egrv5gbIhaHc3wxA8PBTIRGEs8XGAW59cPcdr6PO8G58aW2C801USx80 bn12HhD8Ig8LZuVDr0WUlBMo4ZMGZQhcxylVMN2ps6plUmrlzfWf6kENU R7TJ9Tq9xVJzRd/GPQ9WkF54E/gyOjpnSFqGUjdVzF+m6RABlZCuTMvaH +WMA03cf8kEMd6Jmfw409kA9XO8d6QEaeU8HKVJMvV0CrzhHI79cm7Ayf 39rTiSkRzP2sFWbAabnV76EfAfnSyQVABHTrhdNrYcvKcPd/OFFPVLqJY oY1qGjjqqTjADuakLkxkBbInSppDOYJBtjd5+PNv/DOHJE/oQphRd6oGo w==; X-IronPort-AV: E=McAfee;i="6500,9779,10606"; a="315736587" X-IronPort-AV: E=Sophos;i="5.97,259,1669104000"; d="scan'208";a="315736587" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Jan 2023 23:45:54 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10606"; a="657775510" X-IronPort-AV: E=Sophos;i="5.97,259,1669104000"; d="scan'208";a="657775510" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga007.jf.intel.com with ESMTP; 30 Jan 2023 23:45:53 -0800 From: Lu Baolu To: Joerg Roedel Cc: kan.liang@linux.intel.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 01/12] iommu/vt-d: Remove include/linux/intel-svm.h Date: Tue, 31 Jan 2023 15:37:29 +0800 Message-Id: <20230131073740.378984-2-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230131073740.378984-1-baolu.lu@linux.intel.com> References: <20230131073740.378984-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org There's no need to have a public header for Intel SVA implementation. The device driver should interact with Intel SVA implementation via the IOMMU generic APIs. Reviewed-by: Kevin Tian Signed-off-by: Lu Baolu Link: https://lore.kernel.org/r/20230109014955.147068-2-baolu.lu@linux.inte= l.com --- include/linux/intel-svm.h | 16 ---------------- drivers/iommu/intel/iommu.h | 5 +++++ drivers/iommu/intel/iommu.c | 1 - drivers/iommu/intel/svm.c | 1 - MAINTAINERS | 1 - 5 files changed, 5 insertions(+), 19 deletions(-) delete mode 100644 include/linux/intel-svm.h diff --git a/include/linux/intel-svm.h b/include/linux/intel-svm.h deleted file mode 100644 index f9a0d44f6fdb..000000000000 --- a/include/linux/intel-svm.h +++ /dev/null @@ -1,16 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-only */ -/* - * Copyright =C2=A9 2015 Intel Corporation. - * - * Authors: David Woodhouse - */ - -#ifndef __INTEL_SVM_H__ -#define __INTEL_SVM_H__ - -/* Page Request Queue depth */ -#define PRQ_ORDER 4 -#define PRQ_RING_MASK ((0x1000 << PRQ_ORDER) - 0x20) -#define PRQ_DEPTH ((0x1000 << PRQ_ORDER) >> 5) - -#endif /* __INTEL_SVM_H__ */ diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h index 06e61e474856..f89f63d7a72a 100644 --- a/drivers/iommu/intel/iommu.h +++ b/drivers/iommu/intel/iommu.h @@ -438,6 +438,11 @@ struct q_inval { int free_cnt; }; =20 +/* Page Request Queue depth */ +#define PRQ_ORDER 4 +#define PRQ_RING_MASK ((0x1000 << PRQ_ORDER) - 0x20) +#define PRQ_DEPTH ((0x1000 << PRQ_ORDER) >> 5) + struct dmar_pci_notify_info; =20 #ifdef CONFIG_IRQ_REMAP diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 59df7e42fd53..317af67b6098 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -16,7 +16,6 @@ #include #include #include -#include #include #include #include diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index c76b66263467..d38a54396c23 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -9,7 +9,6 @@ #include #include #include -#include #include #include #include diff --git a/MAINTAINERS b/MAINTAINERS index f781f936ae35..dbc36fa870d1 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -10458,7 +10458,6 @@ L: iommu@lists.linux.dev S: Supported T: git git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu.git F: drivers/iommu/intel/ -F: include/linux/intel-svm.h =20 INTEL IPU3 CSI-2 CIO2 DRIVER M: Yong Zhi --=20 2.34.1 From nobody Sun Sep 14 04:56:24 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD2F2C38142 for ; Tue, 31 Jan 2023 07:46:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230502AbjAaHqV (ORCPT ); Tue, 31 Jan 2023 02:46:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47774 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230007AbjAaHqM (ORCPT ); Tue, 31 Jan 2023 02:46:12 -0500 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 88BC23A5A8 for ; Mon, 30 Jan 2023 23:46:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675151168; x=1706687168; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=x0YLLs+j2MWKb2XKOlEoDSy+Rw/i9cYLM8wl9qJXrpw=; b=VskNPWj/i1Cv9hhOdSYRkPX06iFxJU3WQygguoKeXQpvTDDPPwwiwe6O JUaFvZWOVSD1RS85YdNonKRkFCDA5fdTdtL5PDsbvOhIitnBviNYjWZK7 71SlIP0JoYevRf2OcWiOOGl5usFCM6bRecWr+xRMPNULG55IJ1/ISr5Lf avXD7emJDMs2DuxVxBgOYKgU7ULB5m3XCiTZbZhXiXY4PqQJA/3Jpd8Cn /VyT2ogH8iw9e3CF7IDVQHAffzp4Et3J4pzZiqC0qpPmUiLgvapuH4jlP ZgxXsOQbUzuoOHQQ1QmcM9LwaJtpmXKC/xOePrJYtUcU2Ww8f/S3o5frR Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10606"; a="315736592" X-IronPort-AV: E=Sophos;i="5.97,259,1669104000"; d="scan'208";a="315736592" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Jan 2023 23:45:55 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10606"; a="657775514" X-IronPort-AV: E=Sophos;i="5.97,259,1669104000"; d="scan'208";a="657775514" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga007.jf.intel.com with ESMTP; 30 Jan 2023 23:45:54 -0800 From: Lu Baolu To: Joerg Roedel Cc: kan.liang@linux.intel.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 02/12] iommu/vt-d: Remove unused fields in svm structures Date: Tue, 31 Jan 2023 15:37:30 +0800 Message-Id: <20230131073740.378984-3-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230131073740.378984-1-baolu.lu@linux.intel.com> References: <20230131073740.378984-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" They aren't used anywhere. Remove them to avoid dead code. Reviewed-by: Kevin Tian Signed-off-by: Lu Baolu Link: https://lore.kernel.org/r/20230109014955.147068-3-baolu.lu@linux.inte= l.com --- drivers/iommu/intel/iommu.h | 4 ---- drivers/iommu/intel/svm.c | 2 -- 2 files changed, 6 deletions(-) diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h index f89f63d7a72a..e7c732979364 100644 --- a/drivers/iommu/intel/iommu.h +++ b/drivers/iommu/intel/iommu.h @@ -762,18 +762,14 @@ struct intel_svm_dev { struct device *dev; struct intel_iommu *iommu; struct iommu_sva sva; - u32 pasid; int users; u16 did; - u16 dev_iotlb:1; u16 sid, qdep; }; =20 struct intel_svm { struct mmu_notifier notifier; struct mm_struct *mm; - - unsigned int flags; u32 pasid; struct list_head devs; }; diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index d38a54396c23..d1e445f03aa6 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -351,11 +351,9 @@ static struct iommu_sva *intel_svm_bind_mm(struct inte= l_iommu *iommu, sdev->did =3D FLPT_DEFAULT_DID; sdev->sid =3D PCI_DEVID(info->bus, info->devfn); sdev->users =3D 1; - sdev->pasid =3D svm->pasid; sdev->sva.dev =3D dev; init_rcu_head(&sdev->rcu); if (info->ats_enabled) { - sdev->dev_iotlb =3D 1; sdev->qdep =3D info->ats_qdep; if (sdev->qdep >=3D QI_DEV_EIOTLB_MAX_INVS) sdev->qdep =3D 0; --=20 2.34.1 From nobody Sun Sep 14 04:56:24 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BEDFEC636CD for ; Tue, 31 Jan 2023 07:46:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230516AbjAaHqX (ORCPT ); Tue, 31 Jan 2023 02:46:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47798 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230390AbjAaHqN (ORCPT ); Tue, 31 Jan 2023 02:46:13 -0500 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0928A2940C for ; Mon, 30 Jan 2023 23:46:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675151171; x=1706687171; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=5j6lLF9lS0vgkkZ16tdr3ohX2z3E1DhM0p9ecFybgUU=; b=DiUZMmEIxI0D8cMXK8tMgzW42Oc6HSEld61oIPMFJYn5eNjQgO9baaVE yLfaIsfomw6WyEHZo8moOl4LNAm4j708M8qAgRTQreH3GDgtBHHMQLO23 YvoXI/AP0WzYGT/qLvZindAEavMtCkpuWdOjtC7xlQKEzrijqlzVQ25AE SnY35FLJQhrIC5LOA+5Uk5pIqJJTvqTRpEdyDwaDN7egFuugFK9JIWJZw bLlAn+MjcVv6oLJBVb1Y3oIQKO/NLdBm634FSmkkpveYjxjXRCzzMMeJA u2HWoa4uGJ0/yrv6yKCje2yrp/HSx4TWaeADl8cf9wbhfUjGluSaBHqDT Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10606"; a="315736598" X-IronPort-AV: E=Sophos;i="5.97,259,1669104000"; d="scan'208";a="315736598" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Jan 2023 23:45:57 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10606"; a="657775517" X-IronPort-AV: E=Sophos;i="5.97,259,1669104000"; d="scan'208";a="657775517" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga007.jf.intel.com with ESMTP; 30 Jan 2023 23:45:56 -0800 From: Lu Baolu To: Joerg Roedel Cc: kan.liang@linux.intel.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 03/12] iommu/vt-d: Remove users from intel_svm_dev Date: Tue, 31 Jan 2023 15:37:31 +0800 Message-Id: <20230131073740.378984-4-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230131073740.378984-1-baolu.lu@linux.intel.com> References: <20230131073740.378984-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" It was used as a reference counter of an existing bond between device and user application memory address. Commit be51b1d6bbff ("iommu/sva: Refactoring iommu_sva_bind/unbind_device()") has added this in iommu core. Remove it to avoid duplicate code. Reviewed-by: Kevin Tian Signed-off-by: Lu Baolu Link: https://lore.kernel.org/r/20230109014955.147068-4-baolu.lu@linux.inte= l.com --- drivers/iommu/intel/iommu.h | 1 - drivers/iommu/intel/svm.c | 62 ++++++++++++++++--------------------- 2 files changed, 27 insertions(+), 36 deletions(-) diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h index e7c732979364..2a1619ff0d79 100644 --- a/drivers/iommu/intel/iommu.h +++ b/drivers/iommu/intel/iommu.h @@ -762,7 +762,6 @@ struct intel_svm_dev { struct device *dev; struct intel_iommu *iommu; struct iommu_sva sva; - int users; u16 did; u16 sid, qdep; }; diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index d1e445f03aa6..c7dc53e40c26 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -333,13 +333,6 @@ static struct iommu_sva *intel_svm_bind_mm(struct inte= l_iommu *iommu, } } =20 - /* Find the matching device in svm list */ - sdev =3D svm_lookup_device_by_dev(svm, dev); - if (sdev) { - sdev->users++; - goto success; - } - sdev =3D kzalloc(sizeof(*sdev), GFP_KERNEL); if (!sdev) { ret =3D -ENOMEM; @@ -350,7 +343,6 @@ static struct iommu_sva *intel_svm_bind_mm(struct intel= _iommu *iommu, sdev->iommu =3D iommu; sdev->did =3D FLPT_DEFAULT_DID; sdev->sid =3D PCI_DEVID(info->bus, info->devfn); - sdev->users =3D 1; sdev->sva.dev =3D dev; init_rcu_head(&sdev->rcu); if (info->ats_enabled) { @@ -367,7 +359,7 @@ static struct iommu_sva *intel_svm_bind_mm(struct intel= _iommu *iommu, goto free_sdev; =20 list_add_rcu(&sdev->list, &svm->devs); -success: + return &sdev->sva; =20 free_sdev: @@ -401,32 +393,32 @@ static int intel_svm_unbind_mm(struct device *dev, u3= 2 pasid) mm =3D svm->mm; =20 if (sdev) { - sdev->users--; - if (!sdev->users) { - list_del_rcu(&sdev->list); - /* Flush the PASID cache and IOTLB for this device. - * Note that we do depend on the hardware *not* using - * the PASID any more. Just as we depend on other - * devices never using PASIDs that they have no right - * to use. We have a *shared* PASID table, because it's - * large and has to be physically contiguous. So it's - * hard to be as defensive as we might like. */ - intel_pasid_tear_down_entry(iommu, dev, - svm->pasid, false); - intel_svm_drain_prq(dev, svm->pasid); - kfree_rcu(sdev, rcu); - - if (list_empty(&svm->devs)) { - if (svm->notifier.ops) - mmu_notifier_unregister(&svm->notifier, mm); - pasid_private_remove(svm->pasid); - /* We mandate that no page faults may be outstanding - * for the PASID when intel_svm_unbind_mm() is called. - * If that is not obeyed, subtle errors will happen. - * Let's make them less subtle... */ - memset(svm, 0x6b, sizeof(*svm)); - kfree(svm); - } + list_del_rcu(&sdev->list); + /* + * Flush the PASID cache and IOTLB for this device. + * Note that we do depend on the hardware *not* using + * the PASID any more. Just as we depend on other + * devices never using PASIDs that they have no right + * to use. We have a *shared* PASID table, because it's + * large and has to be physically contiguous. So it's + * hard to be as defensive as we might like. + */ + intel_pasid_tear_down_entry(iommu, dev, svm->pasid, false); + intel_svm_drain_prq(dev, svm->pasid); + kfree_rcu(sdev, rcu); + + if (list_empty(&svm->devs)) { + if (svm->notifier.ops) + mmu_notifier_unregister(&svm->notifier, mm); + pasid_private_remove(svm->pasid); + /* + * We mandate that no page faults may be outstanding + * for the PASID when intel_svm_unbind_mm() is called. + * If that is not obeyed, subtle errors will happen. + * Let's make them less subtle... + */ + memset(svm, 0x6b, sizeof(*svm)); + kfree(svm); } } out: --=20 2.34.1 From nobody Sun Sep 14 04:56:24 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B5B45C636CC for ; Tue, 31 Jan 2023 07:46:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231130AbjAaHq0 (ORCPT ); Tue, 31 Jan 2023 02:46:26 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47802 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230417AbjAaHqN (ORCPT ); Tue, 31 Jan 2023 02:46:13 -0500 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DBDB43B672 for ; Mon, 30 Jan 2023 23:46:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675151171; x=1706687171; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=c+cQH7MsA9K/XtQhXp9SwsXYOGutwpkj5FcsxhadHr4=; b=FtWJb9+IqBjT7ahJFnwhHoh2IeNWwkv29q5eXl0CVskx3hq6d3Gb8ptj 6SPsJYXqE9L1b6nv1Ha88VjDpDpSjYqOBbO74ZtwVPWn/4X+2f5vMLY5D svmIJMDWeRnMd1CQYPwN4fwjP2TY0Thn4dsX2SVl06Cl+PbGL/poNPWvQ LoXOkrDV8rcfcsUbRqw0pdL5jZ7q2tl4f/IFL6B+gNxqUqATApQ17dFNh QgHhkLShH0+QstQlcObZydA0WJ9cfZTwmF9xs8SAQsFi7hfVjbBBtOgou vfRJE99TdS9ccD6J+W2F87gjS7MRrcMoM1jh8SMTNA0zcBNM5I8Okh5ag A==; X-IronPort-AV: E=McAfee;i="6500,9779,10606"; a="315736603" X-IronPort-AV: E=Sophos;i="5.97,259,1669104000"; d="scan'208";a="315736603" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Jan 2023 23:45:58 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10606"; a="657775521" X-IronPort-AV: E=Sophos;i="5.97,259,1669104000"; d="scan'208";a="657775521" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga007.jf.intel.com with ESMTP; 30 Jan 2023 23:45:57 -0800 From: Lu Baolu To: Joerg Roedel Cc: kan.liang@linux.intel.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 04/12] iommu/vt-d: Remove sva from intel_svm_dev Date: Tue, 31 Jan 2023 15:37:32 +0800 Message-Id: <20230131073740.378984-5-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230131073740.378984-1-baolu.lu@linux.intel.com> References: <20230131073740.378984-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" After commit be51b1d6bbff ("iommu/sva: Refactoring iommu_sva_bind/unbind_device()"), the iommu driver doesn't need to return an iommu_sva pointer anymore. This removes the sva field from intel_svm_dev and cleanups the code accordingly. Reviewed-by: Kevin Tian Signed-off-by: Lu Baolu Link: https://lore.kernel.org/r/20230109014955.147068-5-baolu.lu@linux.inte= l.com --- drivers/iommu/intel/iommu.h | 1 - drivers/iommu/intel/svm.c | 23 +++++++++-------------- 2 files changed, 9 insertions(+), 15 deletions(-) diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h index 2a1619ff0d79..f0222ad38757 100644 --- a/drivers/iommu/intel/iommu.h +++ b/drivers/iommu/intel/iommu.h @@ -761,7 +761,6 @@ struct intel_svm_dev { struct rcu_head rcu; struct device *dev; struct intel_iommu *iommu; - struct iommu_sva sva; u16 did; u16 sid, qdep; }; diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index c7dc53e40c26..e7b9bedebcc0 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -298,9 +298,8 @@ static int pasid_to_svm_sdev(struct device *dev, unsign= ed int pasid, return 0; } =20 -static struct iommu_sva *intel_svm_bind_mm(struct intel_iommu *iommu, - struct device *dev, - struct mm_struct *mm) +static int intel_svm_bind_mm(struct intel_iommu *iommu, struct device *dev, + struct mm_struct *mm) { struct device_domain_info *info =3D dev_iommu_priv_get(dev); struct intel_svm_dev *sdev; @@ -312,7 +311,7 @@ static struct iommu_sva *intel_svm_bind_mm(struct intel= _iommu *iommu, if (!svm) { svm =3D kzalloc(sizeof(*svm), GFP_KERNEL); if (!svm) - return ERR_PTR(-ENOMEM); + return -ENOMEM; =20 svm->pasid =3D mm->pasid; svm->mm =3D mm; @@ -322,14 +321,14 @@ static struct iommu_sva *intel_svm_bind_mm(struct int= el_iommu *iommu, ret =3D mmu_notifier_register(&svm->notifier, mm); if (ret) { kfree(svm); - return ERR_PTR(ret); + return ret; } =20 ret =3D pasid_private_add(svm->pasid, svm); if (ret) { mmu_notifier_unregister(&svm->notifier, mm); kfree(svm); - return ERR_PTR(ret); + return ret; } } =20 @@ -343,7 +342,6 @@ static struct iommu_sva *intel_svm_bind_mm(struct intel= _iommu *iommu, sdev->iommu =3D iommu; sdev->did =3D FLPT_DEFAULT_DID; sdev->sid =3D PCI_DEVID(info->bus, info->devfn); - sdev->sva.dev =3D dev; init_rcu_head(&sdev->rcu); if (info->ats_enabled) { sdev->qdep =3D info->ats_qdep; @@ -360,7 +358,7 @@ static struct iommu_sva *intel_svm_bind_mm(struct intel= _iommu *iommu, =20 list_add_rcu(&sdev->list, &svm->devs); =20 - return &sdev->sva; + return 0; =20 free_sdev: kfree(sdev); @@ -371,7 +369,7 @@ static struct iommu_sva *intel_svm_bind_mm(struct intel= _iommu *iommu, kfree(svm); } =20 - return ERR_PTR(ret); + return ret; } =20 /* Caller must hold pasid_mutex */ @@ -843,13 +841,10 @@ static int intel_svm_set_dev_pasid(struct iommu_domai= n *domain, struct device_domain_info *info =3D dev_iommu_priv_get(dev); struct intel_iommu *iommu =3D info->iommu; struct mm_struct *mm =3D domain->mm; - struct iommu_sva *sva; - int ret =3D 0; + int ret; =20 mutex_lock(&pasid_mutex); - sva =3D intel_svm_bind_mm(iommu, dev, mm); - if (IS_ERR(sva)) - ret =3D PTR_ERR(sva); + ret =3D intel_svm_bind_mm(iommu, dev, mm); mutex_unlock(&pasid_mutex); =20 return ret; --=20 2.34.1 From nobody Sun Sep 14 04:56:24 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B45D5C38142 for ; Tue, 31 Jan 2023 07:46:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231142AbjAaHq2 (ORCPT ); Tue, 31 Jan 2023 02:46:28 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47756 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230166AbjAaHqN (ORCPT ); Tue, 31 Jan 2023 02:46:13 -0500 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C3DCB36442 for ; Mon, 30 Jan 2023 23:46:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675151172; x=1706687172; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=cjBgirNm5pIeDvvzUxGA1RbTCC4lfrnHwE9vlSINv4g=; b=mKobrybW3LsJzzBtVBtKiPL5QYjspRhwIketdrOKGQ5HiV9SeDtOdoaw 3z19EqagywN/bbSuQpknbROfRf7m8qG1/YOy6UEIoUMaJrcnE/fFz4/01 xpdnBeA7YGSsGkAVOTGuFnqtL4EEqVvhQsMvdM1U8DwRThvkcm1o7Ifo7 j4Wbm2vJ4x8FrN7F2ZbMeSniLLKBvXn/yxw/An/JJj/Qw3ubR6IeY6O51 Q1f5h3ELcXN6Bs3hLgM3AqNW0tzy2QsLHNDQXV/yG0cAiYlz2cnHV8zrA t6JukmOWooL6sZpMZ/44Yj5z0mndhx1uMUxvQ9dYTKwEvNrHIOqau3r4+ A==; X-IronPort-AV: E=McAfee;i="6500,9779,10606"; a="315736607" X-IronPort-AV: E=Sophos;i="5.97,259,1669104000"; d="scan'208";a="315736607" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Jan 2023 23:46:00 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10606"; a="657775524" X-IronPort-AV: E=Sophos;i="5.97,259,1669104000"; d="scan'208";a="657775524" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga007.jf.intel.com with ESMTP; 30 Jan 2023 23:45:59 -0800 From: Lu Baolu To: Joerg Roedel Cc: kan.liang@linux.intel.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 05/12] iommu/vt-d: Set No Execute Enable bit in PASID table entry Date: Tue, 31 Jan 2023 15:37:33 +0800 Message-Id: <20230131073740.378984-6-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230131073740.378984-1-baolu.lu@linux.intel.com> References: <20230131073740.378984-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Setup No Execute Enable bit (Bit 133) of a scalable mode PASID entry. This is to allow the use of XD bit of the first level page table. Fixes: ddf09b6d43ec ("iommu/vt-d: Setup pasid entries for iova over first l= evel") Signed-off-by: Ashok Raj Signed-off-by: Lu Baolu Reviewed-by: Kevin Tian Link: https://lore.kernel.org/r/20230126095438.354205-1-baolu.lu@linux.inte= l.com --- drivers/iommu/intel/pasid.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/drivers/iommu/intel/pasid.c b/drivers/iommu/intel/pasid.c index fb3c7020028d..ec964ac7d797 100644 --- a/drivers/iommu/intel/pasid.c +++ b/drivers/iommu/intel/pasid.c @@ -364,6 +364,16 @@ static inline void pasid_set_page_snoop(struct pasid_e= ntry *pe, bool value) pasid_set_bits(&pe->val[1], 1 << 23, value << 23); } =20 +/* + * Setup No Execute Enable bit (Bit 133) of a scalable mode PASID + * entry. It is required when XD bit of the first level page table + * entry is about to be set. + */ +static inline void pasid_set_nxe(struct pasid_entry *pe) +{ + pasid_set_bits(&pe->val[2], 1 << 5, 1 << 5); +} + /* * Setup the Page Snoop (PGSNP) field (Bit 88) of a scalable mode * PASID entry. @@ -557,6 +567,7 @@ int intel_pasid_setup_first_level(struct intel_iommu *i= ommu, pasid_set_domain_id(pte, did); pasid_set_address_width(pte, iommu->agaw); pasid_set_page_snoop(pte, !!ecap_smpwc(iommu->ecap)); + pasid_set_nxe(pte); =20 /* Setup Present and PASID Granular Transfer Type: */ pasid_set_translation_type(pte, PASID_ENTRY_PGTT_FL_ONLY); --=20 2.34.1 From nobody Sun Sep 14 04:56:24 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A0C0BC38142 for ; Tue, 31 Jan 2023 07:46:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231226AbjAaHqf (ORCPT ); Tue, 31 Jan 2023 02:46:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47760 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230347AbjAaHqP (ORCPT ); Tue, 31 Jan 2023 02:46:15 -0500 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3C41F3347B for ; Mon, 30 Jan 2023 23:46:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675151173; x=1706687173; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=5j0AQLikYlRRnQOHzKRdwcTeWDE/SCGwovkljtCjx10=; b=ghLX/2vBlZPRrSuAjrJKYmT+Zpr3+O1ptw2FX2zT/k+OElWkjnE24f0Y NleKm+AUj4j4NgDcFFgJvqfoWaprFxE0Y2jsuwqnlE5m3T/SoufIt0zxK jNlY4WFQ107vTwAvRefxbDtncE2SuS0WbLAdsj6Ja4cGToK6Q1GErIzol Apx+vTyLXG+/y6TED4/XOpMLxjyN9jSFVK6qrDtCiQW3bwkJ39ZeaQyIH BliIQ3AAS46U3rJQ5gAlcTP9JD+si3XpbzG410Elw8wVY8XMRKm2UogQM Y2UZxCKP3o8H+aPAZtq5tgq/vpASf2qC75GOICWvdxCG0J3LiWdpw5kYp Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10606"; a="315736611" X-IronPort-AV: E=Sophos;i="5.97,259,1669104000"; d="scan'208";a="315736611" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Jan 2023 23:46:01 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10606"; a="657775532" X-IronPort-AV: E=Sophos;i="5.97,259,1669104000"; d="scan'208";a="657775532" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga007.jf.intel.com with ESMTP; 30 Jan 2023 23:46:00 -0800 From: Lu Baolu To: Joerg Roedel Cc: kan.liang@linux.intel.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 06/12] iommu/vt-d: Support size of the register set in DRHD Date: Tue, 31 Jan 2023 15:37:34 +0800 Message-Id: <20230131073740.378984-7-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230131073740.378984-1-baolu.lu@linux.intel.com> References: <20230131073740.378984-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Kan Liang A new field, which indicates the size of the remapping hardware register set for this remapping unit, is introduced in the DMA-remapping hardware unit definition (DRHD) structure with the VT-d Spec 4.0. With this information, SW doesn't need to 'guess' the size of the register set anymore. Update the struct acpi_dmar_hardware_unit to reflect the field. Store the size of the register set in struct dmar_drhd_unit for each dmar device. The 'size' information is ResvZ for the old BIOS and platforms. Fall back to the old guessing method. There is nothing changed. Signed-off-by: Kan Liang Link: https://lore.kernel.org/r/20230128200428.1459118-2-kan.liang@linux.in= tel.com Signed-off-by: Lu Baolu --- include/linux/dmar.h | 1 + include/acpi/actbl1.h | 2 +- drivers/iommu/intel/dmar.c | 11 +++++++---- 3 files changed, 9 insertions(+), 5 deletions(-) diff --git a/include/linux/dmar.h b/include/linux/dmar.h index d81a51978d01..725d5e6acec0 100644 --- a/include/linux/dmar.h +++ b/include/linux/dmar.h @@ -39,6 +39,7 @@ struct dmar_drhd_unit { struct list_head list; /* list of drhd units */ struct acpi_dmar_header *hdr; /* ACPI header */ u64 reg_base_addr; /* register base address*/ + unsigned long reg_size; /* size of register set */ struct dmar_dev_scope *devices;/* target device array */ int devices_cnt; /* target device count */ u16 segment; /* PCI domain */ diff --git a/include/acpi/actbl1.h b/include/acpi/actbl1.h index 4175dce3967c..bdded0ac46eb 100644 --- a/include/acpi/actbl1.h +++ b/include/acpi/actbl1.h @@ -802,7 +802,7 @@ struct acpi_dmar_pci_path { struct acpi_dmar_hardware_unit { struct acpi_dmar_header header; u8 flags; - u8 reserved; + u8 size; /* Size of the register set */ u16 segment; u64 address; /* Register Base Address */ }; diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c index b00a0ceb2d13..3a40fef1ec1b 100644 --- a/drivers/iommu/intel/dmar.c +++ b/drivers/iommu/intel/dmar.c @@ -427,6 +427,8 @@ static int dmar_parse_one_drhd(struct acpi_dmar_header = *header, void *arg) memcpy(dmaru->hdr, header, header->length); dmaru->reg_base_addr =3D drhd->address; dmaru->segment =3D drhd->segment; + /* The size of the register set is 2 ^ N 4 KB pages. */ + dmaru->reg_size =3D 1UL << (drhd->size + 12); dmaru->include_all =3D drhd->flags & 0x1; /* BIT0: INCLUDE_ALL */ dmaru->devices =3D dmar_alloc_dev_scope((void *)(drhd + 1), ((void *)drhd) + drhd->header.length, @@ -956,17 +958,18 @@ static void unmap_iommu(struct intel_iommu *iommu) /** * map_iommu: map the iommu's registers * @iommu: the iommu to map - * @phys_addr: the physical address of the base resgister + * @drhd: DMA remapping hardware definition structure * * Memory map the iommu's registers. Start w/ a single page, and * possibly expand if that turns out to be insufficent. */ -static int map_iommu(struct intel_iommu *iommu, u64 phys_addr) +static int map_iommu(struct intel_iommu *iommu, struct dmar_drhd_unit *drh= d) { + u64 phys_addr =3D drhd->reg_base_addr; int map_size, err=3D0; =20 iommu->reg_phys =3D phys_addr; - iommu->reg_size =3D VTD_PAGE_SIZE; + iommu->reg_size =3D drhd->reg_size; =20 if (!request_mem_region(iommu->reg_phys, iommu->reg_size, iommu->name)) { pr_err("Can't reserve memory\n"); @@ -1050,7 +1053,7 @@ static int alloc_iommu(struct dmar_drhd_unit *drhd) } sprintf(iommu->name, "dmar%d", iommu->seq_id); =20 - err =3D map_iommu(iommu, drhd->reg_base_addr); + err =3D map_iommu(iommu, drhd); if (err) { pr_err("Failed to map %s\n", iommu->name); goto error_free_seq_id; --=20 2.34.1 From nobody Sun Sep 14 04:56:24 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 58F53C636CC for ; Tue, 31 Jan 2023 07:46:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230158AbjAaHqc (ORCPT ); Tue, 31 Jan 2023 02:46:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47838 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230372AbjAaHqP (ORCPT ); Tue, 31 Jan 2023 02:46:15 -0500 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3C5733929E for ; Mon, 30 Jan 2023 23:46:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675151173; x=1706687173; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=01VhhsN0RQWVHnhfMq91WrceX8fR/9LRlSfIH041cpU=; b=BF/bsw7800rNLTxXKxuzsjJxurNPK1Eu3ynQ3tv5jn+QLUMRtBBV9uSD a3hLI7yZymyuQCsOwf0+J7UqhujvNxP7AnARWSRCKTTqjf4cBzLayP3nN PmJrO3EmN+iptiyebRIXLWo7TcKEQxlHpb9b+hzKZhpW13El/yNrGydnk bqr5V4trcbJMWWNxKYixO1ii0xeRgVbm+E2ou+fo52IafpFgErGWvMHDJ BA0w1GQjbXnQXwXgmR5R5yY1EKRpADnhi5DfRtDGX8z4+RKm2BU3mPZO+ 1HRM5HWkA17GRjpvqrmGaAR3bXTlppvCjqnmfrLSmHt4VPouHCf4Co4/x A==; X-IronPort-AV: E=McAfee;i="6500,9779,10606"; a="315736618" X-IronPort-AV: E=Sophos;i="5.97,259,1669104000"; d="scan'208";a="315736618" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Jan 2023 23:46:03 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10606"; a="657775553" X-IronPort-AV: E=Sophos;i="5.97,259,1669104000"; d="scan'208";a="657775553" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga007.jf.intel.com with ESMTP; 30 Jan 2023 23:46:01 -0800 From: Lu Baolu To: Joerg Roedel Cc: kan.liang@linux.intel.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 07/12] iommu/vt-d: Retrieve IOMMU perfmon capability information Date: Tue, 31 Jan 2023 15:37:35 +0800 Message-Id: <20230131073740.378984-8-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230131073740.378984-1-baolu.lu@linux.intel.com> References: <20230131073740.378984-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Kan Liang The performance monitoring infrastructure, perfmon, is to support collection of information about key events occurring during operation of the remapping hardware, to aid performance tuning and debug. Each remapping hardware unit has capability registers that indicate support for performance monitoring features and enumerate the capabilities. Add alloc_iommu_pmu() to retrieve IOMMU perfmon capability information for each iommu unit. The information is stored in the iommu->pmu data structure. Capability registers are read-only, so it's safe to prefetch and store them in the pmu structure. This could avoid unnecessary VMEXIT when this code is running in the virtualization environment. Add free_iommu_pmu() to free the saved capability information when freeing the iommu unit. Add a kernel config option for the IOMMU perfmon feature. Unless a user explicitly uses the perf tool to monitor the IOMMU perfmon event, there isn't any impact for the existing IOMMU. Enable it by default. Signed-off-by: Kan Liang Link: https://lore.kernel.org/r/20230128200428.1459118-3-kan.liang@linux.in= tel.com Signed-off-by: Lu Baolu --- drivers/iommu/intel/iommu.h | 43 ++++++++- drivers/iommu/intel/perfmon.h | 40 ++++++++ drivers/iommu/intel/dmar.c | 7 ++ drivers/iommu/intel/perfmon.c | 172 ++++++++++++++++++++++++++++++++++ drivers/iommu/intel/Kconfig | 11 +++ drivers/iommu/intel/Makefile | 1 + 6 files changed, 273 insertions(+), 1 deletion(-) create mode 100644 drivers/iommu/intel/perfmon.h create mode 100644 drivers/iommu/intel/perfmon.c diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h index f0222ad38757..f03d4b6bf49a 100644 --- a/drivers/iommu/intel/iommu.h +++ b/drivers/iommu/intel/iommu.h @@ -125,6 +125,11 @@ #define DMAR_MTRR_PHYSMASK8_REG 0x208 #define DMAR_MTRR_PHYSBASE9_REG 0x210 #define DMAR_MTRR_PHYSMASK9_REG 0x218 +#define DMAR_PERFCAP_REG 0x300 +#define DMAR_PERFCFGOFF_REG 0x310 +#define DMAR_PERFOVFOFF_REG 0x318 +#define DMAR_PERFCNTROFF_REG 0x31c +#define DMAR_PERFEVNTCAP_REG 0x380 #define DMAR_VCCAP_REG 0xe30 /* Virtual command capability register */ #define DMAR_VCMD_REG 0xe00 /* Virtual command register */ #define DMAR_VCRSP_REG 0xe10 /* Virtual command response register */ @@ -148,6 +153,7 @@ */ #define cap_esrtps(c) (((c) >> 63) & 1) #define cap_esirtps(c) (((c) >> 62) & 1) +#define cap_ecmds(c) (((c) >> 61) & 1) #define cap_fl5lp_support(c) (((c) >> 60) & 1) #define cap_pi_support(c) (((c) >> 59) & 1) #define cap_fl1gp_support(c) (((c) >> 56) & 1) @@ -179,7 +185,8 @@ * Extended Capability Register */ =20 -#define ecap_rps(e) (((e) >> 49) & 0x1) +#define ecap_pms(e) (((e) >> 51) & 0x1) +#define ecap_rps(e) (((e) >> 49) & 0x1) #define ecap_smpwc(e) (((e) >> 48) & 0x1) #define ecap_flts(e) (((e) >> 47) & 0x1) #define ecap_slts(e) (((e) >> 46) & 0x1) @@ -210,6 +217,22 @@ #define ecap_max_handle_mask(e) (((e) >> 20) & 0xf) #define ecap_sc_support(e) (((e) >> 7) & 0x1) /* Snooping Control */ =20 +/* + * Decoding Perf Capability Register + */ +#define pcap_num_cntr(p) ((p) & 0xffff) +#define pcap_cntr_width(p) (((p) >> 16) & 0x7f) +#define pcap_num_event_group(p) (((p) >> 24) & 0x1f) +#define pcap_filters_mask(p) (((p) >> 32) & 0x1f) +#define pcap_interrupt(p) (((p) >> 50) & 0x1) +/* The counter stride is calculated as 2 ^ (x+10) bytes */ +#define pcap_cntr_stride(p) (1ULL << ((((p) >> 52) & 0x7) + 10)) + +/* + * Decoding Perf Event Capability Register + */ +#define pecap_es(p) ((p) & 0xfffffff) + /* Virtual command interface capability */ #define vccap_pasid(v) (((v) & DMA_VCS_PAS)) /* PASID allocation */ =20 @@ -559,6 +582,22 @@ struct dmar_domain { iommu core */ }; =20 +struct iommu_pmu { + struct intel_iommu *iommu; + u32 num_cntr; /* Number of counters */ + u32 num_eg; /* Number of event group */ + u32 cntr_width; /* Counter width */ + u32 cntr_stride; /* Counter Stride */ + u32 filter; /* Bitmask of filter support */ + void __iomem *base; /* the PerfMon base address */ + void __iomem *cfg_reg; /* counter configuration base address */ + void __iomem *cntr_reg; /* counter 0 address*/ + void __iomem *overflow; /* overflow status register */ + + u64 *evcap; /* Indicates all supported events */ + u32 **cntr_evcap; /* Supported events of each counter. */ +}; + struct intel_iommu { void __iomem *reg; /* Pointer to hardware regs, virtual addr */ u64 reg_phys; /* physical address of hw register set */ @@ -605,6 +644,8 @@ struct intel_iommu { =20 struct dmar_drhd_unit *drhd; void *perf_statistic; + + struct iommu_pmu *pmu; }; =20 /* PCI domain-device relationship */ diff --git a/drivers/iommu/intel/perfmon.h b/drivers/iommu/intel/perfmon.h new file mode 100644 index 000000000000..4b0d9c1fea6f --- /dev/null +++ b/drivers/iommu/intel/perfmon.h @@ -0,0 +1,40 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +/* + * PERFCFGOFF_REG, PERFFRZOFF_REG + * PERFOVFOFF_REG, PERFCNTROFF_REG + */ +#define IOMMU_PMU_NUM_OFF_REGS 4 +#define IOMMU_PMU_OFF_REGS_STEP 4 + +#define IOMMU_PMU_CFG_OFFSET 0x100 +#define IOMMU_PMU_CFG_CNTRCAP_OFFSET 0x80 +#define IOMMU_PMU_CFG_CNTREVCAP_OFFSET 0x84 +#define IOMMU_PMU_CFG_SIZE 0x8 +#define IOMMU_PMU_CFG_FILTERS_OFFSET 0x4 + +#define IOMMU_PMU_CAP_REGS_STEP 8 + +#define iommu_cntrcap_pcc(p) ((p) & 0x1) +#define iommu_cntrcap_cw(p) (((p) >> 8) & 0xff) +#define iommu_cntrcap_ios(p) (((p) >> 16) & 0x1) +#define iommu_cntrcap_egcnt(p) (((p) >> 28) & 0xf) + +#define iommu_event_select(p) ((p) & 0xfffffff) +#define iommu_event_group(p) (((p) >> 28) & 0xf) + +#ifdef CONFIG_INTEL_IOMMU_PERF_EVENTS +int alloc_iommu_pmu(struct intel_iommu *iommu); +void free_iommu_pmu(struct intel_iommu *iommu); +#else +static inline int +alloc_iommu_pmu(struct intel_iommu *iommu) +{ + return 0; +} + +static inline void +free_iommu_pmu(struct intel_iommu *iommu) +{ +} +#endif /* CONFIG_INTEL_IOMMU_PERF_EVENTS */ diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c index 3a40fef1ec1b..7a03cadb13ff 100644 --- a/drivers/iommu/intel/dmar.c +++ b/drivers/iommu/intel/dmar.c @@ -34,6 +34,7 @@ #include "../irq_remapping.h" #include "perf.h" #include "trace.h" +#include "perfmon.h" =20 typedef int (*dmar_res_handler_t)(struct acpi_dmar_header *, void *); struct dmar_res_callback { @@ -1106,6 +1107,9 @@ static int alloc_iommu(struct dmar_drhd_unit *drhd) if (sts & DMA_GSTS_QIES) iommu->gcmd |=3D DMA_GCMD_QIE; =20 + if (alloc_iommu_pmu(iommu)) + pr_debug("Cannot alloc PMU for iommu (seq_id =3D %d)\n", iommu->seq_id); + raw_spin_lock_init(&iommu->register_lock); =20 /* @@ -1140,6 +1144,7 @@ static int alloc_iommu(struct dmar_drhd_unit *drhd) err_sysfs: iommu_device_sysfs_remove(&iommu->iommu); err_unmap: + free_iommu_pmu(iommu); unmap_iommu(iommu); error_free_seq_id: ida_free(&dmar_seq_ids, iommu->seq_id); @@ -1155,6 +1160,8 @@ static void free_iommu(struct intel_iommu *iommu) iommu_device_sysfs_remove(&iommu->iommu); } =20 + free_iommu_pmu(iommu); + if (iommu->irq) { if (iommu->pr_irq) { free_irq(iommu->pr_irq, iommu); diff --git a/drivers/iommu/intel/perfmon.c b/drivers/iommu/intel/perfmon.c new file mode 100644 index 000000000000..db5791a54455 --- /dev/null +++ b/drivers/iommu/intel/perfmon.c @@ -0,0 +1,172 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Support Intel IOMMU PerfMon + * Copyright(c) 2023 Intel Corporation. + */ +#define pr_fmt(fmt) "DMAR: " fmt +#define dev_fmt(fmt) pr_fmt(fmt) + +#include +#include "iommu.h" +#include "perfmon.h" + +static inline void __iomem * +get_perf_reg_address(struct intel_iommu *iommu, u32 offset) +{ + u32 off =3D dmar_readl(iommu->reg + offset); + + return iommu->reg + off; +} + +int alloc_iommu_pmu(struct intel_iommu *iommu) +{ + struct iommu_pmu *iommu_pmu; + int i, j, ret; + u64 perfcap; + u32 cap; + + if (!ecap_pms(iommu->ecap)) + return 0; + + /* The IOMMU PMU requires the ECMD support as well */ + if (!cap_ecmds(iommu->cap)) + return -ENODEV; + + perfcap =3D dmar_readq(iommu->reg + DMAR_PERFCAP_REG); + /* The performance monitoring is not supported. */ + if (!perfcap) + return -ENODEV; + + /* Sanity check for the number of the counters and event groups */ + if (!pcap_num_cntr(perfcap) || !pcap_num_event_group(perfcap)) + return -ENODEV; + + /* The interrupt on overflow is required */ + if (!pcap_interrupt(perfcap)) + return -ENODEV; + + iommu_pmu =3D kzalloc(sizeof(*iommu_pmu), GFP_KERNEL); + if (!iommu_pmu) + return -ENOMEM; + + iommu_pmu->num_cntr =3D pcap_num_cntr(perfcap); + iommu_pmu->cntr_width =3D pcap_cntr_width(perfcap); + iommu_pmu->filter =3D pcap_filters_mask(perfcap); + iommu_pmu->cntr_stride =3D pcap_cntr_stride(perfcap); + iommu_pmu->num_eg =3D pcap_num_event_group(perfcap); + + iommu_pmu->evcap =3D kcalloc(iommu_pmu->num_eg, sizeof(u64), GFP_KERNEL); + if (!iommu_pmu->evcap) { + ret =3D -ENOMEM; + goto free_pmu; + } + + /* Parse event group capabilities */ + for (i =3D 0; i < iommu_pmu->num_eg; i++) { + u64 pcap; + + pcap =3D dmar_readq(iommu->reg + DMAR_PERFEVNTCAP_REG + + i * IOMMU_PMU_CAP_REGS_STEP); + iommu_pmu->evcap[i] =3D pecap_es(pcap); + } + + iommu_pmu->cntr_evcap =3D kcalloc(iommu_pmu->num_cntr, sizeof(u32 *), GFP= _KERNEL); + if (!iommu_pmu->cntr_evcap) { + ret =3D -ENOMEM; + goto free_pmu_evcap; + } + for (i =3D 0; i < iommu_pmu->num_cntr; i++) { + iommu_pmu->cntr_evcap[i] =3D kcalloc(iommu_pmu->num_eg, sizeof(u32), GFP= _KERNEL); + if (!iommu_pmu->cntr_evcap[i]) { + ret =3D -ENOMEM; + goto free_pmu_cntr_evcap; + } + /* + * Set to the global capabilities, will adjust according + * to per-counter capabilities later. + */ + for (j =3D 0; j < iommu_pmu->num_eg; j++) + iommu_pmu->cntr_evcap[i][j] =3D (u32)iommu_pmu->evcap[j]; + } + + iommu_pmu->cfg_reg =3D get_perf_reg_address(iommu, DMAR_PERFCFGOFF_REG); + iommu_pmu->cntr_reg =3D get_perf_reg_address(iommu, DMAR_PERFCNTROFF_REG); + iommu_pmu->overflow =3D get_perf_reg_address(iommu, DMAR_PERFOVFOFF_REG); + + /* + * Check per-counter capabilities. All counters should have the + * same capabilities on Interrupt on Overflow Support and Counter + * Width. + */ + for (i =3D 0; i < iommu_pmu->num_cntr; i++) { + cap =3D dmar_readl(iommu_pmu->cfg_reg + + i * IOMMU_PMU_CFG_OFFSET + + IOMMU_PMU_CFG_CNTRCAP_OFFSET); + if (!iommu_cntrcap_pcc(cap)) + continue; + + /* + * It's possible that some counters have a different + * capability because of e.g., HW bug. Check the corner + * case here and simply drop those counters. + */ + if ((iommu_cntrcap_cw(cap) !=3D iommu_pmu->cntr_width) || + !iommu_cntrcap_ios(cap)) { + iommu_pmu->num_cntr =3D i; + pr_warn("PMU counter capability inconsistent, counter number reduced to= %d\n", + iommu_pmu->num_cntr); + } + + /* Clear the pre-defined events group */ + for (j =3D 0; j < iommu_pmu->num_eg; j++) + iommu_pmu->cntr_evcap[i][j] =3D 0; + + /* Override with per-counter event capabilities */ + for (j =3D 0; j < iommu_cntrcap_egcnt(cap); j++) { + cap =3D dmar_readl(iommu_pmu->cfg_reg + i * IOMMU_PMU_CFG_OFFSET + + IOMMU_PMU_CFG_CNTREVCAP_OFFSET + + (j * IOMMU_PMU_OFF_REGS_STEP)); + iommu_pmu->cntr_evcap[i][iommu_event_group(cap)] =3D iommu_event_select= (cap); + /* + * Some events may only be supported by a specific counter. + * Track them in the evcap as well. + */ + iommu_pmu->evcap[iommu_event_group(cap)] |=3D iommu_event_select(cap); + } + } + + iommu_pmu->iommu =3D iommu; + iommu->pmu =3D iommu_pmu; + + return 0; + +free_pmu_cntr_evcap: + for (i =3D 0; i < iommu_pmu->num_cntr; i++) + kfree(iommu_pmu->cntr_evcap[i]); + kfree(iommu_pmu->cntr_evcap); +free_pmu_evcap: + kfree(iommu_pmu->evcap); +free_pmu: + kfree(iommu_pmu); + + return ret; +} + +void free_iommu_pmu(struct intel_iommu *iommu) +{ + struct iommu_pmu *iommu_pmu =3D iommu->pmu; + + if (!iommu_pmu) + return; + + if (iommu_pmu->evcap) { + int i; + + for (i =3D 0; i < iommu_pmu->num_cntr; i++) + kfree(iommu_pmu->cntr_evcap[i]); + kfree(iommu_pmu->cntr_evcap); + } + kfree(iommu_pmu->evcap); + kfree(iommu_pmu); + iommu->pmu =3D NULL; +} diff --git a/drivers/iommu/intel/Kconfig b/drivers/iommu/intel/Kconfig index b7dff5092fd2..12e1e90fdae1 100644 --- a/drivers/iommu/intel/Kconfig +++ b/drivers/iommu/intel/Kconfig @@ -96,4 +96,15 @@ config INTEL_IOMMU_SCALABLE_MODE_DEFAULT_ON passing intel_iommu=3Dsm_on to the kernel. If not sure, please use the default value. =20 +config INTEL_IOMMU_PERF_EVENTS + def_bool y + bool "Intel IOMMU performance events" + depends on INTEL_IOMMU && PERF_EVENTS + help + Selecting this option will enable the performance monitoring + infrastructure in the Intel IOMMU. It collects information about + key events occurring during operation of the remapping hardware, + to aid performance tuning and debug. These are available on modern + processors which support Intel VT-d 4.0 and later. + endif # INTEL_IOMMU diff --git a/drivers/iommu/intel/Makefile b/drivers/iommu/intel/Makefile index fa0dae16441c..7af3b8a4f2a0 100644 --- a/drivers/iommu/intel/Makefile +++ b/drivers/iommu/intel/Makefile @@ -6,3 +6,4 @@ obj-$(CONFIG_DMAR_PERF) +=3D perf.o obj-$(CONFIG_INTEL_IOMMU_DEBUGFS) +=3D debugfs.o obj-$(CONFIG_INTEL_IOMMU_SVM) +=3D svm.o obj-$(CONFIG_IRQ_REMAP) +=3D irq_remapping.o +obj-$(CONFIG_INTEL_IOMMU_PERF_EVENTS) +=3D perfmon.o --=20 2.34.1 From nobody Sun Sep 14 04:56:24 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E4D3CC636CC for ; Tue, 31 Jan 2023 07:46:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231160AbjAaHqa (ORCPT ); Tue, 31 Jan 2023 02:46:30 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47840 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230315AbjAaHqP (ORCPT ); Tue, 31 Jan 2023 02:46:15 -0500 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A98BD3A5A8 for ; Mon, 30 Jan 2023 23:46:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675151173; x=1706687173; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=/Zucbg5amTcTa+z96MaiZy75/A5VqNpeiaN9HZkfJek=; b=JseWOrmB7xUf/0CLOqEe6K11K4FYLnvyIIje58+RFjlmHV1Pmt6ivnYe gx1x3Q8pCXtAw7bUbfxndfINTI6sSjVVIUUY0x7qVeHNjl1LS07wIPwBD mknyMETboXaDy4F4229x6CNHUZt3YOTY1+cj+kVpra3XBos4XvrjjoG18 rS0rAEb07GoLov4t2JhNS5WV2EyWRYiA0qK4+1B9orAmsIn0XMfagPtxM Qp9tkJImXxRfOvMzKcEE+jKzFTOWX7VeYjX3pVS0VlUS3RcRLMgj249S4 0FDlAW4zEt7gwL2oAUkNchRWw07PfmodDWezLR7uOheNbGhO4zic+SQ4f A==; X-IronPort-AV: E=McAfee;i="6500,9779,10606"; a="315736623" X-IronPort-AV: E=Sophos;i="5.97,259,1669104000"; d="scan'208";a="315736623" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Jan 2023 23:46:05 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10606"; a="657775564" X-IronPort-AV: E=Sophos;i="5.97,259,1669104000"; d="scan'208";a="657775564" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga007.jf.intel.com with ESMTP; 30 Jan 2023 23:46:03 -0800 From: Lu Baolu To: Joerg Roedel Cc: kan.liang@linux.intel.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 08/12] iommu/vt-d: Support Enhanced Command Interface Date: Tue, 31 Jan 2023 15:37:36 +0800 Message-Id: <20230131073740.378984-9-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230131073740.378984-1-baolu.lu@linux.intel.com> References: <20230131073740.378984-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Kan Liang The Enhanced Command Register is to submit command and operand of enhanced commands to DMA Remapping hardware. It can supports up to 256 enhanced commands. There is a HW register to indicate the availability of all 256 enhanced commands. Each bit stands for each command. But there isn't an existing interface to read/write all 256 bits. Introduce the u64 ecmdcap[4] to store the existence of each enhanced command. Read 4 times to get all of them in map_iommu(). Add a helper to facilitate an enhanced command launch. Make sure hardware complete the command. Also add a helper to facilitate the check of PMU essentials. These helpers will be used later. Signed-off-by: Kan Liang Link: https://lore.kernel.org/r/20230128200428.1459118-4-kan.liang@linux.in= tel.com Signed-off-by: Lu Baolu --- drivers/iommu/intel/iommu.h | 33 ++++++++++++++++++++++ drivers/iommu/intel/dmar.c | 10 +++++++ drivers/iommu/intel/iommu.c | 56 +++++++++++++++++++++++++++++++++++++ 3 files changed, 99 insertions(+) diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h index f03d4b6bf49a..f918e83bf91c 100644 --- a/drivers/iommu/intel/iommu.h +++ b/drivers/iommu/intel/iommu.h @@ -130,6 +130,10 @@ #define DMAR_PERFOVFOFF_REG 0x318 #define DMAR_PERFCNTROFF_REG 0x31c #define DMAR_PERFEVNTCAP_REG 0x380 +#define DMAR_ECMD_REG 0x400 +#define DMAR_ECEO_REG 0x408 +#define DMAR_ECRSP_REG 0x410 +#define DMAR_ECCAP_REG 0x430 #define DMAR_VCCAP_REG 0xe30 /* Virtual command capability register */ #define DMAR_VCMD_REG 0xe00 /* Virtual command register */ #define DMAR_VCRSP_REG 0xe10 /* Virtual command response register */ @@ -304,6 +308,26 @@ #define DMA_CCMD_SID(s) (((u64)((s) & 0xffff)) << 16) #define DMA_CCMD_DID(d) ((u64)((d) & 0xffff)) =20 +/* ECMD_REG */ +#define DMA_MAX_NUM_ECMD 256 +#define DMA_MAX_NUM_ECMDCAP (DMA_MAX_NUM_ECMD / 64) +#define DMA_ECMD_REG_STEP 8 +#define DMA_ECMD_ENABLE 0xf0 +#define DMA_ECMD_DISABLE 0xf1 +#define DMA_ECMD_FREEZE 0xf4 +#define DMA_ECMD_UNFREEZE 0xf5 +#define DMA_ECMD_OA_SHIFT 16 +#define DMA_ECMD_ECRSP_IP 0x1 +#define DMA_ECMD_ECCAP3 3 +#define DMA_ECMD_ECCAP3_ECNTS BIT_ULL(48) +#define DMA_ECMD_ECCAP3_DCNTS BIT_ULL(49) +#define DMA_ECMD_ECCAP3_FCNTS BIT_ULL(52) +#define DMA_ECMD_ECCAP3_UFCNTS BIT_ULL(53) +#define DMA_ECMD_ECCAP3_ESSENTIAL (DMA_ECMD_ECCAP3_ECNTS | \ + DMA_ECMD_ECCAP3_DCNTS | \ + DMA_ECMD_ECCAP3_FCNTS | \ + DMA_ECMD_ECCAP3_UFCNTS) + /* FECTL_REG */ #define DMA_FECTL_IM (((u32)1) << 31) =20 @@ -605,6 +629,7 @@ struct intel_iommu { u64 cap; u64 ecap; u64 vccap; + u64 ecmdcap[DMA_MAX_NUM_ECMDCAP]; u32 gcmd; /* Holds TE, EAFL. Don't need SRTP, SFL, WBF */ raw_spinlock_t register_lock; /* protect register handling */ int seq_id; /* sequence id of the iommu */ @@ -840,6 +865,14 @@ extern const struct iommu_ops intel_iommu_ops; extern int intel_iommu_sm; extern int iommu_calculate_agaw(struct intel_iommu *iommu); extern int iommu_calculate_max_sagaw(struct intel_iommu *iommu); +int ecmd_submit_sync(struct intel_iommu *iommu, u8 ecmd, u64 oa, u64 ob); + +static inline bool ecmd_has_pmu_essential(struct intel_iommu *iommu) +{ + return (iommu->ecmdcap[DMA_ECMD_ECCAP3] & DMA_ECMD_ECCAP3_ESSENTIAL) =3D= =3D + DMA_ECMD_ECCAP3_ESSENTIAL; +} + extern int dmar_disabled; extern int intel_iommu_enabled; #else diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c index 7a03cadb13ff..0e429bab436f 100644 --- a/drivers/iommu/intel/dmar.c +++ b/drivers/iommu/intel/dmar.c @@ -1017,6 +1017,16 @@ static int map_iommu(struct intel_iommu *iommu, stru= ct dmar_drhd_unit *drhd) goto release; } } + + if (cap_ecmds(iommu->cap)) { + int i; + + for (i =3D 0; i < DMA_MAX_NUM_ECMDCAP; i++) { + iommu->ecmdcap[i] =3D dmar_readq(iommu->reg + DMAR_ECCAP_REG + + i * DMA_ECMD_REG_STEP); + } + } + err =3D 0; goto out; =20 diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 317af67b6098..e314c30d371a 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -5022,3 +5022,59 @@ void quirk_extra_dev_tlb_flush(struct device_domain_= info *info, pasid, qdep, address, mask); } } + +#define ecmd_get_status_code(res) (((res) & 0xff) >> 1) + +/* + * Function to submit a command to the enhanced command interface. The + * valid enhanced command descriptions are defined in Table 47 of the + * VT-d spec. The VT-d hardware implementation may support some but not + * all commands, which can be determined by checking the Enhanced + * Command Capability Register. + * + * Return values: + * - 0: Command successful without any error; + * - Negative: software error value; + * - Nonzero positive: failure status code defined in Table 48. + */ +int ecmd_submit_sync(struct intel_iommu *iommu, u8 ecmd, u64 oa, u64 ob) +{ + unsigned long flags; + u64 res; + int ret; + + if (!cap_ecmds(iommu->cap)) + return -ENODEV; + + raw_spin_lock_irqsave(&iommu->register_lock, flags); + + res =3D dmar_readq(iommu->reg + DMAR_ECRSP_REG); + if (res & DMA_ECMD_ECRSP_IP) { + ret =3D -EBUSY; + goto err; + } + + /* + * Unconditionally write the operand B, because + * - There is no side effect if an ecmd doesn't require an + * operand B, but we set the register to some value. + * - It's not invoked in any critical path. The extra MMIO + * write doesn't bring any performance concerns. + */ + dmar_writeq(iommu->reg + DMAR_ECEO_REG, ob); + dmar_writeq(iommu->reg + DMAR_ECMD_REG, ecmd | (oa << DMA_ECMD_OA_SHIFT)); + + IOMMU_WAIT_OP(iommu, DMAR_ECRSP_REG, dmar_readq, + !(res & DMA_ECMD_ECRSP_IP), res); + + if (res & DMA_ECMD_ECRSP_IP) { + ret =3D -ETIMEDOUT; + goto err; + } + + ret =3D ecmd_get_status_code(res); +err: + raw_spin_unlock_irqrestore(&iommu->register_lock, flags); + + return ret; +} --=20 2.34.1 From nobody Sun Sep 14 04:56:24 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D80F8C38142 for ; Tue, 31 Jan 2023 07:47:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230354AbjAaHq6 (ORCPT ); Tue, 31 Jan 2023 02:46:58 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47926 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230481AbjAaHqS (ORCPT ); Tue, 31 Jan 2023 02:46:18 -0500 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E41263E0B5 for ; Mon, 30 Jan 2023 23:46:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675151174; x=1706687174; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=J/HSTLQnLsoL2VEVd+gTOR3kh4icAF2RH5nBM/NlT4E=; b=V7J8puf1tUcHIGn2/NvXJgnQCIEEfgYJaYZKipyVAs0PHp7PNUNL7oCd 0S5JRHihVB5TBvE6kaqdN6S/u7amVj6Oa+26rHcbsJfD2Sr3YYoriFxK3 legEowz/+l4pZF1a46FCAEh3hnt0G8S2ATfthzYQL7SVtdFLb5K//Y4Yw G4OlkHG/jzsWOPoHR0rygdfcIGjtO/P9n9dZYRgAqaCCIhC3HyEiWFPsM mO1vOTLpA+ae4I4pfR5C3F5J4cQ2L3XROrN5rxoFNJd17SgcP2GG6RU/V QQy6syirHKVfIh3oQr9gNeQr/3nHR9noV/AMZ5PzUln92XHobGCDcTeiC g==; X-IronPort-AV: E=McAfee;i="6500,9779,10606"; a="315736630" X-IronPort-AV: E=Sophos;i="5.97,259,1669104000"; d="scan'208";a="315736630" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Jan 2023 23:46:07 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10606"; a="657775570" X-IronPort-AV: E=Sophos;i="5.97,259,1669104000"; d="scan'208";a="657775570" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga007.jf.intel.com with ESMTP; 30 Jan 2023 23:46:05 -0800 From: Lu Baolu To: Joerg Roedel Cc: kan.liang@linux.intel.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 09/12] iommu/vt-d: Add IOMMU perfmon support Date: Tue, 31 Jan 2023 15:37:37 +0800 Message-Id: <20230131073740.378984-10-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230131073740.378984-1-baolu.lu@linux.intel.com> References: <20230131073740.378984-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Kan Liang Implement the IOMMU performance monitor capability, which supports the collection of information about key events occurring during operation of the remapping hardware, to aid performance tuning and debug. The IOMMU perfmon support is implemented as part of the IOMMU driver and interfaces with the Linux perf subsystem. The IOMMU PMU has the following unique features compared with the other PMUs. - Support counting. Not support sampling. - Does not support per-thread counting. The scope is system-wide. - Support per-counter capability register. The event constraints can be enumerated. - The available event and event group can also be enumerated. - Extra Enhanced Commands are introduced to control the counters. Add a new variable, struct iommu_pmu *pmu, to in the struct intel_iommu to track the PMU related information. Add iommu_pmu_register() and iommu_pmu_unregister() to register and unregister a IOMMU PMU. The register function setup the IOMMU PMU ops and invoke the standard perf_pmu_register() interface to register a PMU in the perf subsystem. This patch only exposes the functions. The following patch will enable them in the IOMMU driver. The IOMMU PMUs can be found under /sys/bus/event_source/devices/dmar* The available filters and event format can be found at the format folder $ ls /sys/bus/event_source/devices/dmar1/format/ event event_group filter_ats filter_ats_en filter_page_table filter_page_table_en The supported events can be found at the events folder $ ls /sys/bus/event_source/devices/dmar1/events/ ats_blocked fs_nonleaf_hit int_cache_hit_posted iommu_mem_blocked iotlb_hit pasid_cache_lookup ss_nonleaf_hit ctxt_cache_hit fs_nonleaf_lookup int_cache_lookup iommu_mrds iotlb_lookup pg_req_posted ss_nonleaf_lookup ctxt_cache_lookup int_cache_hit_nonposted iommu_clocks iommu_requests pasid_cache_hit pw_occupancy The command below illustrates filter usage with a simple example. $ perf stat -e dmar1/iommu_requests,filter_ats_en=3D0x1,filter_ats=3D0x1/ -a sleep 1 Performance counter stats for 'system wide': 368,947 dmar1/iommu_requests,filter_ats_en=3D0x1,filter_ats=3D0x1/ 1.002592074 seconds time elapsed Signed-off-by: Kan Liang Link: https://lore.kernel.org/r/20230128200428.1459118-5-kan.liang@linux.in= tel.com Signed-off-by: Lu Baolu --- drivers/iommu/intel/iommu.h | 15 + drivers/iommu/intel/perfmon.h | 24 + drivers/iommu/intel/perfmon.c | 526 ++++++++++++++++++ .../sysfs-bus-event_source-devices-iommu | 29 + 4 files changed, 594 insertions(+) create mode 100644 Documentation/ABI/testing/sysfs-bus-event_source-device= s-iommu diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h index f918e83bf91c..d56b3c386366 100644 --- a/drivers/iommu/intel/iommu.h +++ b/drivers/iommu/intel/iommu.h @@ -22,6 +22,7 @@ #include #include #include +#include =20 #include #include @@ -606,6 +607,16 @@ struct dmar_domain { iommu core */ }; =20 +/* + * In theory, the VT-d 4.0 spec can support up to 2 ^ 16 counters. + * But in practice, there are only 14 counters for the existing + * platform. Setting the max number of counters to 64 should be good + * enough for a long time. Also, supporting more than 64 counters + * requires more extras, e.g., extra freeze and overflow registers, + * which is not necessary for now. + */ +#define IOMMU_PMU_IDX_MAX 64 + struct iommu_pmu { struct intel_iommu *iommu; u32 num_cntr; /* Number of counters */ @@ -620,6 +631,10 @@ struct iommu_pmu { =20 u64 *evcap; /* Indicates all supported events */ u32 **cntr_evcap; /* Supported events of each counter. */ + + struct pmu pmu; + DECLARE_BITMAP(used_mask, IOMMU_PMU_IDX_MAX); + struct perf_event *event_list[IOMMU_PMU_IDX_MAX]; }; =20 struct intel_iommu { diff --git a/drivers/iommu/intel/perfmon.h b/drivers/iommu/intel/perfmon.h index 4b0d9c1fea6f..58606af9a2b9 100644 --- a/drivers/iommu/intel/perfmon.h +++ b/drivers/iommu/intel/perfmon.h @@ -7,6 +7,14 @@ #define IOMMU_PMU_NUM_OFF_REGS 4 #define IOMMU_PMU_OFF_REGS_STEP 4 =20 +#define IOMMU_PMU_FILTER_REQUESTER_ID 0x01 +#define IOMMU_PMU_FILTER_DOMAIN 0x02 +#define IOMMU_PMU_FILTER_PASID 0x04 +#define IOMMU_PMU_FILTER_ATS 0x08 +#define IOMMU_PMU_FILTER_PAGE_TABLE 0x10 + +#define IOMMU_PMU_FILTER_EN BIT(31) + #define IOMMU_PMU_CFG_OFFSET 0x100 #define IOMMU_PMU_CFG_CNTRCAP_OFFSET 0x80 #define IOMMU_PMU_CFG_CNTREVCAP_OFFSET 0x84 @@ -20,12 +28,18 @@ #define iommu_cntrcap_ios(p) (((p) >> 16) & 0x1) #define iommu_cntrcap_egcnt(p) (((p) >> 28) & 0xf) =20 +#define IOMMU_EVENT_CFG_EGI_SHIFT 8 +#define IOMMU_EVENT_CFG_ES_SHIFT 32 +#define IOMMU_EVENT_CFG_INT BIT_ULL(1) + #define iommu_event_select(p) ((p) & 0xfffffff) #define iommu_event_group(p) (((p) >> 28) & 0xf) =20 #ifdef CONFIG_INTEL_IOMMU_PERF_EVENTS int alloc_iommu_pmu(struct intel_iommu *iommu); void free_iommu_pmu(struct intel_iommu *iommu); +void iommu_pmu_register(struct intel_iommu *iommu); +void iommu_pmu_unregister(struct intel_iommu *iommu); #else static inline int alloc_iommu_pmu(struct intel_iommu *iommu) @@ -37,4 +51,14 @@ static inline void free_iommu_pmu(struct intel_iommu *iommu) { } + +static inline void +iommu_pmu_register(struct intel_iommu *iommu) +{ +} + +static inline void +iommu_pmu_unregister(struct intel_iommu *iommu) +{ +} #endif /* CONFIG_INTEL_IOMMU_PERF_EVENTS */ diff --git a/drivers/iommu/intel/perfmon.c b/drivers/iommu/intel/perfmon.c index db5791a54455..df9b78736462 100644 --- a/drivers/iommu/intel/perfmon.c +++ b/drivers/iommu/intel/perfmon.c @@ -10,6 +10,504 @@ #include "iommu.h" #include "perfmon.h" =20 +PMU_FORMAT_ATTR(event, "config:0-27"); /* ES: Events Select */ +PMU_FORMAT_ATTR(event_group, "config:28-31"); /* EGI: Event Group Index */ + +static struct attribute *iommu_pmu_format_attrs[] =3D { + &format_attr_event_group.attr, + &format_attr_event.attr, + NULL +}; + +static struct attribute_group iommu_pmu_format_attr_group =3D { + .name =3D "format", + .attrs =3D iommu_pmu_format_attrs, +}; + +/* The available events are added in attr_update later */ +static struct attribute *attrs_empty[] =3D { + NULL +}; + +static struct attribute_group iommu_pmu_events_attr_group =3D { + .name =3D "events", + .attrs =3D attrs_empty, +}; + +static const struct attribute_group *iommu_pmu_attr_groups[] =3D { + &iommu_pmu_format_attr_group, + &iommu_pmu_events_attr_group, + NULL +}; + +static inline struct iommu_pmu *dev_to_iommu_pmu(struct device *dev) +{ + /* + * The perf_event creates its own dev for each PMU. + * See pmu_dev_alloc() + */ + return container_of(dev_get_drvdata(dev), struct iommu_pmu, pmu); +} + +#define IOMMU_PMU_ATTR(_name, _format, _filter) \ + PMU_FORMAT_ATTR(_name, _format); \ + \ +static struct attribute *_name##_attr[] =3D { \ + &format_attr_##_name.attr, \ + NULL \ +}; \ + \ +static umode_t \ +_name##_is_visible(struct kobject *kobj, struct attribute *attr, int i) \ +{ \ + struct device *dev =3D kobj_to_dev(kobj); \ + struct iommu_pmu *iommu_pmu =3D dev_to_iommu_pmu(dev); \ + \ + if (!iommu_pmu) \ + return 0; \ + return (iommu_pmu->filter & _filter) ? attr->mode : 0; \ +} \ + \ +static struct attribute_group _name =3D { \ + .name =3D "format", \ + .attrs =3D _name##_attr, \ + .is_visible =3D _name##_is_visible, \ +}; + +IOMMU_PMU_ATTR(filter_requester_id_en, "config1:0", IOMMU_PMU_FILTER_REQU= ESTER_ID); +IOMMU_PMU_ATTR(filter_domain_en, "config1:1", IOMMU_PMU_FILTER_DOMAIN); +IOMMU_PMU_ATTR(filter_pasid_en, "config1:2", IOMMU_PMU_FILTER_PASID); +IOMMU_PMU_ATTR(filter_ats_en, "config1:3", IOMMU_PMU_FILTER_ATS); +IOMMU_PMU_ATTR(filter_page_table_en, "config1:4", IOMMU_PMU_FILTER_PAGE_T= ABLE); +IOMMU_PMU_ATTR(filter_requester_id, "config1:16-31", IOMMU_PMU_FILTER_REQU= ESTER_ID); +IOMMU_PMU_ATTR(filter_domain, "config1:32-47", IOMMU_PMU_FILTER_DOMAIN); +IOMMU_PMU_ATTR(filter_pasid, "config2:0-21", IOMMU_PMU_FILTER_PASID); +IOMMU_PMU_ATTR(filter_ats, "config2:24-28", IOMMU_PMU_FILTER_ATS); +IOMMU_PMU_ATTR(filter_page_table, "config2:32-36", IOMMU_PMU_FILTER_PAGE_T= ABLE); + +#define iommu_pmu_en_requester_id(e) ((e) & 0x1) +#define iommu_pmu_en_domain(e) (((e) >> 1) & 0x1) +#define iommu_pmu_en_pasid(e) (((e) >> 2) & 0x1) +#define iommu_pmu_en_ats(e) (((e) >> 3) & 0x1) +#define iommu_pmu_en_page_table(e) (((e) >> 4) & 0x1) +#define iommu_pmu_get_requester_id(filter) (((filter) >> 16) & 0xffff) +#define iommu_pmu_get_domain(filter) (((filter) >> 32) & 0xffff) +#define iommu_pmu_get_pasid(filter) ((filter) & 0x3fffff) +#define iommu_pmu_get_ats(filter) (((filter) >> 24) & 0x1f) +#define iommu_pmu_get_page_table(filter) (((filter) >> 32) & 0x1f) + +#define iommu_pmu_set_filter(_name, _config, _filter, _idx, _econfig) \ +{ \ + if ((iommu_pmu->filter & _filter) && iommu_pmu_en_##_name(_econfig)) { \ + dmar_writel(iommu_pmu->cfg_reg + _idx * IOMMU_PMU_CFG_OFFSET + \ + IOMMU_PMU_CFG_SIZE + \ + (ffs(_filter) - 1) * IOMMU_PMU_CFG_FILTERS_OFFSET, \ + iommu_pmu_get_##_name(_config) | IOMMU_PMU_FILTER_EN);\ + } \ +} + +#define iommu_pmu_clear_filter(_filter, _idx) \ +{ \ + if (iommu_pmu->filter & _filter) { \ + dmar_writel(iommu_pmu->cfg_reg + _idx * IOMMU_PMU_CFG_OFFSET + \ + IOMMU_PMU_CFG_SIZE + \ + (ffs(_filter) - 1) * IOMMU_PMU_CFG_FILTERS_OFFSET, \ + 0); \ + } \ +} + +/* + * Define the event attr related functions + * Input: _name: event attr name + * _string: string of the event in sysfs + * _g_idx: event group encoding + * _event: event encoding + */ +#define IOMMU_PMU_EVENT_ATTR(_name, _string, _g_idx, _event) \ + PMU_EVENT_ATTR_STRING(_name, event_attr_##_name, _string) \ + \ +static struct attribute *_name##_attr[] =3D { \ + &event_attr_##_name.attr.attr, \ + NULL \ +}; \ + \ +static umode_t \ +_name##_is_visible(struct kobject *kobj, struct attribute *attr, int i) \ +{ \ + struct device *dev =3D kobj_to_dev(kobj); \ + struct iommu_pmu *iommu_pmu =3D dev_to_iommu_pmu(dev); \ + \ + if (!iommu_pmu) \ + return 0; \ + return (iommu_pmu->evcap[_g_idx] & _event) ? attr->mode : 0; \ +} \ + \ +static struct attribute_group _name =3D { \ + .name =3D "events", \ + .attrs =3D _name##_attr, \ + .is_visible =3D _name##_is_visible, \ +}; + +IOMMU_PMU_EVENT_ATTR(iommu_clocks, "event_group=3D0x0,event=3D0x001", 0x0= , 0x001) +IOMMU_PMU_EVENT_ATTR(iommu_requests, "event_group=3D0x0,event=3D0x002", 0= x0, 0x002) +IOMMU_PMU_EVENT_ATTR(pw_occupancy, "event_group=3D0x0,event=3D0x004", 0x0= , 0x004) +IOMMU_PMU_EVENT_ATTR(ats_blocked, "event_group=3D0x0,event=3D0x008", 0x0,= 0x008) +IOMMU_PMU_EVENT_ATTR(iommu_mrds, "event_group=3D0x1,event=3D0x001", 0x1, = 0x001) +IOMMU_PMU_EVENT_ATTR(iommu_mem_blocked, "event_group=3D0x1,event=3D0x020"= , 0x1, 0x020) +IOMMU_PMU_EVENT_ATTR(pg_req_posted, "event_group=3D0x1,event=3D0x040", 0x= 1, 0x040) +IOMMU_PMU_EVENT_ATTR(ctxt_cache_lookup, "event_group=3D0x2,event=3D0x001"= , 0x2, 0x001) +IOMMU_PMU_EVENT_ATTR(ctxt_cache_hit, "event_group=3D0x2,event=3D0x002", 0= x2, 0x002) +IOMMU_PMU_EVENT_ATTR(pasid_cache_lookup, "event_group=3D0x2,event=3D0x004"= , 0x2, 0x004) +IOMMU_PMU_EVENT_ATTR(pasid_cache_hit, "event_group=3D0x2,event=3D0x008", = 0x2, 0x008) +IOMMU_PMU_EVENT_ATTR(ss_nonleaf_lookup, "event_group=3D0x2,event=3D0x010"= , 0x2, 0x010) +IOMMU_PMU_EVENT_ATTR(ss_nonleaf_hit, "event_group=3D0x2,event=3D0x020", 0= x2, 0x020) +IOMMU_PMU_EVENT_ATTR(fs_nonleaf_lookup, "event_group=3D0x2,event=3D0x040"= , 0x2, 0x040) +IOMMU_PMU_EVENT_ATTR(fs_nonleaf_hit, "event_group=3D0x2,event=3D0x080", 0= x2, 0x080) +IOMMU_PMU_EVENT_ATTR(hpt_nonleaf_lookup, "event_group=3D0x2,event=3D0x100"= , 0x2, 0x100) +IOMMU_PMU_EVENT_ATTR(hpt_nonleaf_hit, "event_group=3D0x2,event=3D0x200", = 0x2, 0x200) +IOMMU_PMU_EVENT_ATTR(iotlb_lookup, "event_group=3D0x3,event=3D0x001", 0x3= , 0x001) +IOMMU_PMU_EVENT_ATTR(iotlb_hit, "event_group=3D0x3,event=3D0x002", 0x3, = 0x002) +IOMMU_PMU_EVENT_ATTR(hpt_leaf_lookup, "event_group=3D0x3,event=3D0x004", = 0x3, 0x004) +IOMMU_PMU_EVENT_ATTR(hpt_leaf_hit, "event_group=3D0x3,event=3D0x008", 0x3= , 0x008) +IOMMU_PMU_EVENT_ATTR(int_cache_lookup, "event_group=3D0x4,event=3D0x001",= 0x4, 0x001) +IOMMU_PMU_EVENT_ATTR(int_cache_hit_nonposted, "event_group=3D0x4,event=3D0= x002", 0x4, 0x002) +IOMMU_PMU_EVENT_ATTR(int_cache_hit_posted, "event_group=3D0x4,event=3D0x00= 4", 0x4, 0x004) + +static const struct attribute_group *iommu_pmu_attr_update[] =3D { + &filter_requester_id_en, + &filter_domain_en, + &filter_pasid_en, + &filter_ats_en, + &filter_page_table_en, + &filter_requester_id, + &filter_domain, + &filter_pasid, + &filter_ats, + &filter_page_table, + &iommu_clocks, + &iommu_requests, + &pw_occupancy, + &ats_blocked, + &iommu_mrds, + &iommu_mem_blocked, + &pg_req_posted, + &ctxt_cache_lookup, + &ctxt_cache_hit, + &pasid_cache_lookup, + &pasid_cache_hit, + &ss_nonleaf_lookup, + &ss_nonleaf_hit, + &fs_nonleaf_lookup, + &fs_nonleaf_hit, + &hpt_nonleaf_lookup, + &hpt_nonleaf_hit, + &iotlb_lookup, + &iotlb_hit, + &hpt_leaf_lookup, + &hpt_leaf_hit, + &int_cache_lookup, + &int_cache_hit_nonposted, + &int_cache_hit_posted, + NULL +}; + +static inline void __iomem * +iommu_event_base(struct iommu_pmu *iommu_pmu, int idx) +{ + return iommu_pmu->cntr_reg + idx * iommu_pmu->cntr_stride; +} + +static inline void __iomem * +iommu_config_base(struct iommu_pmu *iommu_pmu, int idx) +{ + return iommu_pmu->cfg_reg + idx * IOMMU_PMU_CFG_OFFSET; +} + +static inline struct iommu_pmu *iommu_event_to_pmu(struct perf_event *even= t) +{ + return container_of(event->pmu, struct iommu_pmu, pmu); +} + +static inline u64 iommu_event_config(struct perf_event *event) +{ + u64 config =3D event->attr.config; + + return (iommu_event_select(config) << IOMMU_EVENT_CFG_ES_SHIFT) | + (iommu_event_group(config) << IOMMU_EVENT_CFG_EGI_SHIFT) | + IOMMU_EVENT_CFG_INT; +} + +static inline bool is_iommu_pmu_event(struct iommu_pmu *iommu_pmu, + struct perf_event *event) +{ + return event->pmu =3D=3D &iommu_pmu->pmu; +} + +static int iommu_pmu_validate_event(struct perf_event *event) +{ + struct iommu_pmu *iommu_pmu =3D iommu_event_to_pmu(event); + u32 event_group =3D iommu_event_group(event->attr.config); + + if (event_group >=3D iommu_pmu->num_eg) + return -EINVAL; + + return 0; +} + +static int iommu_pmu_validate_group(struct perf_event *event) +{ + struct iommu_pmu *iommu_pmu =3D iommu_event_to_pmu(event); + struct perf_event *sibling; + int nr =3D 0; + + /* + * All events in a group must be scheduled simultaneously. + * Check whether there is enough counters for all the events. + */ + for_each_sibling_event(sibling, event->group_leader) { + if (!is_iommu_pmu_event(iommu_pmu, sibling) || + sibling->state <=3D PERF_EVENT_STATE_OFF) + continue; + + if (++nr > iommu_pmu->num_cntr) + return -EINVAL; + } + + return 0; +} + +static int iommu_pmu_event_init(struct perf_event *event) +{ + struct hw_perf_event *hwc =3D &event->hw; + + if (event->attr.type !=3D event->pmu->type) + return -ENOENT; + + /* sampling not supported */ + if (event->attr.sample_period) + return -EINVAL; + + if (event->cpu < 0) + return -EINVAL; + + if (iommu_pmu_validate_event(event)) + return -EINVAL; + + hwc->config =3D iommu_event_config(event); + + return iommu_pmu_validate_group(event); +} + +static void iommu_pmu_event_update(struct perf_event *event) +{ + struct iommu_pmu *iommu_pmu =3D iommu_event_to_pmu(event); + struct hw_perf_event *hwc =3D &event->hw; + u64 prev_count, new_count, delta; + int shift =3D 64 - iommu_pmu->cntr_width; + +again: + prev_count =3D local64_read(&hwc->prev_count); + new_count =3D dmar_readq(iommu_event_base(iommu_pmu, hwc->idx)); + if (local64_xchg(&hwc->prev_count, new_count) !=3D prev_count) + goto again; + + /* + * The counter width is enumerated. Always shift the counter + * before using it. + */ + delta =3D (new_count << shift) - (prev_count << shift); + delta >>=3D shift; + + local64_add(delta, &event->count); +} + +static void iommu_pmu_start(struct perf_event *event, int flags) +{ + struct iommu_pmu *iommu_pmu =3D iommu_event_to_pmu(event); + struct intel_iommu *iommu =3D iommu_pmu->iommu; + struct hw_perf_event *hwc =3D &event->hw; + u64 count; + + if (WARN_ON_ONCE(!(hwc->state & PERF_HES_STOPPED))) + return; + + if (WARN_ON_ONCE(hwc->idx < 0 || hwc->idx >=3D IOMMU_PMU_IDX_MAX)) + return; + + if (flags & PERF_EF_RELOAD) + WARN_ON_ONCE(!(event->hw.state & PERF_HES_UPTODATE)); + + hwc->state =3D 0; + + /* Always reprogram the period */ + count =3D dmar_readq(iommu_event_base(iommu_pmu, hwc->idx)); + local64_set((&hwc->prev_count), count); + + /* + * The error of ecmd will be ignored. + * - The existing perf_event subsystem doesn't handle the error. + * Only IOMMU PMU returns runtime HW error. We don't want to + * change the existing generic interfaces for the specific case. + * - It's a corner case caused by HW, which is very unlikely to + * happen. There is nothing SW can do. + * - The worst case is that the user will get with + * perf command, which can give the user some hints. + */ + ecmd_submit_sync(iommu, DMA_ECMD_ENABLE, hwc->idx, 0); + + perf_event_update_userpage(event); +} + +static void iommu_pmu_stop(struct perf_event *event, int flags) +{ + struct iommu_pmu *iommu_pmu =3D iommu_event_to_pmu(event); + struct intel_iommu *iommu =3D iommu_pmu->iommu; + struct hw_perf_event *hwc =3D &event->hw; + + if (!(hwc->state & PERF_HES_STOPPED)) { + ecmd_submit_sync(iommu, DMA_ECMD_DISABLE, hwc->idx, 0); + + iommu_pmu_event_update(event); + + hwc->state |=3D PERF_HES_STOPPED | PERF_HES_UPTODATE; + } +} + +static inline int +iommu_pmu_validate_per_cntr_event(struct iommu_pmu *iommu_pmu, + int idx, struct perf_event *event) +{ + u32 event_group =3D iommu_event_group(event->attr.config); + u32 select =3D iommu_event_select(event->attr.config); + + if (!(iommu_pmu->cntr_evcap[idx][event_group] & select)) + return -EINVAL; + + return 0; +} + +static int iommu_pmu_assign_event(struct iommu_pmu *iommu_pmu, + struct perf_event *event) +{ + struct hw_perf_event *hwc =3D &event->hw; + int idx; + + /* + * The counters which support limited events are usually at the end. + * Schedule them first to accommodate more events. + */ + for (idx =3D iommu_pmu->num_cntr - 1; idx >=3D 0; idx--) { + if (test_and_set_bit(idx, iommu_pmu->used_mask)) + continue; + /* Check per-counter event capabilities */ + if (!iommu_pmu_validate_per_cntr_event(iommu_pmu, idx, event)) + break; + clear_bit(idx, iommu_pmu->used_mask); + } + if (idx < 0) + return -EINVAL; + + iommu_pmu->event_list[idx] =3D event; + hwc->idx =3D idx; + + /* config events */ + dmar_writeq(iommu_config_base(iommu_pmu, idx), hwc->config); + + iommu_pmu_set_filter(requester_id, event->attr.config1, + IOMMU_PMU_FILTER_REQUESTER_ID, idx, + event->attr.config1); + iommu_pmu_set_filter(domain, event->attr.config1, + IOMMU_PMU_FILTER_DOMAIN, idx, + event->attr.config1); + iommu_pmu_set_filter(pasid, event->attr.config1, + IOMMU_PMU_FILTER_PASID, idx, + event->attr.config1); + iommu_pmu_set_filter(ats, event->attr.config2, + IOMMU_PMU_FILTER_ATS, idx, + event->attr.config1); + iommu_pmu_set_filter(page_table, event->attr.config2, + IOMMU_PMU_FILTER_PAGE_TABLE, idx, + event->attr.config1); + + return 0; +} + +static int iommu_pmu_add(struct perf_event *event, int flags) +{ + struct iommu_pmu *iommu_pmu =3D iommu_event_to_pmu(event); + struct hw_perf_event *hwc =3D &event->hw; + int ret; + + ret =3D iommu_pmu_assign_event(iommu_pmu, event); + if (ret < 0) + return ret; + + hwc->state =3D PERF_HES_UPTODATE | PERF_HES_STOPPED; + + if (flags & PERF_EF_START) + iommu_pmu_start(event, 0); + + return 0; +} + +static void iommu_pmu_del(struct perf_event *event, int flags) +{ + struct iommu_pmu *iommu_pmu =3D iommu_event_to_pmu(event); + int idx =3D event->hw.idx; + + iommu_pmu_stop(event, PERF_EF_UPDATE); + + iommu_pmu_clear_filter(IOMMU_PMU_FILTER_REQUESTER_ID, idx); + iommu_pmu_clear_filter(IOMMU_PMU_FILTER_DOMAIN, idx); + iommu_pmu_clear_filter(IOMMU_PMU_FILTER_PASID, idx); + iommu_pmu_clear_filter(IOMMU_PMU_FILTER_ATS, idx); + iommu_pmu_clear_filter(IOMMU_PMU_FILTER_PAGE_TABLE, idx); + + iommu_pmu->event_list[idx] =3D NULL; + event->hw.idx =3D -1; + clear_bit(idx, iommu_pmu->used_mask); + + perf_event_update_userpage(event); +} + +static void iommu_pmu_enable(struct pmu *pmu) +{ + struct iommu_pmu *iommu_pmu =3D container_of(pmu, struct iommu_pmu, pmu); + struct intel_iommu *iommu =3D iommu_pmu->iommu; + + ecmd_submit_sync(iommu, DMA_ECMD_UNFREEZE, 0, 0); +} + +static void iommu_pmu_disable(struct pmu *pmu) +{ + struct iommu_pmu *iommu_pmu =3D container_of(pmu, struct iommu_pmu, pmu); + struct intel_iommu *iommu =3D iommu_pmu->iommu; + + ecmd_submit_sync(iommu, DMA_ECMD_FREEZE, 0, 0); +} + +static int __iommu_pmu_register(struct intel_iommu *iommu) +{ + struct iommu_pmu *iommu_pmu =3D iommu->pmu; + + iommu_pmu->pmu.name =3D iommu->name; + iommu_pmu->pmu.task_ctx_nr =3D perf_invalid_context; + iommu_pmu->pmu.event_init =3D iommu_pmu_event_init; + iommu_pmu->pmu.pmu_enable =3D iommu_pmu_enable; + iommu_pmu->pmu.pmu_disable =3D iommu_pmu_disable; + iommu_pmu->pmu.add =3D iommu_pmu_add; + iommu_pmu->pmu.del =3D iommu_pmu_del; + iommu_pmu->pmu.start =3D iommu_pmu_start; + iommu_pmu->pmu.stop =3D iommu_pmu_stop; + iommu_pmu->pmu.read =3D iommu_pmu_event_update; + iommu_pmu->pmu.attr_groups =3D iommu_pmu_attr_groups; + iommu_pmu->pmu.attr_update =3D iommu_pmu_attr_update; + iommu_pmu->pmu.capabilities =3D PERF_PMU_CAP_NO_EXCLUDE; + iommu_pmu->pmu.module =3D THIS_MODULE; + + return perf_pmu_register(&iommu_pmu->pmu, iommu_pmu->pmu.name, -1); +} + static inline void __iomem * get_perf_reg_address(struct intel_iommu *iommu, u32 offset) { @@ -45,11 +543,21 @@ int alloc_iommu_pmu(struct intel_iommu *iommu) if (!pcap_interrupt(perfcap)) return -ENODEV; =20 + /* Check required Enhanced Command Capability */ + if (!ecmd_has_pmu_essential(iommu)) + return -ENODEV; + iommu_pmu =3D kzalloc(sizeof(*iommu_pmu), GFP_KERNEL); if (!iommu_pmu) return -ENOMEM; =20 iommu_pmu->num_cntr =3D pcap_num_cntr(perfcap); + if (iommu_pmu->num_cntr > IOMMU_PMU_IDX_MAX) { + pr_warn_once("The number of IOMMU counters %d > max(%d), clipping!", + iommu_pmu->num_cntr, IOMMU_PMU_IDX_MAX); + iommu_pmu->num_cntr =3D IOMMU_PMU_IDX_MAX; + } + iommu_pmu->cntr_width =3D pcap_cntr_width(perfcap); iommu_pmu->filter =3D pcap_filters_mask(perfcap); iommu_pmu->cntr_stride =3D pcap_cntr_stride(perfcap); @@ -170,3 +678,21 @@ void free_iommu_pmu(struct intel_iommu *iommu) kfree(iommu_pmu); iommu->pmu =3D NULL; } + +void iommu_pmu_register(struct intel_iommu *iommu) +{ + if (!iommu->pmu) + return; + + if (__iommu_pmu_register(iommu)) { + pr_err("Failed to register PMU for iommu (seq_id =3D %d)\n", + iommu->seq_id); + free_iommu_pmu(iommu); + } +} + +void iommu_pmu_unregister(struct intel_iommu *iommu) +{ + if (iommu->pmu) + perf_pmu_unregister(&iommu->pmu->pmu); +} diff --git a/Documentation/ABI/testing/sysfs-bus-event_source-devices-iommu= b/Documentation/ABI/testing/sysfs-bus-event_source-devices-iommu new file mode 100644 index 000000000000..988210a0e8ce --- /dev/null +++ b/Documentation/ABI/testing/sysfs-bus-event_source-devices-iommu @@ -0,0 +1,29 @@ +What: /sys/bus/event_source/devices/dmar*/format +Date: Jan 2023 +KernelVersion: 6.3 +Contact: Kan Liang +Description: Read-only. Attribute group to describe the magic bits + that go into perf_event_attr.config, + perf_event_attr.config1 or perf_event_attr.config2 for + the IOMMU pmu. (See also + ABI/testing/sysfs-bus-event_source-devices-format). + + Each attribute in this group defines a bit range in + perf_event_attr.config, perf_event_attr.config1, + or perf_event_attr.config2. All supported attributes + are listed below (See the VT-d Spec 4.0 for possible + attribute values):: + + event =3D "config:0-27" - event ID + event_group =3D "config:28-31" - event group ID + + filter_requester_en =3D "config1:0" - Enable Requester ID filter + filter_domain_en =3D "config1:1" - Enable Domain ID filter + filter_pasid_en =3D "config1:2" - Enable PASID filter + filter_ats_en =3D "config1:3" - Enable Address Type filter + filter_page_table_en=3D "config1:4" - Enable Page Table Level fi= lter + filter_requester_id =3D "config1:16-31" - Requester ID filter + filter_domain =3D "config1:32-47" - Domain ID filter + filter_pasid =3D "config2:0-21" - PASID filter + filter_ats =3D "config2:24-28" - Address Type filter + filter_page_table =3D "config2:32-36" - Page Table Level filter --=20 2.34.1 From nobody Sun Sep 14 04:56:24 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1258C38142 for ; Tue, 31 Jan 2023 07:46:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230514AbjAaHqh (ORCPT ); Tue, 31 Jan 2023 02:46:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47864 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230444AbjAaHqQ (ORCPT ); Tue, 31 Jan 2023 02:46:16 -0500 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 033EF3E0BB for ; Mon, 30 Jan 2023 23:46:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675151175; x=1706687175; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=bRhbLbcdzD3YZbuz7wKC51Hdu2i1rzT4Orz0g+avwrI=; b=n9NeBySKCxaSrR9QsmRRsq0WvyhgLQDl9o6nhEXcAOHdWaeDvpNiITYf HUWEQZ3/sp5bQ1kEyF/m5LNPoBhn9VU4yNQkgc/F706IERP+0jsqr06yV xdgGaJWkztI/jzQvnFBlyNbKrorzNS4ZdJZBVFM4SWYjD/wHKgDXyTBXq H7XOAtSFt5rVzy2aA2h7Tk32D8XWFnV5Q0XSLPaFZkGTjJgNumrfXwRae ZV4QH+r/DAXsYKTJH0qWpuQaozHppKP968Es+s86x80u2tSK9agra+m4Y sUd6dx9bqpA/z+fuWwWepJrsaG9nSPb5HFdXF49VfGmZFZetpQuVRm6Xl A==; X-IronPort-AV: E=McAfee;i="6500,9779,10606"; a="315736638" X-IronPort-AV: E=Sophos;i="5.97,259,1669104000"; d="scan'208";a="315736638" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Jan 2023 23:46:08 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10606"; a="657775576" X-IronPort-AV: E=Sophos;i="5.97,259,1669104000"; d="scan'208";a="657775576" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga007.jf.intel.com with ESMTP; 30 Jan 2023 23:46:07 -0800 From: Lu Baolu To: Joerg Roedel Cc: kan.liang@linux.intel.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 10/12] iommu/vt-d: Support cpumask for IOMMU perfmon Date: Tue, 31 Jan 2023 15:37:38 +0800 Message-Id: <20230131073740.378984-11-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230131073740.378984-1-baolu.lu@linux.intel.com> References: <20230131073740.378984-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Kan Liang The perf subsystem assumes that all counters are by default per-CPU. So the user space tool reads a counter from each CPU. However, the IOMMU counters are system-wide and can be read from any CPU. Here we use a CPU mask to restrict counting to one CPU to handle the issue. (with CPU hotplug notifier to choose a different CPU if the chosen one is taken off-line). The CPU is exposed to /sys/bus/event_source/devices/dmar*/cpumask for the user space perf tool. Signed-off-by: Kan Liang Link: https://lore.kernel.org/r/20230128200428.1459118-6-kan.liang@linux.in= tel.com Signed-off-by: Lu Baolu --- include/linux/cpuhotplug.h | 1 + drivers/iommu/intel/perfmon.c | 113 ++++++++++++++++-- .../sysfs-bus-event_source-devices-iommu | 8 ++ 3 files changed, 114 insertions(+), 8 deletions(-) diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h index 6c6859bfc454..f2ea348ce3b0 100644 --- a/include/linux/cpuhotplug.h +++ b/include/linux/cpuhotplug.h @@ -221,6 +221,7 @@ enum cpuhp_state { CPUHP_AP_PERF_X86_CQM_ONLINE, CPUHP_AP_PERF_X86_CSTATE_ONLINE, CPUHP_AP_PERF_X86_IDXD_ONLINE, + CPUHP_AP_PERF_X86_IOMMU_PERF_ONLINE, CPUHP_AP_PERF_S390_CF_ONLINE, CPUHP_AP_PERF_S390_SF_ONLINE, CPUHP_AP_PERF_ARM_CCI_ONLINE, diff --git a/drivers/iommu/intel/perfmon.c b/drivers/iommu/intel/perfmon.c index df9b78736462..322d362b85e4 100644 --- a/drivers/iommu/intel/perfmon.c +++ b/drivers/iommu/intel/perfmon.c @@ -34,9 +34,28 @@ static struct attribute_group iommu_pmu_events_attr_grou= p =3D { .attrs =3D attrs_empty, }; =20 +static cpumask_t iommu_pmu_cpu_mask; + +static ssize_t +cpumask_show(struct device *dev, struct device_attribute *attr, char *buf) +{ + return cpumap_print_to_pagebuf(true, buf, &iommu_pmu_cpu_mask); +} +static DEVICE_ATTR_RO(cpumask); + +static struct attribute *iommu_pmu_cpumask_attrs[] =3D { + &dev_attr_cpumask.attr, + NULL +}; + +static struct attribute_group iommu_pmu_cpumask_attr_group =3D { + .attrs =3D iommu_pmu_cpumask_attrs, +}; + static const struct attribute_group *iommu_pmu_attr_groups[] =3D { &iommu_pmu_format_attr_group, &iommu_pmu_events_attr_group, + &iommu_pmu_cpumask_attr_group, NULL }; =20 @@ -679,20 +698,98 @@ void free_iommu_pmu(struct intel_iommu *iommu) iommu->pmu =3D NULL; } =20 +static int iommu_pmu_cpu_online(unsigned int cpu) +{ + if (cpumask_empty(&iommu_pmu_cpu_mask)) + cpumask_set_cpu(cpu, &iommu_pmu_cpu_mask); + + return 0; +} + +static int iommu_pmu_cpu_offline(unsigned int cpu) +{ + struct dmar_drhd_unit *drhd; + struct intel_iommu *iommu; + int target; + + if (!cpumask_test_and_clear_cpu(cpu, &iommu_pmu_cpu_mask)) + return 0; + + target =3D cpumask_any_but(cpu_online_mask, cpu); + + if (target < nr_cpu_ids) + cpumask_set_cpu(target, &iommu_pmu_cpu_mask); + else + target =3D -1; + + rcu_read_lock(); + + for_each_iommu(iommu, drhd) { + if (!iommu->pmu) + continue; + perf_pmu_migrate_context(&iommu->pmu->pmu, cpu, target); + } + rcu_read_unlock(); + + return 0; +} + +static int nr_iommu_pmu; + +static int iommu_pmu_cpuhp_setup(struct iommu_pmu *iommu_pmu) +{ + int ret; + + if (nr_iommu_pmu++) + return 0; + + ret =3D cpuhp_setup_state(CPUHP_AP_PERF_X86_IOMMU_PERF_ONLINE, + "driver/iommu/intel/perfmon:online", + iommu_pmu_cpu_online, + iommu_pmu_cpu_offline); + if (ret) + nr_iommu_pmu =3D 0; + + return ret; +} + +static void iommu_pmu_cpuhp_free(struct iommu_pmu *iommu_pmu) +{ + if (--nr_iommu_pmu) + return; + + cpuhp_remove_state(CPUHP_AP_PERF_X86_IOMMU_PERF_ONLINE); +} + void iommu_pmu_register(struct intel_iommu *iommu) { - if (!iommu->pmu) + struct iommu_pmu *iommu_pmu =3D iommu->pmu; + + if (!iommu_pmu) return; =20 - if (__iommu_pmu_register(iommu)) { - pr_err("Failed to register PMU for iommu (seq_id =3D %d)\n", - iommu->seq_id); - free_iommu_pmu(iommu); - } + if (__iommu_pmu_register(iommu)) + goto err; + + if (iommu_pmu_cpuhp_setup(iommu_pmu)) + goto unregister; + + return; + +unregister: + perf_pmu_unregister(&iommu_pmu->pmu); +err: + pr_err("Failed to register PMU for iommu (seq_id =3D %d)\n", iommu->seq_i= d); + free_iommu_pmu(iommu); } =20 void iommu_pmu_unregister(struct intel_iommu *iommu) { - if (iommu->pmu) - perf_pmu_unregister(&iommu->pmu->pmu); + struct iommu_pmu *iommu_pmu =3D iommu->pmu; + + if (!iommu_pmu) + return; + + iommu_pmu_cpuhp_free(iommu_pmu); + perf_pmu_unregister(&iommu_pmu->pmu); } diff --git a/Documentation/ABI/testing/sysfs-bus-event_source-devices-iommu= b/Documentation/ABI/testing/sysfs-bus-event_source-devices-iommu index 988210a0e8ce..d7af4919302e 100644 --- a/Documentation/ABI/testing/sysfs-bus-event_source-devices-iommu +++ b/Documentation/ABI/testing/sysfs-bus-event_source-devices-iommu @@ -27,3 +27,11 @@ Description: Read-only. Attribute group to describe the= magic bits filter_pasid =3D "config2:0-21" - PASID filter filter_ats =3D "config2:24-28" - Address Type filter filter_page_table =3D "config2:32-36" - Page Table Level filter + +What: /sys/bus/event_source/devices/dmar*/cpumask +Date: Jan 2023 +KernelVersion: 6.3 +Contact: Kan Liang +Description: Read-only. This file always returns the CPU to which the + IOMMU pmu is bound for access to all IOMMU pmu performance + monitoring events. --=20 2.34.1 From nobody Sun Sep 14 04:56:24 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 09116C38142 for ; Tue, 31 Jan 2023 07:46:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231264AbjAaHql (ORCPT ); Tue, 31 Jan 2023 02:46:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47760 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230458AbjAaHqQ (ORCPT ); Tue, 31 Jan 2023 02:46:16 -0500 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 11BF8303C2 for ; Mon, 30 Jan 2023 23:46:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675151175; x=1706687175; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=B3q6XG1ZtMeVnG49oTwHx8Qg7xHy4IhXHyiS5c+KM90=; b=E7B8ZJBTUSx4bGBK/uvLUSWY+A05DdnAA4UGNtlx26Yof5pXbCZtCiQE A4lajBqFs6pNqmMr8DzSqBolR85ZMVCUYTsFC6Be7I6wv6Z4CDHPB6qgQ LW66BaPcFLz27mxUaA6+ME2KwtklP8fWnm8nnT70PEWplmHbNjuTkNDhJ tqHdgCFllDYrLadcnwszpCR2P196tgV9K5/u9+JyZYn/D4wLg3+GpEOb9 f7MIYcmYioQnh/b5wRHrNXP2rXe+IvHizXNFPib2hJCR0qxkLEfm9IhBG 9QAS1LXg60S8zE4n2S/9GMHM2IsBMiag16l6xgLZ+aAyNzQuETNNBPHtH Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10606"; a="315736645" X-IronPort-AV: E=Sophos;i="5.97,259,1669104000"; d="scan'208";a="315736645" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Jan 2023 23:46:10 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10606"; a="657775580" X-IronPort-AV: E=Sophos;i="5.97,259,1669104000"; d="scan'208";a="657775580" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga007.jf.intel.com with ESMTP; 30 Jan 2023 23:46:08 -0800 From: Lu Baolu To: Joerg Roedel Cc: kan.liang@linux.intel.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 11/12] iommu/vt-d: Add IOMMU perfmon overflow handler support Date: Tue, 31 Jan 2023 15:37:39 +0800 Message-Id: <20230131073740.378984-12-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230131073740.378984-1-baolu.lu@linux.intel.com> References: <20230131073740.378984-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Kan Liang While enabled to count events and an event occurrence causes the counter value to increment and roll over to or past zero, this is termed a counter overflow. The overflow can trigger an interrupt. The IOMMU perfmon needs to handle the case properly. New HW IRQs are allocated for each IOMMU device for perfmon. The IRQ IDs are after the SVM range. In the overflow handler, the counter is not frozen. It's very unlikely that the same counter overflows again during the period. But it's possible that other counters overflow at the same time. Read the overflow register at the end of the handler and check whether there are more. Signed-off-by: Kan Liang Link: https://lore.kernel.org/r/20230128200428.1459118-7-kan.liang@linux.in= tel.com Signed-off-by: Lu Baolu --- drivers/iommu/intel/iommu.h | 11 ++++- drivers/iommu/intel/dmar.c | 2 + drivers/iommu/intel/perfmon.c | 82 +++++++++++++++++++++++++++++++++++ drivers/iommu/intel/svm.c | 2 +- 4 files changed, 95 insertions(+), 2 deletions(-) diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h index d56b3c386366..d7c61eb20f6f 100644 --- a/drivers/iommu/intel/iommu.h +++ b/drivers/iommu/intel/iommu.h @@ -130,6 +130,8 @@ #define DMAR_PERFCFGOFF_REG 0x310 #define DMAR_PERFOVFOFF_REG 0x318 #define DMAR_PERFCNTROFF_REG 0x31c +#define DMAR_PERFINTRSTS_REG 0x324 +#define DMAR_PERFINTRCTL_REG 0x328 #define DMAR_PERFEVNTCAP_REG 0x380 #define DMAR_ECMD_REG 0x400 #define DMAR_ECEO_REG 0x408 @@ -357,6 +359,9 @@ =20 #define DMA_VCS_PAS ((u64)1) =20 +/* PERFINTRSTS_REG */ +#define DMA_PERFINTRSTS_PIS ((u32)1) + #define IOMMU_WAIT_OP(iommu, offset, op, cond, sts) \ do { \ cycles_t start_time =3D get_cycles(); \ @@ -635,8 +640,12 @@ struct iommu_pmu { struct pmu pmu; DECLARE_BITMAP(used_mask, IOMMU_PMU_IDX_MAX); struct perf_event *event_list[IOMMU_PMU_IDX_MAX]; + unsigned char irq_name[16]; }; =20 +#define IOMMU_IRQ_ID_OFFSET_PRQ (DMAR_UNITS_SUPPORTED) +#define IOMMU_IRQ_ID_OFFSET_PERF (2 * DMAR_UNITS_SUPPORTED) + struct intel_iommu { void __iomem *reg; /* Pointer to hardware regs, virtual addr */ u64 reg_phys; /* physical address of hw register set */ @@ -650,7 +659,7 @@ struct intel_iommu { int seq_id; /* sequence id of the iommu */ int agaw; /* agaw of this iommu */ int msagaw; /* max sagaw of this iommu */ - unsigned int irq, pr_irq; + unsigned int irq, pr_irq, perf_irq; u16 segment; /* PCI segment# */ unsigned char name[13]; /* Device Name */ =20 diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c index 0e429bab436f..43db6ebe8b57 100644 --- a/drivers/iommu/intel/dmar.c +++ b/drivers/iommu/intel/dmar.c @@ -1879,6 +1879,8 @@ static inline int dmar_msi_reg(struct intel_iommu *io= mmu, int irq) return DMAR_FECTL_REG; else if (iommu->pr_irq =3D=3D irq) return DMAR_PECTL_REG; + else if (iommu->perf_irq =3D=3D irq) + return DMAR_PERFINTRCTL_REG; else BUG(); } diff --git a/drivers/iommu/intel/perfmon.c b/drivers/iommu/intel/perfmon.c index 322d362b85e4..e17d9743a0d8 100644 --- a/drivers/iommu/intel/perfmon.c +++ b/drivers/iommu/intel/perfmon.c @@ -505,6 +505,49 @@ static void iommu_pmu_disable(struct pmu *pmu) ecmd_submit_sync(iommu, DMA_ECMD_FREEZE, 0, 0); } =20 +static void iommu_pmu_counter_overflow(struct iommu_pmu *iommu_pmu) +{ + struct perf_event *event; + u64 status; + int i; + + /* + * Two counters may be overflowed very close. Always check + * whether there are more to handle. + */ + while ((status =3D dmar_readq(iommu_pmu->overflow))) { + for_each_set_bit(i, (unsigned long *)&status, iommu_pmu->num_cntr) { + /* + * Find the assigned event of the counter. + * Accumulate the value into the event->count. + */ + event =3D iommu_pmu->event_list[i]; + if (!event) { + pr_warn_once("Cannot find the assigned event for counter %d\n", i); + continue; + } + iommu_pmu_event_update(event); + } + + dmar_writeq(iommu_pmu->overflow, status); + } +} + +static irqreturn_t iommu_pmu_irq_handler(int irq, void *dev_id) +{ + struct intel_iommu *iommu =3D dev_id; + + if (!dmar_readl(iommu->reg + DMAR_PERFINTRSTS_REG)) + return IRQ_NONE; + + iommu_pmu_counter_overflow(iommu->pmu); + + /* Clear the status bit */ + dmar_writel(iommu->reg + DMAR_PERFINTRSTS_REG, DMA_PERFINTRSTS_PIS); + + return IRQ_HANDLED; +} + static int __iommu_pmu_register(struct intel_iommu *iommu) { struct iommu_pmu *iommu_pmu =3D iommu->pmu; @@ -698,6 +741,38 @@ void free_iommu_pmu(struct intel_iommu *iommu) iommu->pmu =3D NULL; } =20 +static int iommu_pmu_set_interrupt(struct intel_iommu *iommu) +{ + struct iommu_pmu *iommu_pmu =3D iommu->pmu; + int irq, ret; + + irq =3D dmar_alloc_hwirq(IOMMU_IRQ_ID_OFFSET_PERF + iommu->seq_id, iommu-= >node, iommu); + if (irq <=3D 0) + return -EINVAL; + + snprintf(iommu_pmu->irq_name, sizeof(iommu_pmu->irq_name), "dmar%d-perf",= iommu->seq_id); + + iommu->perf_irq =3D irq; + ret =3D request_threaded_irq(irq, NULL, iommu_pmu_irq_handler, + IRQF_ONESHOT, iommu_pmu->irq_name, iommu); + if (ret) { + dmar_free_hwirq(irq); + iommu->perf_irq =3D 0; + return ret; + } + return 0; +} + +static void iommu_pmu_unset_interrupt(struct intel_iommu *iommu) +{ + if (!iommu->perf_irq) + return; + + free_irq(iommu->perf_irq, iommu); + dmar_free_hwirq(iommu->perf_irq); + iommu->perf_irq =3D 0; +} + static int iommu_pmu_cpu_online(unsigned int cpu) { if (cpumask_empty(&iommu_pmu_cpu_mask)) @@ -774,8 +849,14 @@ void iommu_pmu_register(struct intel_iommu *iommu) if (iommu_pmu_cpuhp_setup(iommu_pmu)) goto unregister; =20 + /* Set interrupt for overflow */ + if (iommu_pmu_set_interrupt(iommu)) + goto cpuhp_free; + return; =20 +cpuhp_free: + iommu_pmu_cpuhp_free(iommu_pmu); unregister: perf_pmu_unregister(&iommu_pmu->pmu); err: @@ -790,6 +871,7 @@ void iommu_pmu_unregister(struct intel_iommu *iommu) if (!iommu_pmu) return; =20 + iommu_pmu_unset_interrupt(iommu); iommu_pmu_cpuhp_free(iommu_pmu); perf_pmu_unregister(&iommu_pmu->pmu); } diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index e7b9bedebcc0..7367f56c3bad 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -78,7 +78,7 @@ int intel_svm_enable_prq(struct intel_iommu *iommu) } iommu->prq =3D page_address(pages); =20 - irq =3D dmar_alloc_hwirq(DMAR_UNITS_SUPPORTED + iommu->seq_id, iommu->nod= e, iommu); + irq =3D dmar_alloc_hwirq(IOMMU_IRQ_ID_OFFSET_PRQ + iommu->seq_id, iommu->= node, iommu); if (irq <=3D 0) { pr_err("IOMMU: %s: Failed to create IRQ vector for page request queue\n", iommu->name); --=20 2.34.1 From nobody Sun Sep 14 04:56:24 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E03C0C636CD for ; Tue, 31 Jan 2023 07:46:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229759AbjAaHqs (ORCPT ); Tue, 31 Jan 2023 02:46:48 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47922 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230479AbjAaHqS (ORCPT ); Tue, 31 Jan 2023 02:46:18 -0500 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3BD9B3347B for ; Mon, 30 Jan 2023 23:46:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675151176; x=1706687176; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=tIIPqWLDaXi88jATwWGBnjxjewiMTRegyRpE2yrmVj8=; b=goPRq4GvmKhle/uq6TIv1tHFD+NjLEn3++E1mehXVl4nbY2yMBg5vMEF Yk+VvISIYg8jW89gprvMYXKVF30xOxBv+6wTcM5ZG+Vq74HC8SGEPmT2m cdiHw2kzEvFcn0vpfqurkHak7WhJ4oPkQn31R6wfbZNrfWpSAF2Xrgj0u 75sszs1riRKruyI8nUZNhr6EbyVHr2ns+5nvlGL5hlZQMo2jUQMZa2Uto Gosm2lz+/vVdyCPcerF8Hgzkn4FK/nulxuUbUcCpcKKTWJg1Ri1D48ifJ jPLCsbbp7I3YsK/ILdtX2l72JTQzR5Y6admiighgV+yE+En52kGC7GGJv Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10606"; a="315736651" X-IronPort-AV: E=Sophos;i="5.97,259,1669104000"; d="scan'208";a="315736651" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Jan 2023 23:46:11 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10606"; a="657775587" X-IronPort-AV: E=Sophos;i="5.97,259,1669104000"; d="scan'208";a="657775587" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga007.jf.intel.com with ESMTP; 30 Jan 2023 23:46:10 -0800 From: Lu Baolu To: Joerg Roedel Cc: kan.liang@linux.intel.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 12/12] iommu/vt-d: Enable IOMMU perfmon support Date: Tue, 31 Jan 2023 15:37:40 +0800 Message-Id: <20230131073740.378984-13-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230131073740.378984-1-baolu.lu@linux.intel.com> References: <20230131073740.378984-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Kan Liang Register and enable an IOMMU perfmon for each active IOMMU device. The failure of IOMMU perfmon registration doesn't impact other functionalities of an IOMMU device. Signed-off-by: Kan Liang Link: https://lore.kernel.org/r/20230128200428.1459118-8-kan.liang@linux.in= tel.com Signed-off-by: Lu Baolu --- drivers/iommu/intel/dmar.c | 3 +++ drivers/iommu/intel/iommu.c | 3 +++ 2 files changed, 6 insertions(+) diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c index 43db6ebe8b57..6acfe879589c 100644 --- a/drivers/iommu/intel/dmar.c +++ b/drivers/iommu/intel/dmar.c @@ -1144,6 +1144,8 @@ static int alloc_iommu(struct dmar_drhd_unit *drhd) err =3D iommu_device_register(&iommu->iommu, &intel_iommu_ops, NULL); if (err) goto err_sysfs; + + iommu_pmu_register(iommu); } =20 drhd->iommu =3D iommu; @@ -1166,6 +1168,7 @@ static int alloc_iommu(struct dmar_drhd_unit *drhd) static void free_iommu(struct intel_iommu *iommu) { if (intel_iommu_enabled && !iommu->drhd->ignored) { + iommu_pmu_unregister(iommu); iommu_device_unregister(&iommu->iommu); iommu_device_sysfs_remove(&iommu->iommu); } diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index e314c30d371a..691b306fada5 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -29,6 +29,7 @@ #include "../iommu-sva.h" #include "pasid.h" #include "cap_audit.h" +#include "perfmon.h" =20 #define ROOT_SIZE VTD_PAGE_SIZE #define CONTEXT_SIZE VTD_PAGE_SIZE @@ -4012,6 +4013,8 @@ int __init intel_iommu_init(void) intel_iommu_groups, "%s", iommu->name); iommu_device_register(&iommu->iommu, &intel_iommu_ops, NULL); + + iommu_pmu_register(iommu); } up_read(&dmar_global_lock); =20 --=20 2.34.1