From nobody Mon May 11 00:51:30 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4DE2EC433F5 for ; Wed, 20 Apr 2022 03:03:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1359079AbiDTDGM (ORCPT ); Tue, 19 Apr 2022 23:06:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35616 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1354676AbiDTDGJ (ORCPT ); Tue, 19 Apr 2022 23:06:09 -0400 X-Greylist: delayed 63 seconds by postgrey-1.37 at lindbergh.monkeyblade.net; Tue, 19 Apr 2022 20:03:23 PDT Received: from esa7.hc1455-7.c3s2.iphmx.com (esa7.hc1455-7.c3s2.iphmx.com [139.138.61.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7F08639172 for ; Tue, 19 Apr 2022 20:03:23 -0700 (PDT) X-IronPort-AV: E=McAfee;i="6400,9594,10322"; a="49273787" X-IronPort-AV: E=Sophos;i="5.90,274,1643641200"; d="scan'208";a="49273787" Received: from unknown (HELO oym-r2.gw.nic.fujitsu.com) ([210.162.30.90]) by esa7.hc1455-7.c3s2.iphmx.com with ESMTP; 20 Apr 2022 12:02:17 +0900 Received: from oym-m3.gw.nic.fujitsu.com (oym-nat-oym-m3.gw.nic.fujitsu.com [192.168.87.60]) by oym-r2.gw.nic.fujitsu.com (Postfix) with ESMTP id 43657DB9FA for ; Wed, 20 Apr 2022 12:02:17 +0900 (JST) Received: from yto-om1.fujitsu.com (yto-om1.o.css.fujitsu.com [10.128.89.162]) by oym-m3.gw.nic.fujitsu.com (Postfix) with ESMTP id ACD45D95E9 for ; Wed, 20 Apr 2022 12:02:15 +0900 (JST) Received: from cn-r05-10.example.com (n3235113.np.ts.nmh.cs.fujitsu.co.jp [10.123.235.113]) by yto-om1.fujitsu.com (Postfix) with ESMTP id 5B03940618A82; Wed, 20 Apr 2022 12:02:15 +0900 (JST) From: Kohei Tarumizu To: catalin.marinas@arm.com, will@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, fenghua.yu@intel.com, reinette.chatre@intel.com Cc: tarumizu.kohei@fujitsu.com Subject: [PATCH v3 1/9] drivers: base: Add hardware prefetch control core driver Date: Wed, 20 Apr 2022 12:02:15 +0900 Message-Id: <20220420030223.689259-2-tarumizu.kohei@fujitsu.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220420030223.689259-1-tarumizu.kohei@fujitsu.com> References: <20220420030223.689259-1-tarumizu.kohei@fujitsu.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This driver adds the register/unregister function to create the "prefetch_control" directory and some attribute files. Attributes are only present if the particular cache implements the relevant prefetcher controls If the architecture has control of the CPU's hardware prefetcher behavior, use this function to create sysfs. When registering, it is necessary to provide what type of hardware prefetcher is supported and how to read/write to the register. Following patches add support for A64FX and x86. Signed-off-by: Kohei Tarumizu --- drivers/base/pfctl.c | 458 ++++++++++++++++++++++++++++++++++++++++++ include/linux/pfctl.h | 49 +++++ 2 files changed, 507 insertions(+) create mode 100644 drivers/base/pfctl.c create mode 100644 include/linux/pfctl.h diff --git a/drivers/base/pfctl.c b/drivers/base/pfctl.c new file mode 100644 index 000000000000..035d51c64e33 --- /dev/null +++ b/drivers/base/pfctl.c @@ -0,0 +1,458 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright 2022 FUJITSU LIMITED + * + * This driver provides tunable sysfs interface for Hardware Prefetch Cont= rol. + * See Documentation/ABI/testing/sysfs-devices-system-cpu for more informa= tion. + * + * This code provides architecture-independent functions such as create and + * remove attribute file. + * The implementation of reads and writes to the Hardware Prefetch Control + * register is architecture-dependent. Therefore, each architecture regist= er + * a callback to read and write the register via pfctl_register_driver(). + */ + +#include +#include +#include +#include +#include +#include + +#ifdef pr_fmt +#undef pr_fmt +#endif +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +static DEFINE_PER_CPU(struct device *, cache_device_pcpu); +#define per_cpu_cache_device(cpu) (per_cpu(cache_device_pcpu, cpu)) + +struct pfctl_driver *pdriver; +enum cpuhp_state hp_online; + +static const char dist_auto_string[] =3D "auto"; +static const char strength_strong_string[] =3D "strong"; +static const char strength_weak_string[] =3D "weak"; + +static bool prefetcher_is_available(unsigned int level, enum cache_type ty= pe, + int prefetcher) +{ + if ((level =3D=3D 1) && (type =3D=3D CACHE_TYPE_DATA)) { + if (pdriver->supported_l1d_prefetcher & prefetcher) + return true; + } else if ((level =3D=3D 2) && (type =3D=3D CACHE_TYPE_UNIFIED)) { + if (pdriver->supported_l2_prefetcher & prefetcher) + return true; + } + + return false; +} + +/* + * Returns the cpu number of the cpu_device(/sys/devices/system/cpu/cpuX) + * in the ancestor directory of prefetch_control. + * + * When initializing this driver, it is verified that the cache directory = exists + * under cpu_device. Therefore, the third level up from prefetch_control is + * cpu_device as shown below. + * + * /sys/devices/system/cpu/cpuX/cache/indexX/prefetch_control + */ +static inline unsigned int get_pfctl_device_cpunum(struct device *pfctl_de= v) +{ + return pfctl_dev->parent->parent->parent->id; +} + +#define pfctl_enable_show(prefetcher, pattr) \ +static ssize_t \ +prefetcher##_enable_show(struct device *dev, \ + struct device_attribute *attr, char *buf) \ +{ \ + int ret; \ + u64 val; \ + unsigned int cpu; \ + struct cacheinfo *this_leaf =3D dev_get_drvdata(dev->parent); \ + \ + cpu =3D get_pfctl_device_cpunum(dev); \ + \ + ret =3D pdriver->read_pfreg(pattr, cpu, this_leaf->level, &val); \ + if (ret < 0) \ + return ret; \ + \ + if ((val =3D=3D PFCTL_ENABLE_VAL) || (val =3D=3D PFCTL_DISABLE_VAL)) \ + return sysfs_emit(buf, "%llu\n", val); \ + else \ + return -EINVAL; \ +} + +pfctl_enable_show(hardware_prefetcher, HWPF_ENABLE); +pfctl_enable_show(ip_prefetcher, IPPF_ENABLE); +pfctl_enable_show(adjacent_cache_line_prefetcher, ACLPF_ENABLE); +pfctl_enable_show(stream_detect_prefetcher, SDPF_ENABLE); + +static ssize_t +stream_detect_prefetcher_strength_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + int ret; + u64 val; + unsigned int cpu; + struct cacheinfo *this_leaf =3D dev_get_drvdata(dev->parent); + + cpu =3D get_pfctl_device_cpunum(dev); + + ret =3D pdriver->read_pfreg(SDPF_STRENGTH, cpu, this_leaf->level, &val); + if (ret < 0) + return ret; + + if (val =3D=3D PFCTL_STRONG_VAL) + return sysfs_emit(buf, "%s\n", strength_strong_string); + if (val =3D=3D PFCTL_WEAK_VAL) + return sysfs_emit(buf, "%s\n", strength_weak_string); + else + return -EINVAL; +} + +static ssize_t +stream_detect_prefetcher_strength_available_show(struct device *dev, + struct device_attribute *attr, + char *buf) +{ + ssize_t len =3D 0; + + if (pdriver->supported_sdpf_strength & STRENGTH_STRONG) + len +=3D sysfs_emit_at(buf, len, "%s ", strength_strong_string); + + if (pdriver->supported_sdpf_strength & STRENGTH_WEAK) + len +=3D sysfs_emit_at(buf, len, "%s ", strength_weak_string); + + len +=3D sysfs_emit_at(buf, len, "\n"); + + return len; +} + +static ssize_t +stream_detect_prefetcher_dist_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + int ret; + u64 val; + unsigned int cpu; + struct cacheinfo *this_leaf =3D dev_get_drvdata(dev->parent); + + cpu =3D get_pfctl_device_cpunum(dev); + + ret =3D pdriver->read_pfreg(SDPF_DIST, cpu, this_leaf->level, &val); + if (ret < 0) + return ret; + + if (val =3D=3D PFCTL_DIST_AUTO_VAL) + return sysfs_emit(buf, "%s\n", dist_auto_string); + else + return sysfs_emit(buf, "%llu\n", val); +} + +#define pfctl_enable_store(prefetcher, pattr) \ +static ssize_t \ +prefetcher##_enable_store(struct device *dev, \ + struct device_attribute *attr, \ + const char *buf, size_t count) \ +{ \ + int ret; \ + u64 val; \ + unsigned int cpu; \ + struct cacheinfo *this_leaf =3D dev_get_drvdata(dev->parent); \ + \ + ret =3D kstrtoull(buf, 10, &val); \ + if (ret < 0) \ + return -EINVAL; \ + \ + if ((val !=3D PFCTL_ENABLE_VAL) && (val !=3D PFCTL_DISABLE_VAL)) \ + return -EINVAL; \ + \ + cpu =3D get_pfctl_device_cpunum(dev); \ + \ + ret =3D pdriver->write_pfreg(pattr, cpu, this_leaf->level, val); \ + if (ret < 0) \ + return ret; \ + \ + return count; \ +} + +pfctl_enable_store(hardware_prefetcher, HWPF_ENABLE); +pfctl_enable_store(ip_prefetcher, IPPF_ENABLE); +pfctl_enable_store(adjacent_cache_line_prefetcher, ACLPF_ENABLE); +pfctl_enable_store(stream_detect_prefetcher, SDPF_ENABLE); + +static ssize_t +stream_detect_prefetcher_strength_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + int ret; + u64 val; + unsigned int cpu; + struct cacheinfo *this_leaf =3D dev_get_drvdata(dev->parent); + + if (sysfs_streq(buf, strength_strong_string)) + val =3D PFCTL_STRONG_VAL; + else if (sysfs_streq(buf, strength_weak_string)) + val =3D PFCTL_WEAK_VAL; + else + return -EINVAL; + + cpu =3D get_pfctl_device_cpunum(dev); + + ret =3D pdriver->write_pfreg(SDPF_STRENGTH, cpu, this_leaf->level, val); + if (ret < 0) + return ret; + + return count; +} + +static ssize_t +stream_detect_prefetcher_dist_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + int ret; + u64 val; + unsigned int cpu; + struct cacheinfo *this_leaf =3D dev_get_drvdata(dev->parent); + + if (sysfs_streq(buf, dist_auto_string)) { + val =3D PFCTL_DIST_AUTO_VAL; + } else { + ret =3D kstrtoull(buf, 10, &val); + if (ret < 0 || val < 1) + return -EINVAL; + } + + cpu =3D get_pfctl_device_cpunum(dev); + + ret =3D pdriver->write_pfreg(SDPF_DIST, cpu, this_leaf->level, val); + if (ret < 0) + return ret; + + return count; +} + +static DEVICE_ATTR_ADMIN_RW(hardware_prefetcher_enable); +static DEVICE_ATTR_ADMIN_RW(ip_prefetcher_enable); +static DEVICE_ATTR_ADMIN_RW(adjacent_cache_line_prefetcher_enable); +static DEVICE_ATTR_ADMIN_RW(stream_detect_prefetcher_enable); +static DEVICE_ATTR_ADMIN_RW(stream_detect_prefetcher_strength); +static DEVICE_ATTR_ADMIN_RO(stream_detect_prefetcher_strength_available); +static DEVICE_ATTR_ADMIN_RW(stream_detect_prefetcher_dist); + +static umode_t +pfctl_attrs_is_visible(struct kobject *kobj, struct attribute *attr, int u= nused) +{ + struct device *dev =3D kobj_to_dev(kobj); + struct cacheinfo *this_leaf =3D dev_get_drvdata(dev->parent); + umode_t mode =3D attr->mode; + + if ((attr =3D=3D &dev_attr_hardware_prefetcher_enable.attr) && + (prefetcher_is_available(this_leaf->level, this_leaf->type, HWPF))) + return mode; + + if ((attr =3D=3D &dev_attr_ip_prefetcher_enable.attr) && + (prefetcher_is_available(this_leaf->level, this_leaf->type, IPPF))) + return mode; + + if ((attr =3D=3D &dev_attr_adjacent_cache_line_prefetcher_enable.attr) && + (prefetcher_is_available(this_leaf->level, this_leaf->type, ACLPF))) + return mode; + + if (((attr =3D=3D &dev_attr_stream_detect_prefetcher_enable.attr) || + (attr =3D=3D &dev_attr_stream_detect_prefetcher_strength.attr) || + (attr =3D=3D &dev_attr_stream_detect_prefetcher_strength_available.a= ttr) + || (attr =3D=3D &dev_attr_stream_detect_prefetcher_dist.attr)) && + (prefetcher_is_available(this_leaf->level, this_leaf->type, SDPF))) + return mode; + + return 0; +} + +static struct attribute *pfctl_attrs[] =3D { + &dev_attr_hardware_prefetcher_enable.attr, + &dev_attr_ip_prefetcher_enable.attr, + &dev_attr_adjacent_cache_line_prefetcher_enable.attr, + &dev_attr_stream_detect_prefetcher_enable.attr, + &dev_attr_stream_detect_prefetcher_strength.attr, + &dev_attr_stream_detect_prefetcher_strength_available.attr, + &dev_attr_stream_detect_prefetcher_dist.attr, + NULL, +}; + +static const struct attribute_group pfctl_group =3D { + .attrs =3D pfctl_attrs, + .is_visible =3D pfctl_attrs_is_visible, +}; + +static const struct attribute_group *pfctl_groups[] =3D { + &pfctl_group, + NULL, +}; + +static int find_cache_device(unsigned int cpu) +{ + struct device *cpu_dev =3D get_cpu_device(cpu); + struct device *cache_dev; + + cache_dev =3D device_find_child_by_name(cpu_dev, "cache"); + if (!cache_dev) + return -ENODEV; + + per_cpu_cache_device(cpu) =3D cache_dev; + return 0; +} + +static int _remove_pfctl_attr(struct device *dev, void *data) +{ + struct cacheinfo *leaf =3D dev_get_drvdata(dev); + struct device *pfctl_dev; + + if (!prefetcher_is_available(leaf->level, leaf->type, ANYPF)) + return 0; + + pfctl_dev =3D device_find_child_by_name(dev, "prefetch_control"); + if (!pfctl_dev) + return 0; + + device_unregister(pfctl_dev); + put_device(pfctl_dev); + return 0; +} + +static void remove_pfctl_attr(unsigned int cpu) +{ + struct device *cache_dev =3D per_cpu_cache_device(cpu); + + if (!cache_dev) + return; + + device_for_each_child(cache_dev, NULL, _remove_pfctl_attr); + put_device(cache_dev); +} + +static int _create_pfctl_attr(struct device *dev, void *data) +{ + struct cacheinfo *leaf =3D dev_get_drvdata(dev); + struct device *pfctl_dev; + + if (!prefetcher_is_available(leaf->level, leaf->type, ANYPF)) + return 0; + + pfctl_dev =3D cpu_device_create(dev, NULL, pfctl_groups, + "prefetch_control"); + if (IS_ERR(pfctl_dev)) + return PTR_ERR(pfctl_dev); + + return 0; +} + +static int create_pfctl_attr(unsigned int cpu) +{ + int ret; + struct device *cache_dev =3D per_cpu_cache_device(cpu); + + if (!cache_dev) + return -ENODEV; + + ret =3D device_for_each_child(cache_dev, NULL, _create_pfctl_attr); + if (ret < 0) + return ret; + + return 0; +} + +static int pfctl_online(unsigned int cpu) +{ + int ret; + + ret =3D find_cache_device(cpu); + if (ret < 0) + return ret; + + ret =3D create_pfctl_attr(cpu); + if (ret < 0) + return ret; + + return 0; +} + +static int pfctl_prepare_down(unsigned int cpu) +{ + remove_pfctl_attr(cpu); + + return 0; +} + +/** + * pfctl_register_driver - register a Hardware Prefetch Control driver + * @driver_data: struct pfctl_driver must contain the supported prefetcher= type + * and function pointer for reading and writing hardware pre= fetch + * register. If these are not defined this function return e= rror. + * + * Note: This function must be called after the cache device is initialized + * because it requires access to the cache device. + * (e.g. Call at the late_initcall) + * + * Context: Any context. + * Return: 0 on success, negative error code on failure. + */ +int pfctl_register_driver(struct pfctl_driver *driver_data) +{ + int ret; + + if (pdriver) + return -EEXIST; + + if (!(driver_data->supported_l1d_prefetcher & ANYPF) && + !(driver_data->supported_l2_prefetcher & ANYPF)) + return -EINVAL; + + if ((driver_data->supported_l1d_prefetcher & SDPF) || + (driver_data->supported_l2_prefetcher & SDPF)) + if (!(driver_data->supported_sdpf_strength & ANY_STRENGTH)) + return -EINVAL; + + if (!driver_data->read_pfreg || !driver_data->write_pfreg) + return -EINVAL; + + pdriver =3D driver_data; + + ret =3D cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "base/pfctl:online", + pfctl_online, pfctl_prepare_down); + if (ret < 0) { + pr_err("failed to register hotplug callbacks\n"); + pdriver =3D NULL; + return ret; + } + + hp_online =3D ret; + + return 0; +} +EXPORT_SYMBOL_GPL(pfctl_register_driver); + +/** + * pfctl_unregister_driver - unregister the Hardware Prefetch Control driv= er + * @driver_data: Used to verify that this function is called by the driver= that + * called pfctl_register_driver by determining if driver_dat= a is + * the same. + * + * Context: Any context. + * Return: nothing. + */ +void pfctl_unregister_driver(struct pfctl_driver *driver_data) +{ + if (!pdriver || (driver_data !=3D pdriver)) + return; + + cpuhp_remove_state(hp_online); + + pdriver =3D NULL; +} +EXPORT_SYMBOL_GPL(pfctl_unregister_driver); diff --git a/include/linux/pfctl.h b/include/linux/pfctl.h new file mode 100644 index 000000000000..9340b23f3d03 --- /dev/null +++ b/include/linux/pfctl.h @@ -0,0 +1,49 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _LINUX_PFCTL_H +#define _LINUX_PFCTL_H + +#define PFCTL_ENABLE_VAL 1 +#define PFCTL_DISABLE_VAL 0 +#define PFCTL_STRONG_VAL 0 +#define PFCTL_WEAK_VAL 1 +#define PFCTL_DIST_AUTO_VAL 0 + +enum pfctl_attr { + HWPF_ENABLE, + IPPF_ENABLE, + ACLPF_ENABLE, + SDPF_ENABLE, + SDPF_STRENGTH, + SDPF_DIST, +}; + +enum prefetcher { + HWPF =3D BIT(0), /* Hardware Prefetcher */ + IPPF =3D BIT(1), /* IP Prefetcher */ + ACLPF =3D BIT(2), /* Adjacent Cache Line Prefetcher */ + SDPF =3D BIT(3), /* Stream Detect Prefetcher */ + ANYPF =3D HWPF|IPPF|ACLPF|SDPF, +}; + +enum strength { + STRENGTH_STRONG =3D BIT(0), + STRENGTH_WEAK =3D BIT(1), + ANY_STRENGTH =3D STRENGTH_STRONG|STRENGTH_WEAK, +}; + +struct pfctl_driver { + unsigned int supported_l1d_prefetcher; + unsigned int supported_l2_prefetcher; + + unsigned int supported_sdpf_strength; /* Set this if support SDPF */ + + int (*read_pfreg)(enum pfctl_attr pattr, unsigned int cpu, + unsigned int level, u64 *val); + int (*write_pfreg)(enum pfctl_attr pattr, unsigned int cpu, + unsigned int level, u64 val); +}; + +int pfctl_register_driver(struct pfctl_driver *driver_data); +void pfctl_unregister_driver(struct pfctl_driver *driver_data); + +#endif --=20 2.27.0 From nobody Mon May 11 00:51:30 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 720BCC433EF for ; Wed, 20 Apr 2022 03:03:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1359096AbiDTDGU (ORCPT ); Tue, 19 Apr 2022 23:06:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35640 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1354806AbiDTDGK (ORCPT ); Tue, 19 Apr 2022 23:06:10 -0400 X-Greylist: delayed 63 seconds by postgrey-1.37 at lindbergh.monkeyblade.net; Tue, 19 Apr 2022 20:03:25 PDT Received: from esa6.hc1455-7.c3s2.iphmx.com (esa6.hc1455-7.c3s2.iphmx.com [68.232.139.139]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E3EF5393E3 for ; Tue, 19 Apr 2022 20:03:25 -0700 (PDT) X-IronPort-AV: E=McAfee;i="6400,9594,10322"; a="70939953" X-IronPort-AV: E=Sophos;i="5.90,274,1643641200"; d="scan'208";a="70939953" Received: from unknown (HELO oym-r2.gw.nic.fujitsu.com) ([210.162.30.90]) by esa6.hc1455-7.c3s2.iphmx.com with ESMTP; 20 Apr 2022 12:02:20 +0900 Received: from oym-m2.gw.nic.fujitsu.com (oym-nat-oym-m2.gw.nic.fujitsu.com [192.168.87.59]) by oym-r2.gw.nic.fujitsu.com (Postfix) with ESMTP id 7C33BDB9F5 for ; Wed, 20 Apr 2022 12:02:19 +0900 (JST) Received: from yto-om1.fujitsu.com (yto-om1.o.css.fujitsu.com [10.128.89.162]) by oym-m2.gw.nic.fujitsu.com (Postfix) with ESMTP id A97F0BDC32 for ; Wed, 20 Apr 2022 12:02:18 +0900 (JST) Received: from cn-r05-10.example.com (n3235113.np.ts.nmh.cs.fujitsu.co.jp [10.123.235.113]) by yto-om1.fujitsu.com (Postfix) with ESMTP id 50B8B40618A81; Wed, 20 Apr 2022 12:02:18 +0900 (JST) From: Kohei Tarumizu To: catalin.marinas@arm.com, will@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, fenghua.yu@intel.com, reinette.chatre@intel.com Cc: tarumizu.kohei@fujitsu.com Subject: [PATCH v3 2/9] drivers: base: Add Kconfig/Makefile to build hardware prefetch control core driver Date: Wed, 20 Apr 2022 12:02:16 +0900 Message-Id: <20220420030223.689259-3-tarumizu.kohei@fujitsu.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220420030223.689259-1-tarumizu.kohei@fujitsu.com> References: <20220420030223.689259-1-tarumizu.kohei@fujitsu.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This adds Kconfig/Makefile to build hardware prefetch control core driver. This also adds a MAINTAINERS entry. Signed-off-by: Kohei Tarumizu --- MAINTAINERS | 6 ++++++ drivers/base/Kconfig | 9 +++++++++ drivers/base/Makefile | 1 + 3 files changed, 16 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 40fa1955ca3f..f6640dc053c0 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -8615,6 +8615,12 @@ F: include/linux/hwmon*.h F: include/trace/events/hwmon*.h K: (devm_)?hwmon_device_(un)?register(|_with_groups|_with_info) =20 +HARDWARE PREFETCH CONTROL DRIVERS +M: Kohei Tarumizu +S: Maintained +F: drivers/base/pfctl.c +F: include/linux/pfctl.h + HARDWARE RANDOM NUMBER GENERATOR CORE M: Matt Mackall M: Herbert Xu diff --git a/drivers/base/Kconfig b/drivers/base/Kconfig index 6f04b831a5c0..8f8a69e7f645 100644 --- a/drivers/base/Kconfig +++ b/drivers/base/Kconfig @@ -230,4 +230,13 @@ config GENERIC_ARCH_NUMA Enable support for generic NUMA implementation. Currently, RISC-V and ARM64 use it. =20 +config HWPF_CONTROL + bool "Hardware Prefetch Control driver" + help + This driver allows user to control CPU's Hardware Prefetch behavior. + If the machine supports this behavior, it provides a sysfs interface. + + See Documentation/ABI/testing/sysfs-devices-system-cpu for more + information. + endmenu diff --git a/drivers/base/Makefile b/drivers/base/Makefile index 02f7f1358e86..13f3a0ddf3d1 100644 --- a/drivers/base/Makefile +++ b/drivers/base/Makefile @@ -25,6 +25,7 @@ obj-$(CONFIG_DEV_COREDUMP) +=3D devcoredump.o obj-$(CONFIG_GENERIC_MSI_IRQ_DOMAIN) +=3D platform-msi.o obj-$(CONFIG_GENERIC_ARCH_TOPOLOGY) +=3D arch_topology.o obj-$(CONFIG_GENERIC_ARCH_NUMA) +=3D arch_numa.o +obj-$(CONFIG_HWPF_CONTROL) +=3D pfctl.o =20 obj-y +=3D test/ =20 --=20 2.27.0 From nobody Mon May 11 00:51:30 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5501C433EF for ; Wed, 20 Apr 2022 03:03:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1359119AbiDTDG0 (ORCPT ); Tue, 19 Apr 2022 23:06:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35700 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1359082AbiDTDGN (ORCPT ); Tue, 19 Apr 2022 23:06:13 -0400 Received: from esa7.hc1455-7.c3s2.iphmx.com (esa7.hc1455-7.c3s2.iphmx.com [139.138.61.252]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 702C7393ED for ; Tue, 19 Apr 2022 20:03:28 -0700 (PDT) X-IronPort-AV: E=McAfee;i="6400,9594,10322"; a="49273818" X-IronPort-AV: E=Sophos;i="5.90,274,1643641200"; d="scan'208";a="49273818" Received: from unknown (HELO yto-r3.gw.nic.fujitsu.com) ([218.44.52.219]) by esa7.hc1455-7.c3s2.iphmx.com with ESMTP; 20 Apr 2022 12:02:24 +0900 Received: from yto-m4.gw.nic.fujitsu.com (yto-nat-yto-m4.gw.nic.fujitsu.com [192.168.83.67]) by yto-r3.gw.nic.fujitsu.com (Postfix) with ESMTP id 9158ED502B for ; Wed, 20 Apr 2022 12:02:23 +0900 (JST) Received: from yto-om1.fujitsu.com (yto-om1.o.css.fujitsu.com [10.128.89.162]) by yto-m4.gw.nic.fujitsu.com (Postfix) with ESMTP id BBEF0F103D for ; Wed, 20 Apr 2022 12:02:22 +0900 (JST) Received: from cn-r05-10.example.com (n3235113.np.ts.nmh.cs.fujitsu.co.jp [10.123.235.113]) by yto-om1.fujitsu.com (Postfix) with ESMTP id 323434060C93E; Wed, 20 Apr 2022 12:02:20 +0900 (JST) From: Kohei Tarumizu To: catalin.marinas@arm.com, will@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, fenghua.yu@intel.com, reinette.chatre@intel.com Cc: tarumizu.kohei@fujitsu.com Subject: [PATCH v3 3/9] soc: fujitsu: Add hardware prefetch control support for A64FX Date: Wed, 20 Apr 2022 12:02:17 +0900 Message-Id: <20220420030223.689259-4-tarumizu.kohei@fujitsu.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220420030223.689259-1-tarumizu.kohei@fujitsu.com> References: <20220420030223.689259-1-tarumizu.kohei@fujitsu.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This adds module init/exit code, and creates sysfs attribute files for "stream_detect_prefetcher_enable", "stream_detect_prefetcher_strong" and "stream_detect_prefetcher_dist". This driver works only if part number is FUJITSU_CPU_PART_A64FX at this point. The details of the registers to be read and written in this patch are described below. "https://github.com/fujitsu/A64FX/tree/master/doc/" A64FX_Specification_HPC_Extension_v1_EN.pdf Signed-off-by: Kohei Tarumizu --- drivers/soc/fujitsu/a64fx-pfctl.c | 356 ++++++++++++++++++++++++++++++ 1 file changed, 356 insertions(+) create mode 100644 drivers/soc/fujitsu/a64fx-pfctl.c diff --git a/drivers/soc/fujitsu/a64fx-pfctl.c b/drivers/soc/fujitsu/a64fx-= pfctl.c new file mode 100644 index 000000000000..0e072592be73 --- /dev/null +++ b/drivers/soc/fujitsu/a64fx-pfctl.c @@ -0,0 +1,356 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright 2022 FUJITSU LIMITED + * + * A64FX Hardware Prefetch Control support + */ + +#include +#include +#include +#include +#include +#include + +struct pfctl_driver a64fx_pfctl_driver; + +/* + * Constants for these add the "A64FX_SDPF" prefix to the name described in + * section "1.3.4.2. IMP_PF_STREAM_DETECT_CTRL_EL0" of "A64FX specificatio= n". + * (https://github.com/fujitsu/A64FX/tree/master/doc/A64FX_Specification_H= PC_Extension_v1_EN.pdf") + * See this document for register specification details. + */ +#define A64FX_SDPF_IMP_PF_STREAM_DETECT_CTRL_EL0 sys_reg(3, 3, 11, 4, 0) +#define A64FX_SDPF_V BIT_ULL(63) +#define A64FX_SDPF_L1PF_DIS BIT_ULL(59) +#define A64FX_SDPF_L2PF_DIS BIT_ULL(58) +#define A64FX_SDPF_L1W BIT_ULL(55) +#define A64FX_SDPF_L2W BIT_ULL(54) +#define A64FX_SDPF_L1_DIST GENMASK_ULL(27, 24) +#define A64FX_SDPF_L2_DIST GENMASK_ULL(19, 16) + +#define A64FX_SDPF_MIN_DIST_L1 256 +#define A64FX_SDPF_MIN_DIST_L2 1024 + +struct a64fx_read_info { + enum pfctl_attr pattr; + u64 val; + unsigned int level; + int ret; +}; + +struct a64fx_write_info { + enum pfctl_attr pattr; + u64 val; + unsigned int level; + int ret; +}; + +static int a64fx_get_sdpf_enable(u64 reg, unsigned int level) +{ + u64 val; + + switch (level) { + case 1: + val =3D FIELD_GET(A64FX_SDPF_L1PF_DIS, reg); + break; + case 2: + val =3D FIELD_GET(A64FX_SDPF_L2PF_DIS, reg); + break; + default: + return -EINVAL; + } + + if (val =3D=3D 0) + return PFCTL_ENABLE_VAL; + else if (val =3D=3D 1) + return PFCTL_DISABLE_VAL; + else + return -EINVAL; +} + +static int a64fx_modify_sdpf_enable(u64 *reg, unsigned int level, u64 val) +{ + if (val =3D=3D PFCTL_ENABLE_VAL) + val =3D 0; + else + val =3D 1; + + switch (level) { + case 1: + *reg &=3D ~A64FX_SDPF_L1PF_DIS; + *reg |=3D FIELD_PREP(A64FX_SDPF_L1PF_DIS, val); + break; + case 2: + *reg &=3D ~A64FX_SDPF_L2PF_DIS; + *reg |=3D FIELD_PREP(A64FX_SDPF_L2PF_DIS, val); + break; + default: + return -EINVAL; + } + + return 0; +} + +static int a64fx_get_sdpf_strength(u64 reg, unsigned int level) +{ + u64 val; + + switch (level) { + case 1: + val =3D FIELD_GET(A64FX_SDPF_L1W, reg); + break; + case 2: + val =3D FIELD_GET(A64FX_SDPF_L2W, reg); + break; + default: + return -EINVAL; + } + + if (val =3D=3D 0) + return PFCTL_STRONG_VAL; + else if (val =3D=3D 1) + return PFCTL_WEAK_VAL; + else + return -EINVAL; +} + +static int a64fx_modify_sdpf_strength(u64 *reg, unsigned int level, u64 va= l) +{ + if (val =3D=3D PFCTL_STRONG_VAL) + val =3D 0; + else + val =3D 1; + + switch (level) { + case 1: + *reg &=3D ~A64FX_SDPF_L1W; + *reg |=3D FIELD_PREP(A64FX_SDPF_L1W, val); + break; + case 2: + *reg &=3D ~A64FX_SDPF_L2W; + *reg |=3D FIELD_PREP(A64FX_SDPF_L2W, val); + break; + default: + return -EINVAL; + } + + return 0; +} + +static int a64fx_get_sdpf_dist(u64 reg, unsigned int level) +{ + switch (level) { + case 1: + return FIELD_GET(A64FX_SDPF_L1_DIST, reg) * + A64FX_SDPF_MIN_DIST_L1; + case 2: + return FIELD_GET(A64FX_SDPF_L2_DIST, reg) * + A64FX_SDPF_MIN_DIST_L2; + default: + return -EINVAL; + } +} + +static int a64fx_modify_sdpf_dist(u64 *reg, unsigned int level, u64 val) +{ + switch (level) { + case 1: + val =3D roundup(val, A64FX_SDPF_MIN_DIST_L1) / + A64FX_SDPF_MIN_DIST_L1; + if (!FIELD_FIT(A64FX_SDPF_L1_DIST, val)) + return -EINVAL; + *reg &=3D ~A64FX_SDPF_L1_DIST; + *reg |=3D FIELD_PREP(A64FX_SDPF_L1_DIST, val); + break; + case 2: + val =3D roundup(val, A64FX_SDPF_MIN_DIST_L2) / + A64FX_SDPF_MIN_DIST_L2; + if (!FIELD_FIT(A64FX_SDPF_L2_DIST, val)) + return -EINVAL; + *reg &=3D ~A64FX_SDPF_L2_DIST; + *reg |=3D FIELD_PREP(A64FX_SDPF_L2_DIST, val); + break; + default: + return -EINVAL; + } + + return 0; +} + +static void a64fx_enable_sdpf_verify(u64 *reg) +{ + *reg |=3D FIELD_PREP(A64FX_SDPF_V, 1); +} + +static int a64fx_get_sdpf_params(enum pfctl_attr pattr, u64 reg, + unsigned int level, u64 *val) +{ + int ret; + + switch (pattr) { + case SDPF_ENABLE: + ret =3D a64fx_get_sdpf_enable(reg, level); + break; + case SDPF_STRENGTH: + ret =3D a64fx_get_sdpf_strength(reg, level); + break; + case SDPF_DIST: + ret =3D a64fx_get_sdpf_dist(reg, level); + break; + default: + return -ENOENT; + } + + if (ret < 0) + return ret; + *val =3D ret; + + return 0; +} + +static int a64fx_modify_pfreg_val(enum pfctl_attr pattr, u64 *reg, + unsigned int level, u64 val) +{ + int ret; + + switch (pattr) { + case SDPF_ENABLE: + ret =3D a64fx_modify_sdpf_enable(reg, level, val); + break; + case SDPF_STRENGTH: + ret =3D a64fx_modify_sdpf_strength(reg, level, val); + break; + case SDPF_DIST: + ret =3D a64fx_modify_sdpf_dist(reg, level, val); + break; + default: + return -ENOENT; + } + + if (ret < 0) + return ret; + + a64fx_enable_sdpf_verify(reg); + + return 0; +} + +static void _a64fx_read_pfreg(void *info) +{ + u64 reg; + struct a64fx_read_info *rinfo =3D info; + + reg =3D read_sysreg_s(A64FX_SDPF_IMP_PF_STREAM_DETECT_CTRL_EL0); + + rinfo->ret =3D a64fx_get_sdpf_params(rinfo->pattr, reg, rinfo->level, + &rinfo->val); +} + +static int a64fx_read_pfreg(enum pfctl_attr pattr, unsigned int cpu, + unsigned int level, u64 *val) +{ + struct a64fx_read_info info =3D { + .level =3D level, + .pattr =3D pattr, + }; + + smp_call_function_single(cpu, _a64fx_read_pfreg, &info, true); + if (info.ret < 0) + return info.ret; + + *val =3D info.val; + return 0; +} + +static void _a64fx_write_pfreg(void *info) +{ + u64 reg; + struct a64fx_write_info *winfo =3D info; + + reg =3D read_sysreg_s(A64FX_SDPF_IMP_PF_STREAM_DETECT_CTRL_EL0); + + winfo->ret =3D a64fx_modify_pfreg_val(winfo->pattr, ®, winfo->level, + winfo->val); + if (winfo->ret < 0) + return; + + write_sysreg_s(reg, A64FX_SDPF_IMP_PF_STREAM_DETECT_CTRL_EL0); + + winfo->ret =3D 0; +} + +static int a64fx_write_pfreg(enum pfctl_attr pattr, unsigned int cpu, + unsigned int level, u64 val) +{ + struct a64fx_write_info info =3D { + .level =3D level, + .pattr =3D pattr, + .val =3D val, + }; + + smp_call_function_single(cpu, _a64fx_write_pfreg, &info, true); + return info.ret; +} + +/* + * This driver returns a negative value if it does not support the Hardware + * Prefetch Control or if it is running on a VM guest. + */ +static int __init setup_pfctl_driver_params(void) +{ + unsigned long implementor =3D read_cpuid_implementor(); + unsigned long part_number =3D read_cpuid_part_number(); + + if (!is_kernel_in_hyp_mode()) + return -EINVAL; + + if (implementor !=3D ARM_CPU_IMP_FUJITSU) + return -ENODEV; + + switch (part_number) { + case FUJITSU_CPU_PART_A64FX: + /* A64FX register requires EL2 access */ + if (!has_vhe()) + return -EINVAL; + + a64fx_pfctl_driver.supported_l1d_prefetcher =3D SDPF; + a64fx_pfctl_driver.supported_l2_prefetcher =3D SDPF; + a64fx_pfctl_driver.supported_sdpf_strength =3D + STRENGTH_STRONG|STRENGTH_WEAK; + a64fx_pfctl_driver.read_pfreg =3D a64fx_read_pfreg; + a64fx_pfctl_driver.write_pfreg =3D a64fx_write_pfreg; + break; + default: + return -ENODEV; + } + + return 0; +} + +static int __init a64fx_pfctl_init(void) +{ + int ret; + + ret =3D setup_pfctl_driver_params(); + if (ret < 0) + return ret; + + ret =3D pfctl_register_driver(&a64fx_pfctl_driver); + if (ret < 0) + return ret; + + return 0; +} + +static void __exit a64fx_pfctl_exit(void) +{ + pfctl_unregister_driver(&a64fx_pfctl_driver); +} + +late_initcall(a64fx_pfctl_init); +module_exit(a64fx_pfctl_exit); + +MODULE_LICENSE("GPL"); +MODULE_AUTHOR("FUJITSU LIMITED"); +MODULE_DESCRIPTION("A64FX Hardware Prefetch Control Driver"); --=20 2.27.0 From nobody Mon May 11 00:51:30 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 68254C433F5 for ; Wed, 20 Apr 2022 03:03:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1359128AbiDTDGa (ORCPT ); Tue, 19 Apr 2022 23:06:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35758 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1359099AbiDTDGP (ORCPT ); Tue, 19 Apr 2022 23:06:15 -0400 X-Greylist: delayed 63 seconds by postgrey-1.37 at lindbergh.monkeyblade.net; Tue, 19 Apr 2022 20:03:30 PDT Received: from esa1.hc1455-7.c3s2.iphmx.com (esa1.hc1455-7.c3s2.iphmx.com [207.54.90.47]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DE69639699 for ; Tue, 19 Apr 2022 20:03:30 -0700 (PDT) X-IronPort-AV: E=McAfee;i="6400,9594,10322"; a="70545362" X-IronPort-AV: E=Sophos;i="5.90,274,1643641200"; d="scan'208";a="70545362" Received: from unknown (HELO oym-r4.gw.nic.fujitsu.com) ([210.162.30.92]) by esa1.hc1455-7.c3s2.iphmx.com with ESMTP; 20 Apr 2022 12:02:25 +0900 Received: from oym-m4.gw.nic.fujitsu.com (oym-nat-oym-m4.gw.nic.fujitsu.com [192.168.87.61]) by oym-r4.gw.nic.fujitsu.com (Postfix) with ESMTP id A1FA2E07EA for ; Wed, 20 Apr 2022 12:02:24 +0900 (JST) Received: from yto-om1.fujitsu.com (yto-om1.o.css.fujitsu.com [10.128.89.162]) by oym-m4.gw.nic.fujitsu.com (Postfix) with ESMTP id 5C6C2104546 for ; Wed, 20 Apr 2022 12:02:23 +0900 (JST) Received: from cn-r05-10.example.com (n3235113.np.ts.nmh.cs.fujitsu.co.jp [10.123.235.113]) by yto-om1.fujitsu.com (Postfix) with ESMTP id D882E40618A82; Wed, 20 Apr 2022 12:02:21 +0900 (JST) From: Kohei Tarumizu To: catalin.marinas@arm.com, will@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, fenghua.yu@intel.com, reinette.chatre@intel.com Cc: tarumizu.kohei@fujitsu.com Subject: [PATCH v3 4/9] soc: fujitsu: Add Kconfig/Makefile to build hardware prefetch control driver Date: Wed, 20 Apr 2022 12:02:18 +0900 Message-Id: <20220420030223.689259-5-tarumizu.kohei@fujitsu.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220420030223.689259-1-tarumizu.kohei@fujitsu.com> References: <20220420030223.689259-1-tarumizu.kohei@fujitsu.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This adds Kconfig/Makefile to build hardware prefetch control driver for A64FX support. This also adds a MAINTAINERS entry. Signed-off-by: Kohei Tarumizu --- MAINTAINERS | 1 + drivers/soc/Kconfig | 1 + drivers/soc/Makefile | 1 + drivers/soc/fujitsu/Kconfig | 11 +++++++++++ drivers/soc/fujitsu/Makefile | 2 ++ 5 files changed, 16 insertions(+) create mode 100644 drivers/soc/fujitsu/Kconfig create mode 100644 drivers/soc/fujitsu/Makefile diff --git a/MAINTAINERS b/MAINTAINERS index f6640dc053c0..b359dcc38be3 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -8619,6 +8619,7 @@ HARDWARE PREFETCH CONTROL DRIVERS M: Kohei Tarumizu S: Maintained F: drivers/base/pfctl.c +F: drivers/soc/fujitsu/a64fx-pfctl.c F: include/linux/pfctl.h =20 HARDWARE RANDOM NUMBER GENERATOR CORE diff --git a/drivers/soc/Kconfig b/drivers/soc/Kconfig index c5aae42673d3..d87754799d90 100644 --- a/drivers/soc/Kconfig +++ b/drivers/soc/Kconfig @@ -9,6 +9,7 @@ source "drivers/soc/atmel/Kconfig" source "drivers/soc/bcm/Kconfig" source "drivers/soc/canaan/Kconfig" source "drivers/soc/fsl/Kconfig" +source "drivers/soc/fujitsu/Kconfig" source "drivers/soc/imx/Kconfig" source "drivers/soc/ixp4xx/Kconfig" source "drivers/soc/litex/Kconfig" diff --git a/drivers/soc/Makefile b/drivers/soc/Makefile index 904eec2a7871..6c8ff1792cda 100644 --- a/drivers/soc/Makefile +++ b/drivers/soc/Makefile @@ -12,6 +12,7 @@ obj-$(CONFIG_SOC_CANAAN) +=3D canaan/ obj-$(CONFIG_ARCH_DOVE) +=3D dove/ obj-$(CONFIG_MACH_DOVE) +=3D dove/ obj-y +=3D fsl/ +obj-y +=3D fujitsu/ obj-$(CONFIG_ARCH_GEMINI) +=3D gemini/ obj-y +=3D imx/ obj-y +=3D ixp4xx/ diff --git a/drivers/soc/fujitsu/Kconfig b/drivers/soc/fujitsu/Kconfig new file mode 100644 index 000000000000..d9db05d5055d --- /dev/null +++ b/drivers/soc/fujitsu/Kconfig @@ -0,0 +1,11 @@ +# SPDX-License-Identifier: GPL-2.0-only + +menu "Fujitsu SoC drivers" + +config A64FX_HWPF_CONTROL + tristate "A64FX Hardware Prefetch Control driver" + depends on ARM64 || HWPF_CONTROL + help + This adds Hardware Prefetch driver control support for A64FX. + +endmenu diff --git a/drivers/soc/fujitsu/Makefile b/drivers/soc/fujitsu/Makefile new file mode 100644 index 000000000000..35e284a548bb --- /dev/null +++ b/drivers/soc/fujitsu/Makefile @@ -0,0 +1,2 @@ +# SPDX-License-Identifier: GPL-2.0-only +obj-$(CONFIG_A64FX_HWPF_CONTROL) +=3D a64fx-pfctl.o --=20 2.27.0 From nobody Mon May 11 00:51:30 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 045EFC433EF for ; Wed, 20 Apr 2022 03:03:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1359106AbiDTDGe (ORCPT ); Tue, 19 Apr 2022 23:06:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35766 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1359100AbiDTDGQ (ORCPT ); Tue, 19 Apr 2022 23:06:16 -0400 Received: from esa8.hc1455-7.c3s2.iphmx.com (esa8.hc1455-7.c3s2.iphmx.com [139.138.61.253]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F2D8E396AA for ; Tue, 19 Apr 2022 20:03:30 -0700 (PDT) X-IronPort-AV: E=McAfee;i="6400,9594,10322"; a="58439831" X-IronPort-AV: E=Sophos;i="5.90,274,1643641200"; d="scan'208";a="58439831" Received: from unknown (HELO oym-r2.gw.nic.fujitsu.com) ([210.162.30.90]) by esa8.hc1455-7.c3s2.iphmx.com with ESMTP; 20 Apr 2022 12:02:25 +0900 Received: from oym-m1.gw.nic.fujitsu.com (oym-nat-oym-m1.gw.nic.fujitsu.com [192.168.87.58]) by oym-r2.gw.nic.fujitsu.com (Postfix) with ESMTP id 10211DB9F6 for ; Wed, 20 Apr 2022 12:02:25 +0900 (JST) Received: from yto-om1.fujitsu.com (yto-om1.o.css.fujitsu.com [10.128.89.162]) by oym-m1.gw.nic.fujitsu.com (Postfix) with ESMTP id 3B68FD9C6A for ; Wed, 20 Apr 2022 12:02:24 +0900 (JST) Received: from cn-r05-10.example.com (n3235113.np.ts.nmh.cs.fujitsu.co.jp [10.123.235.113]) by yto-om1.fujitsu.com (Postfix) with ESMTP id AB4EE404AFD51; Wed, 20 Apr 2022 12:02:23 +0900 (JST) From: Kohei Tarumizu To: catalin.marinas@arm.com, will@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, fenghua.yu@intel.com, reinette.chatre@intel.com Cc: tarumizu.kohei@fujitsu.com Subject: [PATCH v3 5/9] arm64: Create cache sysfs directory without ACPI PPTT for hardware prefetch control Date: Wed, 20 Apr 2022 12:02:19 +0900 Message-Id: <20220420030223.689259-6-tarumizu.kohei@fujitsu.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220420030223.689259-1-tarumizu.kohei@fujitsu.com> References: <20220420030223.689259-1-tarumizu.kohei@fujitsu.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This patch create a cache sysfs directory without ACPI PPTT if the CONFIG_HWPF_CONTROL is true. This patch use only the level/type information obtained from CLIDR_EL1, and don't use CCSIDR information. Hardware prefetch control driver need cache sysfs directory and cache level/type information. In ARM processor, these information can be obtained from the register even without PPTT. Therefore, we set the cpu_map_populated to true to create cache sysfs directory if the machine doesn't have PPTT. Signed-off-by: Kohei Tarumizu --- arch/arm64/kernel/cacheinfo.c | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/arch/arm64/kernel/cacheinfo.c b/arch/arm64/kernel/cacheinfo.c index 587543c6c51c..039ec32d0b3d 100644 --- a/arch/arm64/kernel/cacheinfo.c +++ b/arch/arm64/kernel/cacheinfo.c @@ -43,6 +43,21 @@ static void ci_leaf_init(struct cacheinfo *this_leaf, this_leaf->type =3D type; } =20 +#if defined(CONFIG_HWPF_CONTROL) +static bool acpi_has_pptt(void) +{ + struct acpi_table_header *table; + acpi_status status; + + status =3D acpi_get_table(ACPI_SIG_PPTT, 0, &table); + if (ACPI_FAILURE(status)) + return false; + + acpi_put_table(table); + return true; +} +#endif + int init_cache_level(unsigned int cpu) { unsigned int ctype, level, leaves, fw_level; @@ -95,5 +110,19 @@ int populate_cache_leaves(unsigned int cpu) ci_leaf_init(this_leaf++, type, level); } } + +#if defined(CONFIG_HWPF_CONTROL) + /* + * Hardware prefetch functions need cache sysfs directory and cache + * level/type information. In ARM processor, these information can be + * obtained from registers even without PPTT. Therefore, we set the + * cpu_map_populated to true to create cache sysfs directory, if the + * machine doesn't have PPTT. + **/ + if (!acpi_disabled) + if (!acpi_has_pptt()) + this_cpu_ci->cpu_map_populated =3D true; +#endif + return 0; } --=20 2.27.0 From nobody Mon May 11 00:51:30 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21AEAC433EF for ; Wed, 20 Apr 2022 03:03:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1359114AbiDTDGi (ORCPT ); Tue, 19 Apr 2022 23:06:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35790 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1359102AbiDTDGQ (ORCPT ); Tue, 19 Apr 2022 23:06:16 -0400 Received: from esa2.hc1455-7.c3s2.iphmx.com (esa2.hc1455-7.c3s2.iphmx.com [207.54.90.48]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A8E7939172 for ; Tue, 19 Apr 2022 20:03:31 -0700 (PDT) X-IronPort-AV: E=McAfee;i="6400,9594,10322"; a="70387731" X-IronPort-AV: E=Sophos;i="5.90,274,1643641200"; d="scan'208";a="70387731" Received: from unknown (HELO oym-r1.gw.nic.fujitsu.com) ([210.162.30.89]) by esa2.hc1455-7.c3s2.iphmx.com with ESMTP; 20 Apr 2022 12:02:28 +0900 Received: from oym-m3.gw.nic.fujitsu.com (oym-nat-oym-m3.gw.nic.fujitsu.com [192.168.87.60]) by oym-r1.gw.nic.fujitsu.com (Postfix) with ESMTP id DDD36575A4 for ; Wed, 20 Apr 2022 12:02:26 +0900 (JST) Received: from yto-om1.fujitsu.com (yto-om1.o.css.fujitsu.com [10.128.89.162]) by oym-m3.gw.nic.fujitsu.com (Postfix) with ESMTP id 01906D95F2 for ; Wed, 20 Apr 2022 12:02:26 +0900 (JST) Received: from cn-r05-10.example.com (n3235113.np.ts.nmh.cs.fujitsu.co.jp [10.123.235.113]) by yto-om1.fujitsu.com (Postfix) with ESMTP id 7A6F24060C93E; Wed, 20 Apr 2022 12:02:25 +0900 (JST) From: Kohei Tarumizu To: catalin.marinas@arm.com, will@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, fenghua.yu@intel.com, reinette.chatre@intel.com Cc: tarumizu.kohei@fujitsu.com Subject: [PATCH v3 6/9] x86: resctrl: pseudo_lock: Fix to restore to original value when re-enabling hardware prefetch register Date: Wed, 20 Apr 2022 12:02:20 +0900 Message-Id: <20220420030223.689259-7-tarumizu.kohei@fujitsu.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220420030223.689259-1-tarumizu.kohei@fujitsu.com> References: <20220420030223.689259-1-tarumizu.kohei@fujitsu.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The current pseudo_lock.c code overwrittes the value of the MSR_MISC_FEATURE_CONTROL to 0 even if the original value is not 0. Therefore, modify it to save and restore the original values. Signed-off-by: Kohei Tarumizu --- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c b/arch/x86/kernel/cp= u/resctrl/pseudo_lock.c index db813f819ad6..2d713c20f55f 100644 --- a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c +++ b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c @@ -420,6 +420,7 @@ static int pseudo_lock_fn(void *_rdtgrp) struct pseudo_lock_region *plr =3D rdtgrp->plr; u32 rmid_p, closid_p; unsigned long i; + u64 saved_msr; #ifdef CONFIG_KASAN /* * The registers used for local register variables are also used @@ -463,6 +464,7 @@ static int pseudo_lock_fn(void *_rdtgrp) * the buffer and evict pseudo-locked memory read earlier from the * cache. */ + saved_msr =3D __rdmsr(MSR_MISC_FEATURE_CONTROL); __wrmsr(MSR_MISC_FEATURE_CONTROL, prefetch_disable_bits, 0x0); closid_p =3D this_cpu_read(pqr_state.cur_closid); rmid_p =3D this_cpu_read(pqr_state.cur_rmid); @@ -514,7 +516,7 @@ static int pseudo_lock_fn(void *_rdtgrp) __wrmsr(IA32_PQR_ASSOC, rmid_p, closid_p); =20 /* Re-enable the hardware prefetcher(s) */ - wrmsr(MSR_MISC_FEATURE_CONTROL, 0x0, 0x0); + wrmsrl(MSR_MISC_FEATURE_CONTROL, saved_msr); local_irq_enable(); =20 plr->thread_done =3D 1; @@ -873,12 +875,14 @@ static int measure_cycles_lat_fn(void *_plr) struct pseudo_lock_region *plr =3D _plr; unsigned long i; u64 start, end; + u32 saved_low, saved_high; void *mem_r; =20 local_irq_disable(); /* * Disable hardware prefetchers. */ + rdmsr(MSR_MISC_FEATURE_CONTROL, saved_low, saved_high); wrmsr(MSR_MISC_FEATURE_CONTROL, prefetch_disable_bits, 0x0); mem_r =3D READ_ONCE(plr->kmem); /* @@ -895,7 +899,7 @@ static int measure_cycles_lat_fn(void *_plr) end =3D rdtsc_ordered(); trace_pseudo_lock_mem_latency((u32)(end - start)); } - wrmsr(MSR_MISC_FEATURE_CONTROL, 0x0, 0x0); + wrmsr(MSR_MISC_FEATURE_CONTROL, saved_low, saved_high); local_irq_enable(); plr->thread_done =3D 1; wake_up_interruptible(&plr->lock_thread_wq); @@ -945,6 +949,7 @@ static int measure_residency_fn(struct perf_event_attr = *miss_attr, unsigned long i; void *mem_r; u64 tmp; + u32 saved_low, saved_high; =20 miss_event =3D perf_event_create_kernel_counter(miss_attr, plr->cpu, NULL, NULL, NULL); @@ -973,6 +978,7 @@ static int measure_residency_fn(struct perf_event_attr = *miss_attr, /* * Disable hardware prefetchers. */ + rdmsr(MSR_MISC_FEATURE_CONTROL, saved_low, saved_high); wrmsr(MSR_MISC_FEATURE_CONTROL, prefetch_disable_bits, 0x0); =20 /* Initialize rest of local variables */ @@ -1031,7 +1037,7 @@ static int measure_residency_fn(struct perf_event_att= r *miss_attr, */ rmb(); /* Re-enable hardware prefetchers */ - wrmsr(MSR_MISC_FEATURE_CONTROL, 0x0, 0x0); + wrmsr(MSR_MISC_FEATURE_CONTROL, saved_low, saved_high); local_irq_enable(); out_hit: perf_event_release_kernel(hit_event); --=20 2.27.0 From nobody Mon May 11 00:51:30 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE322C433F5 for ; Wed, 20 Apr 2022 03:04:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1359148AbiDTDGw (ORCPT ); Tue, 19 Apr 2022 23:06:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36206 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1359050AbiDTDGb (ORCPT ); Tue, 19 Apr 2022 23:06:31 -0400 Received: from esa9.hc1455-7.c3s2.iphmx.com (esa9.hc1455-7.c3s2.iphmx.com [139.138.36.223]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C488B393E3 for ; Tue, 19 Apr 2022 20:03:35 -0700 (PDT) X-IronPort-AV: E=McAfee;i="6400,9594,10322"; a="58535653" X-IronPort-AV: E=Sophos;i="5.90,274,1643641200"; d="scan'208";a="58535653" Received: from unknown (HELO yto-r2.gw.nic.fujitsu.com) ([218.44.52.218]) by esa9.hc1455-7.c3s2.iphmx.com with ESMTP; 20 Apr 2022 12:02:30 +0900 Received: from yto-m3.gw.nic.fujitsu.com (yto-nat-yto-m3.gw.nic.fujitsu.com [192.168.83.66]) by yto-r2.gw.nic.fujitsu.com (Postfix) with ESMTP id C6586C68BD for ; Wed, 20 Apr 2022 12:02:28 +0900 (JST) Received: from yto-om1.fujitsu.com (yto-om1.o.css.fujitsu.com [10.128.89.162]) by yto-m3.gw.nic.fujitsu.com (Postfix) with ESMTP id 91D3BD9778 for ; Wed, 20 Apr 2022 12:02:27 +0900 (JST) Received: from cn-r05-10.example.com (n3235113.np.ts.nmh.cs.fujitsu.co.jp [10.123.235.113]) by yto-om1.fujitsu.com (Postfix) with ESMTP id 45D434060E7D0; Wed, 20 Apr 2022 12:02:27 +0900 (JST) From: Kohei Tarumizu To: catalin.marinas@arm.com, will@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, fenghua.yu@intel.com, reinette.chatre@intel.com Cc: tarumizu.kohei@fujitsu.com Subject: [PATCH v3 7/9] x86: Add hardware prefetch control support for x86 Date: Wed, 20 Apr 2022 12:02:21 +0900 Message-Id: <20220420030223.689259-8-tarumizu.kohei@fujitsu.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220420030223.689259-1-tarumizu.kohei@fujitsu.com> References: <20220420030223.689259-1-tarumizu.kohei@fujitsu.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This adds module init/exit code, and creates sysfs attribute file "hardware_prefetcher_enable", "ip_prefetcher_enable" and "adjacent_cache_line_prefetcher_enable" for x86. This driver works only if the model is INTEL_FAM6_BROADWELL_X at this point. If you would like to support a new model with the same register specifications as INTEL_FAM6_BROADWELL_X, it is possible to add the model settings to array of broadwell_cpu_ids[]. The details of the registers to be read and written in this patch are described below: "https://www.intel.com/content/www/us/en/developer/articles/technical/intel= -sdm.html" Volume 4 Signed-off-by: Kohei Tarumizu --- arch/x86/kernel/cpu/x86-pfctl.c | 347 ++++++++++++++++++++++++++++++++ 1 file changed, 347 insertions(+) create mode 100644 arch/x86/kernel/cpu/x86-pfctl.c diff --git a/arch/x86/kernel/cpu/x86-pfctl.c b/arch/x86/kernel/cpu/x86-pfct= l.c new file mode 100644 index 000000000000..153b7a46ba80 --- /dev/null +++ b/arch/x86/kernel/cpu/x86-pfctl.c @@ -0,0 +1,347 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright 2022 FUJITSU LIMITED + * + * x86 Hardware Prefetch Control support + */ + +#include +#include +#include +#include +#include +#include +#include +#include + +struct pfctl_driver x86_pfctl_driver; + +/************************************** + * Intle BROADWELL support + **************************************/ + +/* + * The register specification for each bits of Intel BROADWELL is as + * follow: + * + * [0] L2 Hardware Prefetcher Disable (R/W) + * [1] L2 Adjacent Cache Line Prefetcher Disable (R/W) + * [2] DCU Hardware Prefetcher Disable (R/W) + * [3] DCU IP Prefetcher Disable (R/W) + * [63:4] Reserved + * + * See "Intel 64 and IA-32 Architectures Software Developer's Manual" + * (https://www.intel.com/content/www/us/en/developer/articles/technical/i= ntel-sdm.html) + * for register specification details. + */ +#define BROADWELL_L2_HWPF_FIELD BIT_ULL(0) +#define BROADWELL_L2_ACLPF_FIELD BIT_ULL(1) +#define BROADWELL_DCU_HWPF_FIELD BIT_ULL(2) +#define BROADWELL_DCU_IPPF_FIELD BIT_ULL(3) + +struct broadwell_read_info { + enum pfctl_attr pattr; + u64 val; + unsigned int level; + int ret; +}; + +struct broadwell_write_info { + enum pfctl_attr pattr; + u64 val; + unsigned int level; + int ret; +}; + +static int broadwell_get_hwpf_enable(u64 reg, unsigned int level) +{ + u64 val; + + switch (level) { + case 1: + val =3D FIELD_GET(BROADWELL_DCU_HWPF_FIELD, reg); + break; + case 2: + val =3D FIELD_GET(BROADWELL_L2_HWPF_FIELD, reg); + break; + default: + return -EINVAL; + } + + if (val =3D=3D 0) + return PFCTL_ENABLE_VAL; + else if (val =3D=3D 1) + return PFCTL_DISABLE_VAL; + else + return -EINVAL; +} + +static int broadwell_modify_hwpf_enable(u64 *reg, unsigned int level, u64 = val) +{ + if (val =3D=3D PFCTL_ENABLE_VAL) + val =3D 0; + else + val =3D 1; + + switch (level) { + case 1: + *reg &=3D ~BROADWELL_DCU_HWPF_FIELD; + *reg |=3D FIELD_PREP(BROADWELL_DCU_HWPF_FIELD, val); + break; + case 2: + *reg &=3D ~BROADWELL_L2_HWPF_FIELD; + *reg |=3D FIELD_PREP(BROADWELL_L2_HWPF_FIELD, val); + break; + default: + return -EINVAL; + } + + return 0; +} + +static int broadwell_get_ippf_enable(u64 reg, unsigned int level) +{ + u64 val; + + switch (level) { + case 1: + val =3D FIELD_GET(BROADWELL_DCU_IPPF_FIELD, reg); + break; + default: + return -EINVAL; + } + + if (val =3D=3D 0) + return PFCTL_ENABLE_VAL; + else if (val =3D=3D 1) + return PFCTL_DISABLE_VAL; + else + return -EINVAL; +} + +static int broadwell_modify_ippf_enable(u64 *reg, unsigned int level, u64 = val) +{ + if (val =3D=3D PFCTL_ENABLE_VAL) + val =3D 0; + else + val =3D 1; + + switch (level) { + case 1: + *reg &=3D ~BROADWELL_DCU_IPPF_FIELD; + *reg |=3D FIELD_PREP(BROADWELL_DCU_IPPF_FIELD, val); + break; + default: + return -EINVAL; + } + + return 0; +} + +static int broadwell_get_aclpf_enable(u64 reg, unsigned int level) +{ + u64 val; + + switch (level) { + case 2: + val =3D FIELD_GET(BROADWELL_L2_ACLPF_FIELD, reg); + break; + default: + return -EINVAL; + } + + if (val =3D=3D 0) + return PFCTL_ENABLE_VAL; + else if (val =3D=3D 1) + return PFCTL_DISABLE_VAL; + else + return -EINVAL; +} + +static int broadwell_modify_aclpf_enable(u64 *reg, unsigned int level, u64= val) +{ + if (val =3D=3D PFCTL_ENABLE_VAL) + val =3D 0; + else + val =3D 1; + + switch (level) { + case 2: + *reg &=3D ~BROADWELL_L2_ACLPF_FIELD; + *reg |=3D FIELD_PREP(BROADWELL_L2_ACLPF_FIELD, val); + break; + default: + return -EINVAL; + } + + return 0; +} + +static int broadwell_get_pfctl_params(enum pfctl_attr pattr, u64 reg, + unsigned int level, u64 *val) +{ + int ret; + + switch (pattr) { + case HWPF_ENABLE: + ret =3D broadwell_get_hwpf_enable(reg, level); + break; + case IPPF_ENABLE: + ret =3D broadwell_get_ippf_enable(reg, level); + break; + case ACLPF_ENABLE: + ret =3D broadwell_get_aclpf_enable(reg, level); + break; + default: + return -ENOENT; + } + + if (ret < 0) + return ret; + *val =3D ret; + + return 0; +} + +static int broadwell_modify_pfreg(enum pfctl_attr pattr, u64 *reg, + unsigned int level, u64 val) +{ + int ret; + + switch (pattr) { + case HWPF_ENABLE: + ret =3D broadwell_modify_hwpf_enable(reg, level, val); + break; + case IPPF_ENABLE: + ret =3D broadwell_modify_ippf_enable(reg, level, val); + break; + case ACLPF_ENABLE: + ret =3D broadwell_modify_aclpf_enable(reg, level, val); + break; + default: + return -ENOENT; + } + + if (ret < 0) + return ret; + + return 0; +} + +static void _broadwell_read_pfreg(void *info) +{ + u64 reg; + struct broadwell_read_info *rinfo =3D info; + + rdmsrl(MSR_MISC_FEATURE_CONTROL, reg); + + rinfo->ret =3D broadwell_get_pfctl_params(rinfo->pattr, reg, rinfo->level, + &rinfo->val); + if (rinfo->ret < 0) + return; +} + +static int broadwell_read_pfreg(enum pfctl_attr pattr, unsigned int cpu, + unsigned int level, u64 *val) +{ + struct broadwell_read_info info =3D { + .level =3D level, + .pattr =3D pattr, + }; + + smp_call_function_single(cpu, _broadwell_read_pfreg, &info, true); + if (info.ret < 0) + return info.ret; + + *val =3D info.val; + return 0; +} + +static void _broadwell_write_pfreg(void *info) +{ + u64 reg; + struct broadwell_write_info *winfo =3D info; + + rdmsrl(MSR_MISC_FEATURE_CONTROL, reg); + + winfo->ret =3D broadwell_modify_pfreg(winfo->pattr, ®, winfo->level, + winfo->val); + if (winfo->ret < 0) + return; + + wrmsrl(MSR_MISC_FEATURE_CONTROL, reg); +} + +static int broadwell_write_pfreg(enum pfctl_attr pattr, unsigned int cpu, + unsigned int level, u64 val) +{ + struct broadwell_write_info info =3D { + .level =3D level, + .pattr =3D pattr, + .val =3D val, + }; + + smp_call_function_single(cpu, _broadwell_write_pfreg, &info, true); + return info.ret; +} + +/* + * In addition to BROADWELL_X, NEHALEM and others have same register + * specifications as those represented by BROADWELL_XXX_FIELD. + * If you want to add support for these processor, add the new target model + * here. + */ +static const struct x86_cpu_id broadwell_cpu_ids[] =3D { + X86_MATCH_INTEL_FAM6_MODEL(BROADWELL_X, NULL), + {} +}; + +/***** end of Intel BROADWELL support *****/ + +/* + * This driver returns a negative value if it does not support the Hardware + * Prefetch Control or if it is running on a VM guest. + */ +static int __init setup_pfctl_driver_params(void) +{ + if (boot_cpu_has(X86_FEATURE_HYPERVISOR)) + return -EINVAL; + + if (x86_match_cpu(broadwell_cpu_ids)) { + x86_pfctl_driver.supported_l1d_prefetcher =3D HWPF|IPPF; + x86_pfctl_driver.supported_l2_prefetcher =3D HWPF|ACLPF; + x86_pfctl_driver.read_pfreg =3D broadwell_read_pfreg; + x86_pfctl_driver.write_pfreg =3D broadwell_write_pfreg; + } else { + return -ENODEV; + } + + return 0; +} + +static int __init x86_pfctl_init(void) +{ + int ret; + + ret =3D setup_pfctl_driver_params(); + if (ret < 0) + return ret; + + ret =3D pfctl_register_driver(&x86_pfctl_driver); + if (ret < 0) + return ret; + + return 0; +} + +static void __exit x86_pfctl_exit(void) +{ + pfctl_unregister_driver(&x86_pfctl_driver); +} + +late_initcall(x86_pfctl_init); +module_exit(x86_pfctl_exit); + +MODULE_LICENSE("GPL"); +MODULE_AUTHOR("FUJITSU LIMITED"); +MODULE_DESCRIPTION("x86 Hardware Prefetch Control Driver"); --=20 2.27.0 From nobody Mon May 11 00:51:30 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8E2EEC433EF for ; Wed, 20 Apr 2022 03:04:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1359173AbiDTDGr (ORCPT ); Tue, 19 Apr 2022 23:06:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35900 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1359111AbiDTDGT (ORCPT ); Tue, 19 Apr 2022 23:06:19 -0400 Received: from esa2.hc1455-7.c3s2.iphmx.com (esa2.hc1455-7.c3s2.iphmx.com [207.54.90.48]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 85DFB393FA for ; Tue, 19 Apr 2022 20:03:33 -0700 (PDT) X-IronPort-AV: E=McAfee;i="6400,9594,10322"; a="70387751" X-IronPort-AV: E=Sophos;i="5.90,274,1643641200"; d="scan'208";a="70387751" Received: from unknown (HELO oym-r2.gw.nic.fujitsu.com) ([210.162.30.90]) by esa2.hc1455-7.c3s2.iphmx.com with ESMTP; 20 Apr 2022 12:02:31 +0900 Received: from oym-m3.gw.nic.fujitsu.com (oym-nat-oym-m3.gw.nic.fujitsu.com [192.168.87.60]) by oym-r2.gw.nic.fujitsu.com (Postfix) with ESMTP id 20974DB9F5 for ; Wed, 20 Apr 2022 12:02:30 +0900 (JST) Received: from yto-om1.fujitsu.com (yto-om1.o.css.fujitsu.com [10.128.89.162]) by oym-m3.gw.nic.fujitsu.com (Postfix) with ESMTP id 4B28FD95F6 for ; Wed, 20 Apr 2022 12:02:29 +0900 (JST) Received: from cn-r05-10.example.com (n3235113.np.ts.nmh.cs.fujitsu.co.jp [10.123.235.113]) by yto-om1.fujitsu.com (Postfix) with ESMTP id EBFCB4060ED87; Wed, 20 Apr 2022 12:02:28 +0900 (JST) From: Kohei Tarumizu To: catalin.marinas@arm.com, will@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, fenghua.yu@intel.com, reinette.chatre@intel.com Cc: tarumizu.kohei@fujitsu.com Subject: [PATCH v3 8/9] x86: Add Kconfig/Makefile to build hardware prefetch control driver Date: Wed, 20 Apr 2022 12:02:22 +0900 Message-Id: <20220420030223.689259-9-tarumizu.kohei@fujitsu.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220420030223.689259-1-tarumizu.kohei@fujitsu.com> References: <20220420030223.689259-1-tarumizu.kohei@fujitsu.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This adds Kconfig/Makefile to build hardware prefetch control driver for x86 support. This also adds a MAINTAINERS entry. Signed-off-by: Kohei Tarumizu --- MAINTAINERS | 1 + arch/x86/Kconfig | 6 ++++++ arch/x86/kernel/cpu/Makefile | 2 ++ 3 files changed, 9 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index b359dcc38be3..4de219599e52 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -8618,6 +8618,7 @@ K: (devm_)?hwmon_device_(un)?register(|_with_groups|_= with_info) HARDWARE PREFETCH CONTROL DRIVERS M: Kohei Tarumizu S: Maintained +F: arch/x86/kernel/cpu/x86-pfctl.c F: drivers/base/pfctl.c F: drivers/soc/fujitsu/a64fx-pfctl.c F: include/linux/pfctl.h diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index b0142e01002e..1ef47c29c338 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -1359,6 +1359,12 @@ config X86_CPUID with major 203 and minors 0 to 31 for /dev/cpu/0/cpuid to /dev/cpu/31/cpuid. =20 +config X86_HWPF_CONTROL + tristate "x86 Hardware Prefetch Control support" + depends on HWPF_CONTROL + help + This adds Hardware Prefetch driver control support for X86. + choice prompt "High Memory Support" default HIGHMEM4G diff --git a/arch/x86/kernel/cpu/Makefile b/arch/x86/kernel/cpu/Makefile index 9661e3e802be..1aa13dad17a3 100644 --- a/arch/x86/kernel/cpu/Makefile +++ b/arch/x86/kernel/cpu/Makefile @@ -56,6 +56,8 @@ obj-$(CONFIG_X86_LOCAL_APIC) +=3D perfctr-watchdog.o obj-$(CONFIG_HYPERVISOR_GUEST) +=3D vmware.o hypervisor.o mshyperv.o obj-$(CONFIG_ACRN_GUEST) +=3D acrn.o =20 +obj-$(CONFIG_X86_HWPF_CONTROL) +=3D x86-pfctl.o + ifdef CONFIG_X86_FEATURE_NAMES quiet_cmd_mkcapflags =3D MKCAP $@ cmd_mkcapflags =3D $(CONFIG_SHELL) $(srctree)/$(src)/mkcapflags.sh $= @ $^ --=20 2.27.0 From nobody Mon May 11 00:51:30 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3B750C433EF for ; Wed, 20 Apr 2022 03:04:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233642AbiDTDHA (ORCPT ); Tue, 19 Apr 2022 23:07:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35766 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1359153AbiDTDGg (ORCPT ); Tue, 19 Apr 2022 23:06:36 -0400 X-Greylist: delayed 66 seconds by postgrey-1.37 at lindbergh.monkeyblade.net; Tue, 19 Apr 2022 20:03:41 PDT Received: from esa12.hc1455-7.c3s2.iphmx.com (esa12.hc1455-7.c3s2.iphmx.com [139.138.37.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A84233A73B for ; Tue, 19 Apr 2022 20:03:41 -0700 (PDT) X-IronPort-AV: E=McAfee;i="6400,9594,10322"; a="50159653" X-IronPort-AV: E=Sophos;i="5.90,274,1643641200"; d="scan'208";a="50159653" Received: from unknown (HELO yto-r4.gw.nic.fujitsu.com) ([218.44.52.220]) by esa12.hc1455-7.c3s2.iphmx.com with ESMTP; 20 Apr 2022 12:02:31 +0900 Received: from yto-m3.gw.nic.fujitsu.com (yto-nat-yto-m3.gw.nic.fujitsu.com [192.168.83.66]) by yto-r4.gw.nic.fujitsu.com (Postfix) with ESMTP id 5ED49C9454 for ; Wed, 20 Apr 2022 12:02:32 +0900 (JST) Received: from yto-om1.fujitsu.com (yto-om1.o.css.fujitsu.com [10.128.89.162]) by yto-m3.gw.nic.fujitsu.com (Postfix) with ESMTP id 41A1CD976C for ; Wed, 20 Apr 2022 12:02:31 +0900 (JST) Received: from cn-r05-10.example.com (n3235113.np.ts.nmh.cs.fujitsu.co.jp [10.123.235.113]) by yto-om1.fujitsu.com (Postfix) with ESMTP id F233C4060E7D0; Wed, 20 Apr 2022 12:02:30 +0900 (JST) From: Kohei Tarumizu To: catalin.marinas@arm.com, will@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, fenghua.yu@intel.com, reinette.chatre@intel.com Cc: tarumizu.kohei@fujitsu.com Subject: [PATCH v3 9/9] docs: ABI: Add sysfs documentation interface of hardware prefetch control driver Date: Wed, 20 Apr 2022 12:02:23 +0900 Message-Id: <20220420030223.689259-10-tarumizu.kohei@fujitsu.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220420030223.689259-1-tarumizu.kohei@fujitsu.com> References: <20220420030223.689259-1-tarumizu.kohei@fujitsu.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This describes the sysfs interface implemented by the hardware prefetch control driver. Signed-off-by: Kohei Tarumizu --- .../ABI/testing/sysfs-devices-system-cpu | 98 +++++++++++++++++++ 1 file changed, 98 insertions(+) diff --git a/Documentation/ABI/testing/sysfs-devices-system-cpu b/Documenta= tion/ABI/testing/sysfs-devices-system-cpu index 2ad01cad7f1c..0da4c1bac51e 100644 --- a/Documentation/ABI/testing/sysfs-devices-system-cpu +++ b/Documentation/ABI/testing/sysfs-devices-system-cpu @@ -688,3 +688,101 @@ Description: (RO) the list of CPUs that are isolated and don't participate in load balancing. These CPUs are set by boot parameter "isolcpus=3D". + +What: /sys/devices/system/cpu/cpu*/cache/index*/prefetch_control/hardware= _prefetcher_enable + /sys/devices/system/cpu/cpu*/cache/index*/prefetch_control/ip_prefetcher= _enable + /sys/devices/system/cpu/cpu*/cache/index*/prefetch_control/adjacent_cach= e_line_prefetcher_enable +Date: March 2022 +Contact: Linux kernel mailing list +Description: Parameters for some Intel CPU's hardware prefetch control + + This sysfs interface provides Hardware Prefetch control + attribute for some Intel processors. Attributes are only + present if the particular cache implements the relevant + prefetcher controls. + + *_prefetcher_enable: + (RW) control this prefetcher's enablement state. + Read returns current status: + 0: this prefetcher is disabled + 1: this prefetcher is enabled + + - Attribute mapping + + Some Intel processors have MSR 0x1a4. This register has several + specifications depending on the model. This interface provides + a one-to-one attribute file to control all the tunable + parameters the CPU provides of the following. + + - "* Hardware Prefetcher Disable (R/W)" + corresponds to the "hardware_prefetcher_enable" + + - "* Adjacent Cache Line Prefetcher Disable (R/W)" + corresponds to the "adjacent_cache_line_prefetcher_enable" + + - "* IP Prefetcher Disable (R/W)" + corresponds to the "ip_prefetcher_enable" + +What: /sys/devices/system/cpu/cpu*/cache/index*/prefetch_control/stream_d= etect_prefetcher_enable + /sys/devices/system/cpu/cpu*/cache/index*/prefetch_control/stream_detect= _prefetcher_strength + /sys/devices/system/cpu/cpu*/cache/index*/prefetch_control/stream_detect= _prefetcher_strength_available + /sys/devices/system/cpu/cpu*/cache/index*/prefetch_control/stream_detect= _prefetcher_dist +Date: March 2022 +Contact: Linux kernel mailing list +Description: Parameters for A64FX's hardware prefetch control + + This sysfs interface provides Hardware Prefetch control + attribute for the processor A64FX. Attributes are only + present if the particular cache implements the relevant + prefetcher controls. + + stream_detect_prefetcher_enable: + (RW) control the prefetcher's enablement state. + Read returns current status: + 0: this prefetcher is disabled + 1: this prefetcher is enabled + + stream_detect_prefetcher_strength: + (RW) control the prefetcher operation's strongness state. + Read returns current status: + weak: prefetch operation is weak + strong: prefetch operation is strong + + Strong prefetch operation is surely executed, if there is + no corresponding data in cache. + Weak prefetch operation allows the hardware not to execute + operation depending on hardware state. + + + stream_detect_prefetcher_strength_available: + (RO) displays a space separated list of available strongness + state. + + stream_detect_prefetcher_dist: + (RW) control the prefetcher distance value. + Read return current prefetcher distance value in bytes + or the string "auto". + + Write either a value in byte or the string "auto" to this + parameter. If you write a value less than multiples of a + specific value, it is rounded up. + + The string "auto" have a special meaning. This means that + instead of setting dist to a user-specified value, it + operates using hardware-specific values. + + - Attribute mapping + + The processor A64FX has register IMP_PF_STREAM_DETECT_CTRL_EL0 + for Hardware Prefetch Control. This attribute maps each + specification to the following. + + - "L*PF_DIS": enablement of hardware prefetcher + corresponds to the "stream_detect_prefetcher_enable" + + - "L*W": strongness of hardware prefetcher + corresponds to "stream_detect_prefetcher_strength" + and "stream_detect_prefetcher_strength_available" + + - "L*_DIST": distance of hardware prefetcher + corresponds to the "stream_detect_prefetcher_dist" --=20 2.27.0