From nobody Wed Dec 17 13:54:13 2025 Received: from mail-pg1-f169.google.com (mail-pg1-f169.google.com [209.85.215.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E0AF7EAD4 for ; Fri, 14 Jun 2024 14:22:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718374933; cv=none; b=S10avYxg0oiqFLOOXHJySgwWPTkRsFMxd37aFks+jljLyIfn0i+JAXT42IyTSl5/3YtTS55b0KO7Z4hCV9ihNjNMirkgSTp6l2BXTAbOEeNKYn5TfIDTGb7jJHgXAWONF5w7/NVsowk5bavZIffRO0y5HksUEPoA19K7uT7Z4SU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718374933; c=relaxed/simple; bh=UT3jwponDbjl5OONR3Oj3tAS9bDmUperLc/haaGrdAo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=QnWT8V1EPetOSPuVvccbkuWeX4FyzQ7LDQHTCT+VrQQPrWn+yYZtttEmeSWRu3e4AUnDp0icurulrn4hM39AYL2xH2FT3mGFgDSI7Yk54U3mN8CaFjDbwTrAfYrHxG9MkQHdz3EBLq4cMo6h82WhUb5kcJzmhggP7J5qW3Zobxg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com; spf=pass smtp.mailfrom=sifive.com; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b=SmebI6Xv; arc=none smtp.client-ip=209.85.215.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sifive.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="SmebI6Xv" Received: by mail-pg1-f169.google.com with SMTP id 41be03b00d2f7-6e5fd488d9fso1705859a12.3 for ; Fri, 14 Jun 2024 07:22:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1718374930; x=1718979730; darn=vger.kernel.org; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=x4rb9oTxKJjPNdgedRtYvM9+kkgOfbw//+2IpuNP3u0=; b=SmebI6XvZZoATiIYVMnbhxy0IrMnkyGAxb8NVABwKCU/ZyEaPMqxm4jhV//Qv2AxQz 5+oq3uNWkYcyxDTYvUo1LaANgZKRw3ws5ZsLCSVK22K2w95HRTz+c4nuQ3twRM4YnR9P qcLIVGMd3I52dO5VMXOqVaZEnwXDwYTUJJskg+wr+PipESwoHhdG5yJkxuONfs1zYUPA RPNsmmrBBOzIBL6fuSG/FJPEUEbDaDNsjh5LenCt02M18i+lBmjgpH3kZH3q/IKrFllv fN66fLBQze9A+8RhMSYc4LCkI1a9OUNRsTsPScbWBaFkGwNJUjA7N3JA9t2Tin8Q7U40 n6DA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718374930; x=1718979730; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=x4rb9oTxKJjPNdgedRtYvM9+kkgOfbw//+2IpuNP3u0=; b=xQ3FnsQ+zyBboOh4hFAem6rhex+Y8ARwaooVmaA0mt2O5lAvpZH3xhQk4p4Tqo8Ryp Il5Xsq5PKEuxWyIO5adA/MwSwTDYIBse3RT165bCIlwmTtqAc6nlbGPR1e+dJC9YOelT z+8MNi5D39nlOiEGAXrCm/OXcT8zPWp+n9lUCqHdmM8lC8jveUEm9niqaDsMD2eldRMs G52jxl/XixThqraaDcTs0s2r735YtIwRfKGnVSTif/okqNGqFZVxK3cdk7F5eNk3qN7G 152ljU0tWL8UEDttJM9JxRyAz7wrp9hJ8P2znFTWjTSnoqWkZwIRhPX36YCULZgJ4kx2 HiPQ== X-Forwarded-Encrypted: i=1; AJvYcCUOBxocTKQxo9DwUWM3j8hAdd7yPPtgRxZOoywzzG4ElCIWXxp61p2CySIZaL8DAbjcDiet+MCbUnZuVvN5ZytaG+RZi5xEkwvA9RZ4 X-Gm-Message-State: AOJu0Yx32p6r8ABulzfjOXXxYma3fCNTPR+lh7QxfUrn+r2Eb6k99HQy cu34Tr4Fj/XpKewS+mN3eS3PJlIa+m9MA1h9wS1Cjrdf1j+bsc+YhUfXcjYHNuk= X-Google-Smtp-Source: AGHT+IEJt+rLxgOJ7sJAERmwv+9zCjQ5igS1rVXxOX0yXhtIC/U/Sk1uEt2Pa5Lkl3QWCML+x9LzJA== X-Received: by 2002:a17:902:cec6:b0:1f7:36a8:671d with SMTP id d9443c01a7336-1f8625cf720mr34825615ad.25.1718374930121; Fri, 14 Jun 2024 07:22:10 -0700 (PDT) Received: from hsinchu26.internal.sifive.com (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1f855e559d9sm32522005ad.35.2024.06.14.07.22.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Jun 2024 07:22:09 -0700 (PDT) From: Zong Li To: joro@8bytes.org, will@kernel.org, robin.murphy@arm.com, tjeznach@rivosinc.com, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, jgg@ziepe.ca, kevin.tian@intel.com, linux-kernel@vger.kernel.org, iommu@lists.linux.dev, linux-riscv@lists.infradead.org Cc: Zong Li Subject: [RFC PATCH v2 01/10] iommu/riscv: add RISC-V IOMMU PMU support Date: Fri, 14 Jun 2024 22:21:47 +0800 Message-Id: <20240614142156.29420-2-zong.li@sifive.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240614142156.29420-1-zong.li@sifive.com> References: <20240614142156.29420-1-zong.li@sifive.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This patch implements the RISC-V IOMMU hardware performance monitor, it includes the counting ans sampling mode. Specification doesn't define the event ID for counting the number of clock cycles, there is no associated iohpmevt0. But we need an event for counting cycle in perf, reserve the maximum number of event ID for it now. Signed-off-by: Zong Li --- drivers/iommu/riscv/Makefile | 2 +- drivers/iommu/riscv/iommu-bits.h | 16 ++ drivers/iommu/riscv/iommu-pmu.c | 479 +++++++++++++++++++++++++++++++ drivers/iommu/riscv/iommu.h | 8 + 4 files changed, 504 insertions(+), 1 deletion(-) create mode 100644 drivers/iommu/riscv/iommu-pmu.c diff --git a/drivers/iommu/riscv/Makefile b/drivers/iommu/riscv/Makefile index f54c9ed17d41..d36625a1fd08 100644 --- a/drivers/iommu/riscv/Makefile +++ b/drivers/iommu/riscv/Makefile @@ -1,3 +1,3 @@ # SPDX-License-Identifier: GPL-2.0-only -obj-$(CONFIG_RISCV_IOMMU) +=3D iommu.o iommu-platform.o +obj-$(CONFIG_RISCV_IOMMU) +=3D iommu.o iommu-platform.o iommu-pmu.o obj-$(CONFIG_RISCV_IOMMU_PCI) +=3D iommu-pci.o diff --git a/drivers/iommu/riscv/iommu-bits.h b/drivers/iommu/riscv/iommu-b= its.h index 98daf0e1a306..60523449f016 100644 --- a/drivers/iommu/riscv/iommu-bits.h +++ b/drivers/iommu/riscv/iommu-bits.h @@ -17,6 +17,7 @@ #include #include #include +#include =20 /* * Chapter 5: Memory Mapped register interface @@ -207,6 +208,7 @@ enum riscv_iommu_ddtp_modes { /* 5.22 Performance monitoring event counters (31 * 64bits) */ #define RISCV_IOMMU_REG_IOHPMCTR_BASE 0x0068 #define RISCV_IOMMU_REG_IOHPMCTR(_n) (RISCV_IOMMU_REG_IOHPMCTR_BASE + ((_n= ) * 0x8)) +#define RISCV_IOMMU_IOHPMCTR_COUNTER GENMASK_ULL(63, 0) =20 /* 5.23 Performance monitoring event selectors (31 * 64bits) */ #define RISCV_IOMMU_REG_IOHPMEVT_BASE 0x0160 @@ -250,6 +252,20 @@ enum riscv_iommu_hpmevent_id { RISCV_IOMMU_HPMEVENT_MAX =3D 9 }; =20 +/* Use maximum event ID for cycle event */ +#define RISCV_IOMMU_HPMEVENT_CYCLE GENMASK_ULL(14, 0) + +#define RISCV_IOMMU_HPM_COUNTER_NUM 32 + +struct riscv_iommu_pmu { + struct pmu pmu; + void __iomem *reg; + int num_counters; + u64 mask_counter; + struct perf_event *events[RISCV_IOMMU_IOHPMEVT_CNT + 1]; + DECLARE_BITMAP(used_counters, RISCV_IOMMU_IOHPMEVT_CNT + 1); +}; + /* 5.24 Translation request IOVA (64bits) */ #define RISCV_IOMMU_REG_TR_REQ_IOVA 0x0258 #define RISCV_IOMMU_TR_REQ_IOVA_VPN GENMASK_ULL(63, 12) diff --git a/drivers/iommu/riscv/iommu-pmu.c b/drivers/iommu/riscv/iommu-pm= u.c new file mode 100644 index 000000000000..5fc45aaf4ca3 --- /dev/null +++ b/drivers/iommu/riscv/iommu-pmu.c @@ -0,0 +1,479 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2024 SiFive + * + * Authors + * Zong Li + */ + +#include + +#include "iommu.h" +#include "iommu-bits.h" + +#define to_riscv_iommu_pmu(p) (container_of(p, struct riscv_iommu_pmu, pmu= )) + +#define RISCV_IOMMU_PMU_ATTR_EXTRACTOR(_name, _mask) \ + static inline u32 get_##_name(struct perf_event *event) \ + { \ + return FIELD_GET(_mask, event->attr.config); \ + } \ + +RISCV_IOMMU_PMU_ATTR_EXTRACTOR(event, RISCV_IOMMU_IOHPMEVT_EVENTID); +RISCV_IOMMU_PMU_ATTR_EXTRACTOR(partial_matching, RISCV_IOMMU_IOHPMEVT_DMAS= K); +RISCV_IOMMU_PMU_ATTR_EXTRACTOR(pid_pscid, RISCV_IOMMU_IOHPMEVT_PID_PSCID); +RISCV_IOMMU_PMU_ATTR_EXTRACTOR(did_gscid, RISCV_IOMMU_IOHPMEVT_DID_GSCID); +RISCV_IOMMU_PMU_ATTR_EXTRACTOR(filter_pid_pscid, RISCV_IOMMU_IOHPMEVT_PV_P= SCV); +RISCV_IOMMU_PMU_ATTR_EXTRACTOR(filter_did_gscid, RISCV_IOMMU_IOHPMEVT_DV_G= SCV); +RISCV_IOMMU_PMU_ATTR_EXTRACTOR(filter_id_type, RISCV_IOMMU_IOHPMEVT_IDT); + +/* Formats */ +PMU_FORMAT_ATTR(event, "config:0-14"); +PMU_FORMAT_ATTR(partial_matching, "config:15"); +PMU_FORMAT_ATTR(pid_pscid, "config:16-35"); +PMU_FORMAT_ATTR(did_gscid, "config:36-59"); +PMU_FORMAT_ATTR(filter_pid_pscid, "config:60"); +PMU_FORMAT_ATTR(filter_did_gscid, "config:61"); +PMU_FORMAT_ATTR(filter_id_type, "config:62"); + +static struct attribute *riscv_iommu_pmu_formats[] =3D { + &format_attr_event.attr, + &format_attr_partial_matching.attr, + &format_attr_pid_pscid.attr, + &format_attr_did_gscid.attr, + &format_attr_filter_pid_pscid.attr, + &format_attr_filter_did_gscid.attr, + &format_attr_filter_id_type.attr, + NULL, +}; + +static const struct attribute_group riscv_iommu_pmu_format_group =3D { + .name =3D "format", + .attrs =3D riscv_iommu_pmu_formats, +}; + +/* Events */ +static ssize_t riscv_iommu_pmu_event_show(struct device *dev, + struct device_attribute *attr, + char *page) +{ + struct perf_pmu_events_attr *pmu_attr; + + pmu_attr =3D container_of(attr, struct perf_pmu_events_attr, attr); + + return sprintf(page, "event=3D0x%02llx\n", pmu_attr->id); +} + +PMU_EVENT_ATTR(cycle, event_attr_cycle, + RISCV_IOMMU_HPMEVENT_CYCLE, riscv_iommu_pmu_event_show); +PMU_EVENT_ATTR(dont_count, event_attr_dont_count, + RISCV_IOMMU_HPMEVENT_INVALID, riscv_iommu_pmu_event_show); +PMU_EVENT_ATTR(untranslated_req, event_attr_untranslated_req, + RISCV_IOMMU_HPMEVENT_URQ, riscv_iommu_pmu_event_show); +PMU_EVENT_ATTR(translated_req, event_attr_translated_req, + RISCV_IOMMU_HPMEVENT_TRQ, riscv_iommu_pmu_event_show); +PMU_EVENT_ATTR(ats_trans_req, event_attr_ats_trans_req, + RISCV_IOMMU_HPMEVENT_ATS_RQ, riscv_iommu_pmu_event_show); +PMU_EVENT_ATTR(tlb_miss, event_attr_tlb_miss, + RISCV_IOMMU_HPMEVENT_TLB_MISS, riscv_iommu_pmu_event_show); +PMU_EVENT_ATTR(ddt_walks, event_attr_ddt_walks, + RISCV_IOMMU_HPMEVENT_DD_WALK, riscv_iommu_pmu_event_show); +PMU_EVENT_ATTR(pdt_walks, event_attr_pdt_walks, + RISCV_IOMMU_HPMEVENT_PD_WALK, riscv_iommu_pmu_event_show); +PMU_EVENT_ATTR(s_vs_pt_walks, event_attr_s_vs_pt_walks, + RISCV_IOMMU_HPMEVENT_S_VS_WALKS, riscv_iommu_pmu_event_show); +PMU_EVENT_ATTR(g_pt_walks, event_attr_g_pt_walks, + RISCV_IOMMU_HPMEVENT_G_WALKS, riscv_iommu_pmu_event_show); + +static struct attribute *riscv_iommu_pmu_events[] =3D { + &event_attr_cycle.attr.attr, + &event_attr_dont_count.attr.attr, + &event_attr_untranslated_req.attr.attr, + &event_attr_translated_req.attr.attr, + &event_attr_ats_trans_req.attr.attr, + &event_attr_tlb_miss.attr.attr, + &event_attr_ddt_walks.attr.attr, + &event_attr_pdt_walks.attr.attr, + &event_attr_s_vs_pt_walks.attr.attr, + &event_attr_g_pt_walks.attr.attr, + NULL, +}; + +static const struct attribute_group riscv_iommu_pmu_events_group =3D { + .name =3D "events", + .attrs =3D riscv_iommu_pmu_events, +}; + +static const struct attribute_group *riscv_iommu_pmu_attr_grps[] =3D { + &riscv_iommu_pmu_format_group, + &riscv_iommu_pmu_events_group, + NULL, +}; + +/* PMU Operations */ +static void riscv_iommu_pmu_set_counter(struct riscv_iommu_pmu *pmu, u32 i= dx, + u64 value) +{ + void __iomem *addr =3D pmu->reg + RISCV_IOMMU_REG_IOHPMCYCLES; + + if (WARN_ON_ONCE(idx < 0 || idx > pmu->num_counters)) + return; + + writeq(FIELD_PREP(RISCV_IOMMU_IOHPMCTR_COUNTER, value), addr + idx * 8); +} + +static u64 riscv_iommu_pmu_get_counter(struct riscv_iommu_pmu *pmu, u32 id= x) +{ + void __iomem *addr =3D pmu->reg + RISCV_IOMMU_REG_IOHPMCYCLES; + u64 value; + + if (WARN_ON_ONCE(idx < 0 || idx > pmu->num_counters)) + return -EINVAL; + + value =3D readq(addr + idx * 8); + + return FIELD_GET(RISCV_IOMMU_IOHPMCTR_COUNTER, value); +} + +static u64 riscv_iommu_pmu_get_event(struct riscv_iommu_pmu *pmu, u32 idx) +{ + void __iomem *addr =3D pmu->reg + RISCV_IOMMU_REG_IOHPMEVT_BASE; + + if (WARN_ON_ONCE(idx < 0 || idx > pmu->num_counters)) + return 0; + + /* There is no associtated IOHPMEVT0 for IOHPMCYCLES */ + if (idx =3D=3D 0) + return 0; + + return readq(addr + (idx - 1) * 8); +} + +static void riscv_iommu_pmu_set_event(struct riscv_iommu_pmu *pmu, u32 idx, + u64 value) +{ + void __iomem *addr =3D pmu->reg + RISCV_IOMMU_REG_IOHPMEVT_BASE; + + if (WARN_ON_ONCE(idx < 0 || idx > pmu->num_counters)) + return; + + /* There is no associtated IOHPMEVT0 for IOHPMCYCLES */ + if (idx =3D=3D 0) + return; + + writeq(value, addr + (idx - 1) * 8); +} + +static void riscv_iommu_pmu_enable_counter(struct riscv_iommu_pmu *pmu, u3= 2 idx) +{ + void __iomem *addr =3D pmu->reg + RISCV_IOMMU_REG_IOCOUNTINH; + u32 value =3D readl(addr); + + writel(value & ~BIT(idx), addr); +} + +static void riscv_iommu_pmu_disable_counter(struct riscv_iommu_pmu *pmu, u= 32 idx) +{ + void __iomem *addr =3D pmu->reg + RISCV_IOMMU_REG_IOCOUNTINH; + u32 value =3D readl(addr); + + writel(value | BIT(idx), addr); +} + +static void riscv_iommu_pmu_enable_ovf_intr(struct riscv_iommu_pmu *pmu, u= 32 idx) +{ + u64 value; + + if (get_event(pmu->events[idx]) =3D=3D RISCV_IOMMU_HPMEVENT_CYCLE) { + value =3D riscv_iommu_pmu_get_counter(pmu, idx) & ~RISCV_IOMMU_IOHPMCYCL= ES_OF; + writeq(value, pmu->reg + RISCV_IOMMU_REG_IOHPMCYCLES); + } else { + value =3D riscv_iommu_pmu_get_event(pmu, idx) & ~RISCV_IOMMU_IOHPMEVT_OF; + writeq(value, pmu->reg + RISCV_IOMMU_REG_IOHPMEVT_BASE + (idx - 1) * 8); + } +} + +static void riscv_iommu_pmu_disable_ovf_intr(struct riscv_iommu_pmu *pmu, = u32 idx) +{ + u64 value; + + if (get_event(pmu->events[idx]) =3D=3D RISCV_IOMMU_HPMEVENT_CYCLE) { + value =3D riscv_iommu_pmu_get_counter(pmu, idx) | RISCV_IOMMU_IOHPMCYCLE= S_OF; + writeq(value, pmu->reg + RISCV_IOMMU_REG_IOHPMCYCLES); + } else { + value =3D riscv_iommu_pmu_get_event(pmu, idx) | RISCV_IOMMU_IOHPMEVT_OF; + writeq(value, pmu->reg + RISCV_IOMMU_REG_IOHPMEVT_BASE + (idx - 1) * 8); + } +} + +static void riscv_iommu_pmu_start_all(struct riscv_iommu_pmu *pmu) +{ + int idx; + + for_each_set_bit(idx, pmu->used_counters, pmu->num_counters) { + riscv_iommu_pmu_enable_ovf_intr(pmu, idx); + riscv_iommu_pmu_enable_counter(pmu, idx); + } +} + +static void riscv_iommu_pmu_stop_all(struct riscv_iommu_pmu *pmu) +{ + writel(GENMASK_ULL(pmu->num_counters - 1, 0), + pmu->reg + RISCV_IOMMU_REG_IOCOUNTINH); +} + +/* PMU APIs */ +static int riscv_iommu_pmu_set_period(struct perf_event *event) +{ + struct riscv_iommu_pmu *pmu =3D to_riscv_iommu_pmu(event->pmu); + struct hw_perf_event *hwc =3D &event->hw; + s64 left =3D local64_read(&hwc->period_left); + s64 period =3D hwc->sample_period; + u64 max_period =3D pmu->mask_counter; + int ret =3D 0; + + if (unlikely(left <=3D -period)) { + left =3D period; + local64_set(&hwc->period_left, left); + hwc->last_period =3D period; + ret =3D 1; + } + + if (unlikely(left <=3D 0)) { + left +=3D period; + local64_set(&hwc->period_left, left); + hwc->last_period =3D period; + ret =3D 1; + } + + /* + * Limit the maximum period to prevent the counter value + * from overtaking the one we are about to program. In + * effect we are reducing max_period to account for + * interrupt latency (and we are being very conservative). + */ + if (left > (max_period >> 1)) + left =3D (max_period >> 1); + + local64_set(&hwc->prev_count, (u64)-left); + riscv_iommu_pmu_set_counter(pmu, hwc->idx, (u64)(-left) & max_period); + perf_event_update_userpage(event); + + return ret; +} + +static int riscv_iommu_pmu_event_init(struct perf_event *event) +{ + struct riscv_iommu_pmu *pmu =3D to_riscv_iommu_pmu(event->pmu); + struct hw_perf_event *hwc =3D &event->hw; + + hwc->idx =3D -1; + hwc->config =3D event->attr.config; + + if (!is_sampling_event(event)) { + /* + * For non-sampling runs, limit the sample_period to half + * of the counter width. That way, the new counter value + * is far less likely to overtake the previous one unless + * you have some serious IRQ latency issues. + */ + hwc->sample_period =3D pmu->mask_counter >> 1; + hwc->last_period =3D hwc->sample_period; + local64_set(&hwc->period_left, hwc->sample_period); + } + + return 0; +} + +static void riscv_iommu_pmu_update(struct perf_event *event) +{ + struct hw_perf_event *hwc =3D &event->hw; + struct riscv_iommu_pmu *pmu =3D to_riscv_iommu_pmu(event->pmu); + u64 delta, prev, now; + u32 idx =3D hwc->idx; + + do { + prev =3D local64_read(&hwc->prev_count); + now =3D riscv_iommu_pmu_get_counter(pmu, idx); + } while (local64_cmpxchg(&hwc->prev_count, prev, now) !=3D prev); + + delta =3D FIELD_GET(RISCV_IOMMU_IOHPMCTR_COUNTER, now - prev) & pmu->mask= _counter; + local64_add(delta, &event->count); + local64_sub(delta, &hwc->period_left); +} + +static void riscv_iommu_pmu_start(struct perf_event *event, int flags) +{ + struct riscv_iommu_pmu *pmu =3D to_riscv_iommu_pmu(event->pmu); + struct hw_perf_event *hwc =3D &event->hw; + + if (WARN_ON_ONCE(!(event->hw.state & PERF_HES_STOPPED))) + return; + + if (flags & PERF_EF_RELOAD) + WARN_ON_ONCE(!(event->hw.state & PERF_HES_UPTODATE)); + + hwc->state =3D 0; + riscv_iommu_pmu_set_period(event); + riscv_iommu_pmu_set_event(pmu, hwc->idx, hwc->config); + riscv_iommu_pmu_enable_ovf_intr(pmu, hwc->idx); + riscv_iommu_pmu_enable_counter(pmu, hwc->idx); + + perf_event_update_userpage(event); +} + +static void riscv_iommu_pmu_stop(struct perf_event *event, int flags) +{ + struct riscv_iommu_pmu *pmu =3D to_riscv_iommu_pmu(event->pmu); + struct hw_perf_event *hwc =3D &event->hw; + + if (hwc->state & PERF_HES_STOPPED) + return; + + riscv_iommu_pmu_set_event(pmu, hwc->idx, RISCV_IOMMU_HPMEVENT_INVALID); + riscv_iommu_pmu_disable_counter(pmu, hwc->idx); + + if ((flags & PERF_EF_UPDATE) && !(hwc->state & PERF_HES_UPTODATE)) + riscv_iommu_pmu_update(event); + + hwc->state |=3D PERF_HES_STOPPED | PERF_HES_UPTODATE; +} + +static int riscv_iommu_pmu_add(struct perf_event *event, int flags) +{ + struct hw_perf_event *hwc =3D &event->hw; + struct riscv_iommu_pmu *pmu =3D to_riscv_iommu_pmu(event->pmu); + unsigned int num_counters =3D pmu->num_counters; + int idx; + + /* Reserve index zero for iohpmcycles */ + if (get_event(event) =3D=3D RISCV_IOMMU_HPMEVENT_CYCLE) + idx =3D 0; + else + idx =3D find_next_zero_bit(pmu->used_counters, num_counters, 1); + + if (idx =3D=3D num_counters) + return -EAGAIN; + + set_bit(idx, pmu->used_counters); + + pmu->events[idx] =3D event; + hwc->idx =3D idx; + hwc->state =3D PERF_HES_STOPPED | PERF_HES_UPTODATE; + + if (flags & PERF_EF_START) + riscv_iommu_pmu_start(event, flags); + + /* Propagate changes to the userspace mapping. */ + perf_event_update_userpage(event); + + return 0; +} + +static void riscv_iommu_pmu_read(struct perf_event *event) +{ + riscv_iommu_pmu_update(event); +} + +static void riscv_iommu_pmu_del(struct perf_event *event, int flags) +{ + struct hw_perf_event *hwc =3D &event->hw; + struct riscv_iommu_pmu *pmu =3D to_riscv_iommu_pmu(event->pmu); + int idx =3D hwc->idx; + + riscv_iommu_pmu_stop(event, PERF_EF_UPDATE); + pmu->events[idx] =3D NULL; + clear_bit(idx, pmu->used_counters); + perf_event_update_userpage(event); +} + +irqreturn_t riscv_iommu_pmu_handle_irq(struct riscv_iommu_pmu *pmu) +{ + struct perf_sample_data data; + struct pt_regs *regs; + u32 ovf =3D readl(pmu->reg + RISCV_IOMMU_REG_IOCOUNTOVF); + int idx; + + if (!ovf) + return IRQ_NONE; + + riscv_iommu_pmu_stop_all(pmu); + + regs =3D get_irq_regs(); + + for_each_set_bit(idx, (unsigned long *)&ovf, pmu->num_counters) { + struct perf_event *event =3D pmu->events[idx]; + struct hw_perf_event *hwc; + + if (WARN_ON_ONCE(!event) || !is_sampling_event(event)) + continue; + + hwc =3D &event->hw; + + riscv_iommu_pmu_update(event); + perf_sample_data_init(&data, 0, hwc->last_period); + if (!riscv_iommu_pmu_set_period(event)) + continue; + + if (perf_event_overflow(event, &data, regs)) + riscv_iommu_pmu_stop(event, 0); + } + + riscv_iommu_pmu_start_all(pmu); + + return IRQ_HANDLED; +} + +int riscv_iommu_pmu_init(struct riscv_iommu_pmu *pmu, void __iomem *reg, + const char *dev_name) +{ + char *name; + int ret; + + pmu->reg =3D reg; + pmu->num_counters =3D RISCV_IOMMU_HPM_COUNTER_NUM; + pmu->mask_counter =3D RISCV_IOMMU_IOHPMCTR_COUNTER; + + pmu->pmu =3D (struct pmu) { + .task_ctx_nr =3D perf_invalid_context, + .event_init =3D riscv_iommu_pmu_event_init, + .add =3D riscv_iommu_pmu_add, + .del =3D riscv_iommu_pmu_del, + .start =3D riscv_iommu_pmu_start, + .stop =3D riscv_iommu_pmu_stop, + .read =3D riscv_iommu_pmu_read, + .attr_groups =3D riscv_iommu_pmu_attr_grps, + .capabilities =3D PERF_PMU_CAP_NO_EXCLUDE, + .module =3D THIS_MODULE, + }; + + name =3D kasprintf(GFP_KERNEL, "riscv_iommu_pmu_%s", dev_name); + + ret =3D perf_pmu_register(&pmu->pmu, name, -1); + if (ret) { + pr_err("Failed to register riscv_iommu_pmu_%s: %d\n", + dev_name, ret); + return ret; + } + + /* Stop all counters and later start the counter with perf */ + riscv_iommu_pmu_stop_all(pmu); + + pr_info("riscv_iommu_pmu_%s: Registered with %d counters\n", + dev_name, pmu->num_counters); + + return 0; +} + +void riscv_iommu_pmu_uninit(struct riscv_iommu_pmu *pmu) +{ + int idx; + + /* Disable interrupt and functions */ + for_each_set_bit(idx, pmu->used_counters, pmu->num_counters) { + riscv_iommu_pmu_disable_counter(pmu, idx); + riscv_iommu_pmu_disable_ovf_intr(pmu, idx); + } + + perf_pmu_unregister(&pmu->pmu); +} diff --git a/drivers/iommu/riscv/iommu.h b/drivers/iommu/riscv/iommu.h index b1c4664542b4..92659a8a75ae 100644 --- a/drivers/iommu/riscv/iommu.h +++ b/drivers/iommu/riscv/iommu.h @@ -60,11 +60,19 @@ struct riscv_iommu_device { unsigned int ddt_mode; dma_addr_t ddt_phys; u64 *ddt_root; + + /* hardware performance monitor */ + struct riscv_iommu_pmu pmu; }; =20 int riscv_iommu_init(struct riscv_iommu_device *iommu); void riscv_iommu_remove(struct riscv_iommu_device *iommu); =20 +int riscv_iommu_pmu_init(struct riscv_iommu_pmu *pmu, void __iomem *reg, + const char *name); +void riscv_iommu_pmu_uninit(struct riscv_iommu_pmu *pmu); +irqreturn_t riscv_iommu_pmu_handle_irq(struct riscv_iommu_pmu *pmu); + #define riscv_iommu_readl(iommu, addr) \ readl_relaxed((iommu)->reg + (addr)) =20 --=20 2.17.1 From nobody Wed Dec 17 13:54:13 2025 Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 225819441 for ; Fri, 14 Jun 2024 14:22:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718374935; cv=none; b=UcAc+tO9eHr43ZVzkKZ8MKJV18XJMWu5NDuk2EJaFp7/u9jQehv1jDreie0KCUV0qxeVItpVo2sc9v8a4U0zdB+U/RGWaXpkhQmHVF8L+qAbdhKUzp8SLzTccW4eY2Id4DJlEoQZIZcgmS3lNedmrbHGx0Q3kqpGamhgMTzHYio= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718374935; c=relaxed/simple; bh=AFqoKum5maZXoq29R1hOtMdbeMl8SDzKa1SfdPLj4w8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=sEiPY/InjFwaiTDKFWwjdgpubpqd4FRm86u3CfRKA4GVe3xYkge3theE1FA4Dxrkm7knDlGKdxUau9VI+y5XxVFu6aiPMKBNZWNUzMOOF+RdSOAVDgB2eVCdI89xkFaasRuHXgHkIdW62jniXCmqYGgdloXaV+SiMqZyOaPPP9o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com; spf=pass smtp.mailfrom=sifive.com; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b=eXlgTSok; arc=none smtp.client-ip=209.85.214.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sifive.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="eXlgTSok" Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-1f6c7cdec83so19211725ad.0 for ; Fri, 14 Jun 2024 07:22:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1718374933; x=1718979733; darn=vger.kernel.org; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=tlFDKoGPkyb3nGW5qL4azlR55hFLQbYBhLCB+CenNg8=; b=eXlgTSokqviq1/nMstq/JL7UJmJ9gArRjzutdDf+qxhddYor+VNDeP9hfOgZqmAOyS /6wAaCXSpKf7pngRxOTpFk/mzXmURFDtLBMCzIP+v2LlJXwqmBiWMt/kuAoAipjoRBB+ 1AOZwN9slrpHx50zloTb+zTzsUZg5kflHEB7SVBH+snx6ywa/upsgjJF8+TXgWSuDk1a IyivfiMhzZWYW01kkq+Mpiaa3rd2nqY+iDLbRdIzgKB266NM5zS7R4syNCq03+wDG8WM c09/dl+sRQ8NWCnFzrBXkZuQ5J8NgSQslWFUcmXkhGqPkmBCwM1sr8/jK80XcpEv2BwB NugQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718374933; x=1718979733; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=tlFDKoGPkyb3nGW5qL4azlR55hFLQbYBhLCB+CenNg8=; b=omdKrp58d+Riigv8OzxgezaA9N01XfgSC83oCOD0b521Rl9cBhsGEUAmvHeOwPIXHE eXOTsjuispcaSwLBdUS4/85KeQcngUsXGATVM4tNrL0i4oZ6Kc1a2TC2BFSlpF7ghVFT Av0QuExPJXjuWcr9SIL3ge5crgaftuf8xXsnB4PqxvaglVzeG00tiMXXI1/Cz4Mz44d7 Y/KOGztKQDJmat1wvTKR6AK1198ZVvBQo2NAb4WlIZOvJ55KX/TZI53T1St99Gvy0BB5 RVlS/QXCBLqCXlS55zHtxktJliZYw0EDz7sI9G5IvbkqQlJ2f4E9uWyzAEWrKBKAhjJ/ N7ow== X-Forwarded-Encrypted: i=1; AJvYcCWEm1UoOrCWjkplM4MJ55jElTr6RAT5R2MljQhzW7Yyv2DDzsC0VMY90KAHEVdwOsvi526NXw5k1bZWOY6/YGdg9J7/eW3Tm1FP8As7 X-Gm-Message-State: AOJu0Yw4ttc7/MVe5WbLDV4lXG2AEKf+MVt0d9ECN2wkz/s64QihS8qa sDYhCmqVltAeJH3X8JLC91x0XsQoX4kRfBsMv3yDF4UfIqVcBb/rXkn17SF9NfwP5HO1ns7ESvT K7d8= X-Google-Smtp-Source: AGHT+IE4q8GqM7rVw79GCJsFYmYJxenhwqq3dMfDvs+NWax7S8ItEx42VF35t3MI4KJSDGoZTtuT2g== X-Received: by 2002:a17:902:7883:b0:1f7:35e0:5af4 with SMTP id d9443c01a7336-1f8627e18bcmr22056005ad.30.1718374933563; Fri, 14 Jun 2024 07:22:13 -0700 (PDT) Received: from hsinchu26.internal.sifive.com (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1f855e559d9sm32522005ad.35.2024.06.14.07.22.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Jun 2024 07:22:13 -0700 (PDT) From: Zong Li To: joro@8bytes.org, will@kernel.org, robin.murphy@arm.com, tjeznach@rivosinc.com, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, jgg@ziepe.ca, kevin.tian@intel.com, linux-kernel@vger.kernel.org, iommu@lists.linux.dev, linux-riscv@lists.infradead.org Cc: Zong Li Subject: [RFC PATCH v2 02/10] iommu/riscv: support HPM and interrupt handling Date: Fri, 14 Jun 2024 22:21:48 +0800 Message-Id: <20240614142156.29420-3-zong.li@sifive.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240614142156.29420-1-zong.li@sifive.com> References: <20240614142156.29420-1-zong.li@sifive.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This patch initialize the pmu stuff and uninitialize it when driver removing. The interrupt handling is also provided, this handler need to be primary handler instead of thread function, because pt_regs is empty when threading the IRQ, but pt_regs is necessary by perf_event_overflow. Signed-off-by: Zong Li --- drivers/iommu/riscv/iommu.c | 65 +++++++++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) diff --git a/drivers/iommu/riscv/iommu.c b/drivers/iommu/riscv/iommu.c index 8b6a64c1ad8d..1716b2251f38 100644 --- a/drivers/iommu/riscv/iommu.c +++ b/drivers/iommu/riscv/iommu.c @@ -540,6 +540,62 @@ static irqreturn_t riscv_iommu_fltq_process(int irq, v= oid *data) return IRQ_HANDLED; } =20 +/* + * IOMMU Hardware performance monitor + */ + +/* HPM interrupt primary handler */ +static irqreturn_t riscv_iommu_hpm_irq_handler(int irq, void *dev_id) +{ + struct riscv_iommu_device *iommu =3D (struct riscv_iommu_device *)dev_id; + + /* Process pmu irq */ + riscv_iommu_pmu_handle_irq(&iommu->pmu); + + /* Clear performance monitoring interrupt pending */ + riscv_iommu_writel(iommu, RISCV_IOMMU_REG_IPSR, RISCV_IOMMU_IPSR_PMIP); + + return IRQ_HANDLED; +} + +/* HPM initialization */ +static int riscv_iommu_hpm_enable(struct riscv_iommu_device *iommu) +{ + int rc; + + if (!(iommu->caps & RISCV_IOMMU_CAPABILITIES_HPM)) + return 0; + + /* + * pt_regs is empty when threading the IRQ, but pt_regs is necessary + * by perf_event_overflow. Use primary handler instead of thread + * function for PM IRQ. + * + * Set the IRQF_ONESHOT flag because this IRQ might be shared with + * other threaded IRQs by other queues. + */ + rc =3D devm_request_irq(iommu->dev, + iommu->irqs[riscv_iommu_queue_vec(iommu, RISCV_IOMMU_IPSR_PMIP)], + riscv_iommu_hpm_irq_handler, IRQF_ONESHOT | IRQF_SHARED, NULL, io= mmu); + if (rc) + return rc; + + return riscv_iommu_pmu_init(&iommu->pmu, iommu->reg, dev_name(iommu->dev)= ); +} + +/* HPM uninitialization */ +static void riscv_iommu_hpm_disable(struct riscv_iommu_device *iommu) +{ + if (!(iommu->caps & RISCV_IOMMU_CAPABILITIES_HPM)) + return; + + devm_free_irq(iommu->dev, + iommu->irqs[riscv_iommu_queue_vec(iommu, RISCV_IOMMU_IPSR_PMIP)], + iommu); + + riscv_iommu_pmu_uninit(&iommu->pmu); +} + /* Lookup and initialize device context info structure. */ static struct riscv_iommu_dc *riscv_iommu_get_dc(struct riscv_iommu_device= *iommu, unsigned int devid) @@ -1612,6 +1668,9 @@ void riscv_iommu_remove(struct riscv_iommu_device *io= mmu) riscv_iommu_iodir_set_mode(iommu, RISCV_IOMMU_DDTP_IOMMU_MODE_OFF); riscv_iommu_queue_disable(&iommu->cmdq); riscv_iommu_queue_disable(&iommu->fltq); + + if (iommu->caps & RISCV_IOMMU_CAPABILITIES_HPM) + riscv_iommu_pmu_uninit(&iommu->pmu); } =20 int riscv_iommu_init(struct riscv_iommu_device *iommu) @@ -1651,6 +1710,10 @@ int riscv_iommu_init(struct riscv_iommu_device *iomm= u) if (rc) goto err_queue_disable; =20 + rc =3D riscv_iommu_hpm_enable(iommu); + if (rc) + goto err_hpm_disable; + rc =3D iommu_device_sysfs_add(&iommu->iommu, NULL, NULL, "riscv-iommu@%s", dev_name(iommu->dev)); if (rc) { @@ -1669,6 +1732,8 @@ int riscv_iommu_init(struct riscv_iommu_device *iommu) err_remove_sysfs: iommu_device_sysfs_remove(&iommu->iommu); err_iodir_off: + riscv_iommu_hpm_disable(iommu); +err_hpm_disable: riscv_iommu_iodir_set_mode(iommu, RISCV_IOMMU_DDTP_IOMMU_MODE_OFF); err_queue_disable: riscv_iommu_queue_disable(&iommu->fltq); --=20 2.17.1 From nobody Wed Dec 17 13:54:13 2025 Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1E10A15E8C for ; Fri, 14 Jun 2024 14:22:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718374939; cv=none; b=AmDusqUs9VKh4dx9o7kURE8d1keK0Xw8qAQBlThboUtQRn7kTwGLM3Ie9kDVQRyTIGCKxP7QdfyORZG1JtLxNYXAwB5u9Pzn/70xd2E5YsM7P6gFeGQjFqHHvU/1DGoXhMELkcWYj74JzEi9Xhm9XfmzyMDtBCx3sbjhGanRjsE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718374939; c=relaxed/simple; bh=7AgwP/NIXpZI892UfooA/8JyK1Fg8ru9MhfleWRh558=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=ZOQbn9bbJI40NgcbMjGCD6q3Ya2duiC/hIdzwG8uEpkqBEbki7lKgemJWAAMC48h80ri5tzBRtVWFPIyPCLa+2hU3ieVBKAfa5eaA9pDC2QJxfOtR/rqH6JbudrQJd9oByXPhX30zK6vC8suQ+RDBi/S0VryJR7eDC7v5Oqnn8Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com; spf=pass smtp.mailfrom=sifive.com; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b=VUJ1YZyo; arc=none smtp.client-ip=209.85.214.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sifive.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="VUJ1YZyo" Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-1f47f07acd3so20009895ad.0 for ; Fri, 14 Jun 2024 07:22:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1718374937; x=1718979737; darn=vger.kernel.org; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=KYeKWdSVxdzca9c5AXktAohba5fPqz4T3QpSza6gBEI=; b=VUJ1YZyoCENRRAriz7QBLtDYfasGYEx2JRCb9349aNy3Jcvqs/fqb53l2O7dlHArIC 34H0FC2fW6ucuXVRSuUbWYFXOM8Vy20XfcuW6lhlm300aDStAn5zKD1QO0HwQAWfHmTK kgseWgafRniQ1Pm/MLnmODJBa2tN6/lOW1LBVDitW/N7Sid7OfVlAhkGg9QZkgbXigwl kWvKgV23n4mrzcyVOVu6dDvXJ0IpMRZA+j+7Fa+zfggvQEQvzNkte/1pGzc1arYZKF50 UdK1SYhvzUGMPAsagqG9T2aOtkLyEQyQ53On9eidKiDEy7aFtAUsTYE/aXjKl//SQt7s F7cQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718374937; x=1718979737; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=KYeKWdSVxdzca9c5AXktAohba5fPqz4T3QpSza6gBEI=; b=atXRRG0Xez4r3X2UKVjAMrz7TU0dgD43wQm7iYufYLXlp2LYuNPmR0zK+Vbmceb9lp 9gksLZJYLHu9OZSJQEVQOQgW6IkFZ2ZNtiitnRNs3nxx7YJjyyEjwp5n0406jJQhv8zC 5CZG/7599ARVlOi2LbctnQPMn4Zwobr4hZwwcpex4t8tk2Mfkq7DYlN6Q9PJFn7FQzvl ltAGIedQ/HqcfYxGd/TJFFpyRVTRoJnms6k5vcpGuNMYu27a2zLGIYZuE9oaHWWm8d5Q QI160CY/iWkJq9PCdCvRtCmUugkJkyIxklUhMiSi4KA2esZLurkbhuwDDjWiozXriAL0 sKzg== X-Forwarded-Encrypted: i=1; AJvYcCXRnJAV0S83IvVMurPLEjetR3xiSo9qeqX1+Fp8xJZjop/x+1pa3F5+T0vkVU4oN4OkWPSHosD818sI53NcdPN3nFoRl9lBh3BHj0Hy X-Gm-Message-State: AOJu0YyqeDJ7NMF98QpOWY7GPEpOUbgudS+BkOaSHIPZDn0zuvmPEfHd YI0+3ir7ypQ7egEo466cu0IRQSrDx8O1u6Ih5YJQVZJ1WADOSOGZdGK3VDa1LYQ= X-Google-Smtp-Source: AGHT+IEaapuErqClzYkwdWIAoziKliaNOF4tmsWVaA2CtvzOJToxjwlNEcROXRtWO3Y8fdwM3JdA+g== X-Received: by 2002:a17:903:1250:b0:1f7:2050:9a76 with SMTP id d9443c01a7336-1f8625c0d68mr33773445ad.8.1718374937555; Fri, 14 Jun 2024 07:22:17 -0700 (PDT) Received: from hsinchu26.internal.sifive.com (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1f855e559d9sm32522005ad.35.2024.06.14.07.22.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Jun 2024 07:22:17 -0700 (PDT) From: Zong Li To: joro@8bytes.org, will@kernel.org, robin.murphy@arm.com, tjeznach@rivosinc.com, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, jgg@ziepe.ca, kevin.tian@intel.com, linux-kernel@vger.kernel.org, iommu@lists.linux.dev, linux-riscv@lists.infradead.org Cc: Zong Li Subject: [RFC PATCH v2 03/10] iommu/riscv: use data structure instead of individual values Date: Fri, 14 Jun 2024 22:21:49 +0800 Message-Id: <20240614142156.29420-4-zong.li@sifive.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240614142156.29420-1-zong.li@sifive.com> References: <20240614142156.29420-1-zong.li@sifive.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The parameter will be increased when we need to set up more bit fields in the device context. Use a data structure to wrap them up. Signed-off-by: Zong Li --- drivers/iommu/riscv/iommu.c | 31 +++++++++++++++++++------------ 1 file changed, 19 insertions(+), 12 deletions(-) diff --git a/drivers/iommu/riscv/iommu.c b/drivers/iommu/riscv/iommu.c index 1716b2251f38..9aeb4b20c145 100644 --- a/drivers/iommu/riscv/iommu.c +++ b/drivers/iommu/riscv/iommu.c @@ -1045,7 +1045,7 @@ static void riscv_iommu_iotlb_inval(struct riscv_iomm= u_domain *domain, * interim translation faults. */ static void riscv_iommu_iodir_update(struct riscv_iommu_device *iommu, - struct device *dev, u64 fsc, u64 ta) + struct device *dev, struct riscv_iommu_dc *new_dc) { struct iommu_fwspec *fwspec =3D dev_iommu_fwspec_get(dev); struct riscv_iommu_dc *dc; @@ -1079,10 +1079,10 @@ static void riscv_iommu_iodir_update(struct riscv_i= ommu_device *iommu, for (i =3D 0; i < fwspec->num_ids; i++) { dc =3D riscv_iommu_get_dc(iommu, fwspec->ids[i]); tc =3D READ_ONCE(dc->tc); - tc |=3D ta & RISCV_IOMMU_DC_TC_V; + tc |=3D new_dc->ta & RISCV_IOMMU_DC_TC_V; =20 - WRITE_ONCE(dc->fsc, fsc); - WRITE_ONCE(dc->ta, ta & RISCV_IOMMU_PC_TA_PSCID); + WRITE_ONCE(dc->fsc, new_dc->fsc); + WRITE_ONCE(dc->ta, new_dc->ta & RISCV_IOMMU_PC_TA_PSCID); /* Update device context, write TC.V as the last step. */ dma_wmb(); WRITE_ONCE(dc->tc, tc); @@ -1369,20 +1369,20 @@ static int riscv_iommu_attach_paging_domain(struct = iommu_domain *iommu_domain, struct riscv_iommu_domain *domain =3D iommu_domain_to_riscv(iommu_domain); struct riscv_iommu_device *iommu =3D dev_to_iommu(dev); struct riscv_iommu_info *info =3D dev_iommu_priv_get(dev); - u64 fsc, ta; + struct riscv_iommu_dc dc =3D {0}; =20 if (!riscv_iommu_pt_supported(iommu, domain->pgd_mode)) return -ENODEV; =20 - fsc =3D FIELD_PREP(RISCV_IOMMU_PC_FSC_MODE, domain->pgd_mode) | - FIELD_PREP(RISCV_IOMMU_PC_FSC_PPN, virt_to_pfn(domain->pgd_root)); - ta =3D FIELD_PREP(RISCV_IOMMU_PC_TA_PSCID, domain->pscid) | - RISCV_IOMMU_PC_TA_V; + dc.fsc =3D FIELD_PREP(RISCV_IOMMU_PC_FSC_MODE, domain->pgd_mode) | + FIELD_PREP(RISCV_IOMMU_PC_FSC_PPN, virt_to_pfn(domain->pgd_root)); + dc.ta =3D FIELD_PREP(RISCV_IOMMU_PC_TA_PSCID, domain->pscid) | + RISCV_IOMMU_PC_TA_V; =20 if (riscv_iommu_bond_link(domain, dev)) return -ENOMEM; =20 - riscv_iommu_iodir_update(iommu, dev, fsc, ta); + riscv_iommu_iodir_update(iommu, dev, &dc); riscv_iommu_bond_unlink(info->domain, dev); info->domain =3D domain; =20 @@ -1484,9 +1484,12 @@ static int riscv_iommu_attach_blocking_domain(struct= iommu_domain *iommu_domain, { struct riscv_iommu_device *iommu =3D dev_to_iommu(dev); struct riscv_iommu_info *info =3D dev_iommu_priv_get(dev); + struct riscv_iommu_dc dc =3D {0}; + + dc.fsc =3D RISCV_IOMMU_FSC_BARE; =20 /* Make device context invalid, translation requests will fault w/ #258 */ - riscv_iommu_iodir_update(iommu, dev, RISCV_IOMMU_FSC_BARE, 0); + riscv_iommu_iodir_update(iommu, dev, &dc); riscv_iommu_bond_unlink(info->domain, dev); info->domain =3D NULL; =20 @@ -1505,8 +1508,12 @@ static int riscv_iommu_attach_identity_domain(struct= iommu_domain *iommu_domain, { struct riscv_iommu_device *iommu =3D dev_to_iommu(dev); struct riscv_iommu_info *info =3D dev_iommu_priv_get(dev); + struct riscv_iommu_dc dc =3D {0}; + + dc.fsc =3D RISCV_IOMMU_FSC_BARE; + dc.ta =3D RISCV_IOMMU_PC_TA_V; =20 - riscv_iommu_iodir_update(iommu, dev, RISCV_IOMMU_FSC_BARE, RISCV_IOMMU_PC= _TA_V); + riscv_iommu_iodir_update(iommu, dev, &dc); riscv_iommu_bond_unlink(info->domain, dev); info->domain =3D NULL; =20 --=20 2.17.1 From nobody Wed Dec 17 13:54:13 2025 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B9E0D17C79 for ; Fri, 14 Jun 2024 14:22:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718374942; cv=none; b=pmfLUzfNUOUVBRtcis0VEQvrbxvSnQWbl9eCgfEbjKthAS9w1rJV+WYzZzfwH2VLWuFQt0G06dU14yGk5X6XnML9vB19s/CS8ZgeeYcUid9diXMyWTXTxHREajVeNDCv0YxwGkM1UMTeSPuAcPlVHo2Ilw7ifkIcgaFUX16R6bE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718374942; c=relaxed/simple; bh=skj2/BWRX5MDxu4BMRzPV1cfBbCsNidIjZimfPy4/xk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=Cwg1kFSx3/DhNnEIrk+yaqgeEcwxiK5DzmGxnYk9aUnCuuLAjKT+7d4CR0NJ5ZZvebG5LiZyi1k0EZqg3PS/oWCsiJmab0d/FhiVclLDbRBDfRK0FJ6bV6R7qD9O0cjrzhmxrucMiQd170ohtHCU8Mjd1DRltUP+9WC2sPlXUYk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com; spf=pass smtp.mailfrom=sifive.com; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b=Lmq6k/xn; arc=none smtp.client-ip=209.85.214.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sifive.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="Lmq6k/xn" Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-1f8395a530dso20581175ad.0 for ; Fri, 14 Jun 2024 07:22:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1718374940; x=1718979740; darn=vger.kernel.org; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=M7iZUV99NQN3RgJM91vO01sfzoKXhBrbq1qPqNaXoaI=; b=Lmq6k/xnHHKEm2EmYx8erFaet3tZe6n31YPknnqKkZ8LOHBcMoM5j4iCRKw4Mg/Eoo 57Pn8PCiHIeMObm30+cfxxIHMuCxGzef73j3PV5Qd6inhlDO8dWjf64wcScChcMDIB63 T1AK6JjqveqP2K/pk24tABkIJzVKEi6oH/0AOH12YfmgoBvDI+84Q61OLYyWEvwSFztA Rr+Gwd7eDosfp/Y2smxSSapG2wq2OBueNqQVL8YF6ghzT3HprmOssFSVH4Ds9O8qy8EI 27FRnqh1d/MFn1lO9l67vJ01HWTZKoc699oEEY1YNGMZlm4PmtRRsp+leuZA5/pS4cSX s7fA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718374940; x=1718979740; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=M7iZUV99NQN3RgJM91vO01sfzoKXhBrbq1qPqNaXoaI=; b=MWGIERSrDosJKDqe0vbKnT1k1r8qkqFYExNywitlD9bmwwCGVPvQK3E34+ELTyZ7q2 szqxtlwese4R9zb86AG5G6wWI6ZYlyj2jfq6oQy4kmbuvFk/wuinJ6pWqxjkk67S+Aa+ fORmuFPt/qjSaVzqaSVD4LivwISXfPuYcHrCC+kSCPxnSsO/2KWKoeslT/T5TVzIe+q6 CE4Wp1MhMpwNjWOUgm5jzwvL5wqKF48ynVwjiCbolwW3x7Jbqa8WzAIJl8+m/T8zY4Le t0q7TNi90ZTOzXiLEkXxOcU5XRdA1aPmNCBXivHLqBXsbMoP/NE7me3ekgq/sbtZGeIx TAxA== X-Forwarded-Encrypted: i=1; AJvYcCVZX7Fmm4bJJucs95XRADZMDknLVJAuLkCr1FqN283jgWLrYZDiamqjJPoF4DQ0bp2eg3FQ6F04XbhKXjwsfHpebCnMBwPIVbleWI0E X-Gm-Message-State: AOJu0YzThfbUtHWLh9ovSGphcbcOLJdllS/QeeJIo08MbServUtVjH83 9/CHeFjtH98OtgTIKSe3XbBQe2X3F/hz1rmBmLqtYVjRmVYOObnSKq2vTXvY/yU= X-Google-Smtp-Source: AGHT+IGJERSTIzaOjggWWP0rQrcqidvG0Fn1shO0SdaiGkk+fF3agyv6rp5JLRqhxN5jNm2HLZv71g== X-Received: by 2002:a17:902:6906:b0:1f7:21fd:ab83 with SMTP id d9443c01a7336-1f8629feaeemr22250365ad.54.1718374940068; Fri, 14 Jun 2024 07:22:20 -0700 (PDT) Received: from hsinchu26.internal.sifive.com (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1f855e559d9sm32522005ad.35.2024.06.14.07.22.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Jun 2024 07:22:19 -0700 (PDT) From: Zong Li To: joro@8bytes.org, will@kernel.org, robin.murphy@arm.com, tjeznach@rivosinc.com, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, jgg@ziepe.ca, kevin.tian@intel.com, linux-kernel@vger.kernel.org, iommu@lists.linux.dev, linux-riscv@lists.infradead.org Cc: Zong Li Subject: [RFC PATCH v2 04/10] iommu/riscv: add iotlb_sync_map operation support Date: Fri, 14 Jun 2024 22:21:50 +0800 Message-Id: <20240614142156.29420-5-zong.li@sifive.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240614142156.29420-1-zong.li@sifive.com> References: <20240614142156.29420-1-zong.li@sifive.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Add iotlb_sync_map operation for flush IOTLB. Software must flush the IOTLB after each page table. Signed-off-by: Zong Li --- drivers/iommu/riscv/Makefile | 1 + drivers/iommu/riscv/iommu.c | 11 +++++++++++ 2 files changed, 12 insertions(+) diff --git a/drivers/iommu/riscv/Makefile b/drivers/iommu/riscv/Makefile index d36625a1fd08..f02ce6ebfbd0 100644 --- a/drivers/iommu/riscv/Makefile +++ b/drivers/iommu/riscv/Makefile @@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0-only obj-$(CONFIG_RISCV_IOMMU) +=3D iommu.o iommu-platform.o iommu-pmu.o obj-$(CONFIG_RISCV_IOMMU_PCI) +=3D iommu-pci.o +obj-$(CONFIG_SIFIVE_IOMMU) +=3D iommu-sifive.o diff --git a/drivers/iommu/riscv/iommu.c b/drivers/iommu/riscv/iommu.c index 9aeb4b20c145..df7aeb2571ae 100644 --- a/drivers/iommu/riscv/iommu.c +++ b/drivers/iommu/riscv/iommu.c @@ -1115,6 +1115,16 @@ static void riscv_iommu_iotlb_sync(struct iommu_doma= in *iommu_domain, riscv_iommu_iotlb_inval(domain, gather->start, gather->end); } =20 +static int riscv_iommu_iotlb_sync_map(struct iommu_domain *iommu_domain, + unsigned long iova, size_t size) +{ + struct riscv_iommu_domain *domain =3D iommu_domain_to_riscv(iommu_domain); + + riscv_iommu_iotlb_inval(domain, iova, iova + size - 1); + + return 0; +} + static inline size_t get_page_size(size_t size) { if (size >=3D IOMMU_PAGE_SIZE_512G) @@ -1396,6 +1406,7 @@ static const struct iommu_domain_ops riscv_iommu_pagi= ng_domain_ops =3D { .unmap_pages =3D riscv_iommu_unmap_pages, .iova_to_phys =3D riscv_iommu_iova_to_phys, .iotlb_sync =3D riscv_iommu_iotlb_sync, + .iotlb_sync_map =3D riscv_iommu_iotlb_sync_map, .flush_iotlb_all =3D riscv_iommu_iotlb_flush_all, }; =20 --=20 2.17.1 From nobody Wed Dec 17 13:54:13 2025 Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3A68320B20 for ; Fri, 14 Jun 2024 14:22:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718374944; cv=none; b=TFIGT3qayCTgFOejpBA+sSw3xrVJPTFult33EhovliZ75ZK0XRI0Cs/90v2gH55JMEtIa42Ds8XoZWaNG8XUPspGojc+8pVY1Zutkjk4w0LFcnJLhVdqLm+tC6KzEd3EbR1PvjqF5U8re3P0s90Fym+Z0dEoopth5s+4Ay3r+fQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718374944; c=relaxed/simple; bh=HQCoIv3Xpbx6DOVWD61lR4O3qkiOmoi7Eje3+sHhmEc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=Q9RhrKqwErvxuqSvqrbdRe59mF8/8aE05VpmahX9KH9vDigMnIjfFnHzsMxV54wp4yvZ1/NDx5XRWQa9mTnkaVRO6vcAVSDUaoZHFDJbF1/iY5N5jqFmezTXGtStPlH6k8otflBCyRKDrvf0EzAaApepajtQK7L+zOl+jbMCY0c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com; spf=pass smtp.mailfrom=sifive.com; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b=IJlrisRM; arc=none smtp.client-ip=209.85.214.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sifive.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="IJlrisRM" Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-1f862f7c7edso9038015ad.3 for ; Fri, 14 Jun 2024 07:22:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1718374942; x=1718979742; darn=vger.kernel.org; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=3Daef6kUbuf4+6WFIjDcTl7WQW1O4tQM7jDjBoLtjZ4=; b=IJlrisRMrOQYPG46ITS2iloUP2y9aX83eTnTSVFPolwiQj7g1aDCrOthwl/Jeo2cl0 0bxi+MO5uJOG5WsuXCDyxM4Qa+0EjdrihWn/IXjKGWiRWcOkdSaq5zxAyadddixJv0vF yNIuCxb+mzfe6ozvA9dCx/dRGXJTRVM0uRFQXpPK6QfASv2O6xXp9mKR2zy+gmh2PNQz cXWwZfhlHD5COTb0hGNfgj5R/czH00sxVrqqLTxkz8Q1P+Poblih5O7QILGwcy4JohNq AmaSVAUfXr6/ltqSbEV8ti/G2ODZMxGdKD1V5OK79bVSyIKC9K0sqGfI0Pf0os6jxwkM yV/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718374942; x=1718979742; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=3Daef6kUbuf4+6WFIjDcTl7WQW1O4tQM7jDjBoLtjZ4=; b=FanuJCscdbNFHjXx1XDJswqkGBjs9kxdJgsIi+LWULc+fv8VZIydbYn57To9Vsf2Pl nQAKCYlS4HqsAtFJ68Icu8whpTOYvwqQZNFxQIk6rBSHL0c5erTA5m5XsLf9gMP+2rEF B2QHCj9PZgc6edptDPsHVqlO+FOm5g+0cvd/8oKuQQ5NeXD6SXki62eYGI9YOoPWaqE7 ZvVCikey90vYFspF9J4M9NQw2Ep2h/JJGPL+ObN7qF9vqrB8sZKZYGITr3W9XVi/ZgBo 69RnmeqKFNUs+3rEeaTYF18W2yUsB7irFEhHrdmwcCjh1kj3rUov/JqTuOFF6csi9W/C L6qg== X-Forwarded-Encrypted: i=1; AJvYcCWJJ2015xTnE98jig1f0zQOlcxrM2a07YNwchOysLDCCCF8H/CPSKOlRTTSqVQrKuNQHMR8be68mEfX2cVVjjHplENj3zopSGqNo505 X-Gm-Message-State: AOJu0YznIeYyjN5UqgGoFBk7X1OLjbI59JNGZ06Mzi917sqGNNZVZte+ VHYD1XMnq6GoLgJqeWNJkRlAiKynDNepNpNUy07UlayYKZ8yyomUavCyzUz5EYU= X-Google-Smtp-Source: AGHT+IFqgOZYvBJ4kxh5nI8iWhYcYwWgqXcjlHpYLYmYilMYTECy6aBNWzRZ7wIFJH/yInxDq6z0lg== X-Received: by 2002:a17:903:32ca:b0:1f7:c33:aa92 with SMTP id d9443c01a7336-1f8625c06f1mr32662045ad.9.1718374942583; Fri, 14 Jun 2024 07:22:22 -0700 (PDT) Received: from hsinchu26.internal.sifive.com (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1f855e559d9sm32522005ad.35.2024.06.14.07.22.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Jun 2024 07:22:22 -0700 (PDT) From: Zong Li To: joro@8bytes.org, will@kernel.org, robin.murphy@arm.com, tjeznach@rivosinc.com, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, jgg@ziepe.ca, kevin.tian@intel.com, linux-kernel@vger.kernel.org, iommu@lists.linux.dev, linux-riscv@lists.infradead.org Cc: Zong Li Subject: [RFC PATCH v2 05/10] iommu/riscv: support GSCID and GVMA invalidation command Date: Fri, 14 Jun 2024 22:21:51 +0800 Message-Id: <20240614142156.29420-6-zong.li@sifive.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240614142156.29420-1-zong.li@sifive.com> References: <20240614142156.29420-1-zong.li@sifive.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This patch adds a ID Allocator for GSCID and a wrap for setting up GSCID in IOTLB invalidation command. Set up iohgatp to enable second stage table and flush stage-2 table if the GSCID is set. The GSCID of domain should be freed when release domain. GSCID will be allocated for parent domain in nested IOMMU process. Signed-off-by: Zong Li --- drivers/iommu/riscv/iommu-bits.h | 7 ++++++ drivers/iommu/riscv/iommu.c | 39 ++++++++++++++++++++++++++++---- 2 files changed, 41 insertions(+), 5 deletions(-) diff --git a/drivers/iommu/riscv/iommu-bits.h b/drivers/iommu/riscv/iommu-b= its.h index 60523449f016..214735a335fd 100644 --- a/drivers/iommu/riscv/iommu-bits.h +++ b/drivers/iommu/riscv/iommu-bits.h @@ -731,6 +731,13 @@ static inline void riscv_iommu_cmd_inval_vma(struct ri= scv_iommu_command *cmd) cmd->dword1 =3D 0; } =20 +static inline void riscv_iommu_cmd_inval_gvma(struct riscv_iommu_command *= cmd) +{ + cmd->dword0 =3D FIELD_PREP(RISCV_IOMMU_CMD_OPCODE, RISCV_IOMMU_CMD_IOTINV= AL_OPCODE) | + FIELD_PREP(RISCV_IOMMU_CMD_FUNC, RISCV_IOMMU_CMD_IOTINVAL_FUNC_GVM= A); + cmd->dword1 =3D 0; +} + static inline void riscv_iommu_cmd_inval_set_addr(struct riscv_iommu_comma= nd *cmd, u64 addr) { diff --git a/drivers/iommu/riscv/iommu.c b/drivers/iommu/riscv/iommu.c index df7aeb2571ae..45309bd096e5 100644 --- a/drivers/iommu/riscv/iommu.c +++ b/drivers/iommu/riscv/iommu.c @@ -45,6 +45,10 @@ static DEFINE_IDA(riscv_iommu_pscids); #define RISCV_IOMMU_MAX_PSCID (BIT(20) - 1) =20 +/* IOMMU GSCID allocation namespace. */ +static DEFINE_IDA(riscv_iommu_gscids); +#define RISCV_IOMMU_MAX_GSCID (BIT(16) - 1) + /* Device resource-managed allocations */ struct riscv_iommu_devres { void *addr; @@ -845,6 +849,7 @@ struct riscv_iommu_domain { struct list_head bonds; spinlock_t lock; /* protect bonds list updates. */ int pscid; + int gscid; int amo_enabled:1; int numa_node; unsigned int pgd_mode; @@ -993,20 +998,33 @@ static void riscv_iommu_iotlb_inval(struct riscv_iomm= u_domain *domain, rcu_read_lock(); =20 prev =3D NULL; + list_for_each_entry_rcu(bond, &domain->bonds, list) { iommu =3D dev_to_iommu(bond->dev); =20 /* * IOTLB invalidation request can be safely omitted if already sent - * to the IOMMU for the same PSCID, and with domain->bonds list + * to the IOMMU for the same PSCID/GSCID, and with domain->bonds list * arranged based on the device's IOMMU, it's sufficient to check * last device the invalidation was sent to. */ if (iommu =3D=3D prev) continue; =20 - riscv_iommu_cmd_inval_vma(&cmd); - riscv_iommu_cmd_inval_set_pscid(&cmd, domain->pscid); + /* + * S2 domain needs to flush entries in stage-2 page table, its + * bond list has host devices and pass-through devices, the GVMA + * command is no effect on host devices, because there are no + * mapping of host devices in stage-2 page table. + */ + if (domain->gscid) { + riscv_iommu_cmd_inval_gvma(&cmd); + riscv_iommu_cmd_inval_set_gscid(&cmd, domain->gscid); + } else { + riscv_iommu_cmd_inval_vma(&cmd); + riscv_iommu_cmd_inval_set_pscid(&cmd, domain->pscid); + } + if (len && len < RISCV_IOMMU_IOTLB_INVAL_LIMIT) { for (iova =3D start; iova < end; iova +=3D PAGE_SIZE) { riscv_iommu_cmd_inval_set_addr(&cmd, iova); @@ -1015,6 +1033,7 @@ static void riscv_iommu_iotlb_inval(struct riscv_iomm= u_domain *domain, } else { riscv_iommu_cmd_send(iommu, &cmd); } + prev =3D iommu; } =20 @@ -1083,6 +1102,7 @@ static void riscv_iommu_iodir_update(struct riscv_iom= mu_device *iommu, =20 WRITE_ONCE(dc->fsc, new_dc->fsc); WRITE_ONCE(dc->ta, new_dc->ta & RISCV_IOMMU_PC_TA_PSCID); + WRITE_ONCE(dc->iohgatp, new_dc->iohgatp); /* Update device context, write TC.V as the last step. */ dma_wmb(); WRITE_ONCE(dc->tc, tc); @@ -1354,6 +1374,9 @@ static void riscv_iommu_free_paging_domain(struct iom= mu_domain *iommu_domain) if ((int)domain->pscid > 0) ida_free(&riscv_iommu_pscids, domain->pscid); =20 + if ((int)domain->gscid > 0) + ida_free(&riscv_iommu_gscids, domain->gscid); + riscv_iommu_pte_free(domain, _io_pte_entry(pfn, _PAGE_TABLE), NULL); kfree(domain); } @@ -1384,8 +1407,14 @@ static int riscv_iommu_attach_paging_domain(struct i= ommu_domain *iommu_domain, if (!riscv_iommu_pt_supported(iommu, domain->pgd_mode)) return -ENODEV; =20 - dc.fsc =3D FIELD_PREP(RISCV_IOMMU_PC_FSC_MODE, domain->pgd_mode) | - FIELD_PREP(RISCV_IOMMU_PC_FSC_PPN, virt_to_pfn(domain->pgd_root)); + if (domain->gscid) + dc.iohgatp =3D FIELD_PREP(RISCV_IOMMU_DC_IOHGATP_MODE, domain->pgd_mode)= | + FIELD_PREP(RISCV_IOMMU_DC_IOHGATP_GSCID, domain->gscid) | + FIELD_PREP(RISCV_IOMMU_DC_IOHGATP_PPN, virt_to_pfn(domain->pgd_roo= t)); + else + dc.fsc =3D FIELD_PREP(RISCV_IOMMU_PC_FSC_MODE, domain->pgd_mode) | + FIELD_PREP(RISCV_IOMMU_PC_FSC_PPN, virt_to_pfn(domain->pgd_root)); + dc.ta =3D FIELD_PREP(RISCV_IOMMU_PC_TA_PSCID, domain->pscid) | RISCV_IOMMU_PC_TA_V; =20 --=20 2.17.1 From nobody Wed Dec 17 13:54:13 2025 Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B1BBE28DC7 for ; Fri, 14 Jun 2024 14:22:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718374947; cv=none; b=ZuBdkJhsKWpRXC1akUY85GJChkOCbyOodQBmG7/P6fQ5CcbEjfoJXNMNLzVFCP10Ey7pYPeBr10iH+iOLzk3O24OJc5u4gv/kteeoejeJ31OmYo7gISMwV/6KsrZiC2aisVE5vbXVIHf+NkLdrLRH76N32SX979SB8DeEELvqPE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718374947; c=relaxed/simple; bh=nU8H57vWK8w7/x5rQqwOK9kd4fm89ZlCq3T7oTy5L6U=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=hAyaGM2zHHN5t+y63isKVEDVwtBmXc2OUmN9k8F/0gweW1jyDlSryDzqnw97wFj+riEkT/RMixHp/MK2FCnUy2aMXNxFLxRXNXyW3+l5wSWAb+GMEmjZ/OINvZX/1WPgnKAw77BTpFwB3MjT9FHVfZTBMHKXGbtHmUbyvEE7pfQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com; spf=pass smtp.mailfrom=sifive.com; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b=RVMZ0TJV; arc=none smtp.client-ip=209.85.214.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sifive.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="RVMZ0TJV" Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-1f480624d0fso17251525ad.1 for ; Fri, 14 Jun 2024 07:22:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1718374945; x=1718979745; darn=vger.kernel.org; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=E4t/LszTGtZ4bHNAAc59INUfcR6I2TK1LJsq3cDy7I4=; b=RVMZ0TJViWdUPG8q2gDzh/t4LCRvUs4BVbZoyP1TqBec4pa91/bls5dXfOwUgdcChc Osy1wCXRKRIbFG0c/7zNdHknmnJBSvBmigFOPjqNLl+lIX0lsluu2zQyDzIolAT+HePL rqLwGAqGka7SrGj4hkGnSViBrAB7cUt78Kyf254jH3iHLU8t8zUH7GOZ7gY/+T1BunOF qdp8FXh4395BMwnSZDLA3nz0xTNEdT2+U7yMCROQiShmYAauEGPE7utir95NMZG31IjV eBDDUVfe87tAgMM32DYv69fhe2CvtEE5IqbswuMZZkqxo0khH8/UuBCZXpH7AR+rYFC9 8zKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718374945; x=1718979745; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=E4t/LszTGtZ4bHNAAc59INUfcR6I2TK1LJsq3cDy7I4=; b=g42iE3CKyVhoHj02eD3ScsFIoKrJcMFlvqFrykUJj89eBfGcIYX6dRtj2F374S7Qa5 NyB2Iy9kmgC5UAzVaIbkmEoKr5binvnQ3xnUCVqIQ7EzuX+kZPIzIyEPtOHXFR3UU4cQ d3KnbXgwmXJIUo/7tZLNnCdVgodRbGmvNXtp5XRiBfix8IxfgX3Pzvhh9fRuFMyYZp6c 9skug2g+wYVWNtk2QB8G3q0RyeRcWCN1TBi96zNAtLrEErxL9MwxrSg14LckXo+X7mgP g9dxYsUmLLiEZ7sNeheOQ+9jqriS9AFUXRO5laBxuVePBlYMbE3TfZoiSRkbG6q8PmW7 qo1Q== X-Forwarded-Encrypted: i=1; AJvYcCXBfY3dgmQKemHhX+HoM1kkD40rr7uFiz3q/xOcDKf4YJnakItwF8j7l6CoPjhIqy/qkULV00gYT/6Cr3VL+sdQnlPAfO+zx1KmUSQt X-Gm-Message-State: AOJu0Yx2hELgJVdACe8etcAcOCOP+YA4pn7otSQF0SU/km0a2FzoBpj+ TP7Gxb+w2ZpPojun6AII166TlOLxOZVi4kSMuR+s9w4AHZZFu5n14Fl1vw4UWqLVcR5iX1GRyBZ 7X9c= X-Google-Smtp-Source: AGHT+IEUOmCzKpXiteK6r7fTuiEwyVLmZFa6bNiW2D8099v/kAcWiUtRw3i/QaJjtdfif4s5IWfWAw== X-Received: by 2002:a17:902:f78b:b0:1f8:70d7:d7ee with SMTP id d9443c01a7336-1f870d7dc6bmr4584665ad.46.1718374945100; Fri, 14 Jun 2024 07:22:25 -0700 (PDT) Received: from hsinchu26.internal.sifive.com (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1f855e559d9sm32522005ad.35.2024.06.14.07.22.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Jun 2024 07:22:24 -0700 (PDT) From: Zong Li To: joro@8bytes.org, will@kernel.org, robin.murphy@arm.com, tjeznach@rivosinc.com, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, jgg@ziepe.ca, kevin.tian@intel.com, linux-kernel@vger.kernel.org, iommu@lists.linux.dev, linux-riscv@lists.infradead.org Cc: Zong Li Subject: [RFC PATCH v2 06/10] iommu/riscv: support nested iommu for getting iommu hardware information Date: Fri, 14 Jun 2024 22:21:52 +0800 Message-Id: <20240614142156.29420-7-zong.li@sifive.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240614142156.29420-1-zong.li@sifive.com> References: <20240614142156.29420-1-zong.li@sifive.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This patch implements .hw_info operation and the related data structures for passing the IOMMU hardware capabilities for iommufd. Signed-off-by: Zong Li Reviewed-by: Jason Gunthorpe --- drivers/iommu/riscv/iommu.c | 20 ++++++++++++++++++++ include/uapi/linux/iommufd.h | 18 ++++++++++++++++++ 2 files changed, 38 insertions(+) diff --git a/drivers/iommu/riscv/iommu.c b/drivers/iommu/riscv/iommu.c index 45309bd096e5..2130106e421f 100644 --- a/drivers/iommu/riscv/iommu.c +++ b/drivers/iommu/riscv/iommu.c @@ -19,6 +19,7 @@ #include #include #include +#include =20 #include "../iommu-pages.h" #include "iommu-bits.h" @@ -1567,6 +1568,24 @@ static struct iommu_domain riscv_iommu_identity_doma= in =3D { } }; =20 +static void *riscv_iommu_hw_info(struct device *dev, u32 *length, u32 *typ= e) +{ + struct riscv_iommu_device *iommu =3D dev_to_iommu(dev); + struct iommu_hw_info_riscv_iommu *info; + + info =3D kzalloc(sizeof(*info), GFP_KERNEL); + if (!info) + return ERR_PTR(-ENOMEM); + + info->capability =3D iommu->caps; + info->fctl =3D riscv_iommu_readl(iommu, RISCV_IOMMU_REG_FCTL); + + *length =3D sizeof(*info); + *type =3D IOMMU_HW_INFO_TYPE_RISCV_IOMMU; + + return info; +} + static int riscv_iommu_device_domain_type(struct device *dev) { return 0; @@ -1644,6 +1663,7 @@ static void riscv_iommu_release_device(struct device = *dev) static const struct iommu_ops riscv_iommu_ops =3D { .pgsize_bitmap =3D SZ_4K, .of_xlate =3D riscv_iommu_of_xlate, + .hw_info =3D riscv_iommu_hw_info, .identity_domain =3D &riscv_iommu_identity_domain, .blocked_domain =3D &riscv_iommu_blocking_domain, .release_domain =3D &riscv_iommu_blocking_domain, diff --git a/include/uapi/linux/iommufd.h b/include/uapi/linux/iommufd.h index 1dfeaa2e649e..736f4408b5e0 100644 --- a/include/uapi/linux/iommufd.h +++ b/include/uapi/linux/iommufd.h @@ -475,15 +475,33 @@ struct iommu_hw_info_vtd { __aligned_u64 ecap_reg; }; =20 +/** + * struct iommu_hw_info_riscv_iommu - RISCV IOMMU hardware information + * + * @capability: Value of RISC-V IOMMU capability register defined in + * RISC-V IOMMU spec section 5.3 IOMMU capabilities + * @fctl: Value of RISC-V IOMMU feature control register defined in + * RISC-V IOMMU spec section 5.4 Features-control register + * + * Don't advertise ATS support to the guest because driver doesn't support= it. + */ +struct iommu_hw_info_riscv_iommu { + __aligned_u64 capability; + __u32 fctl; + __u32 __reserved; +}; + /** * enum iommu_hw_info_type - IOMMU Hardware Info Types * @IOMMU_HW_INFO_TYPE_NONE: Used by the drivers that do not report hardwa= re * info * @IOMMU_HW_INFO_TYPE_INTEL_VTD: Intel VT-d iommu info type + * @IOMMU_HW_INFO_TYPE_RISCV_IOMMU: RISC-V iommu info type */ enum iommu_hw_info_type { IOMMU_HW_INFO_TYPE_NONE, IOMMU_HW_INFO_TYPE_INTEL_VTD, + IOMMU_HW_INFO_TYPE_RISCV_IOMMU, }; =20 /** --=20 2.17.1 From nobody Wed Dec 17 13:54:13 2025 Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B9FBD3B1A4 for ; Fri, 14 Jun 2024 14:22:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718374950; cv=none; b=apd2uVo6M7CA9+raugk7WF1ElU/iGXNLFrWYcoBff6ufRAkoFn/tdgP5TUxfbxATCcw0Zy96whODrX+95X72CnX/jFPggJIA0YIu1Pg7nmQGwSCyEYsCmKzT9PcFj3D8vpx25G1qd8qkbGFaGsqyy0BB/K1h6h0NIKhBvnCsiiY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718374950; c=relaxed/simple; bh=0WtTHd7iSMjiPjoMS8n7GM1ddQK8hL6JoJsay6C2jRs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=bbsrPuYwZwqsTImwLUhlokIxgJH7Dp81C3xlhk57J8axes0Pgp6Tdh98Eg0voFqQVO1zfyEqnmzy/rh5Av3OKzuxBt6YfcNLAEfRs4nP2bV3VtU3Qo2/oxJFx56uGTLmhXkoRW9Ku8pLSKD9hqy3L2NEFTyHwKtAWPmu4G6aKxM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com; spf=pass smtp.mailfrom=sifive.com; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b=kAcxfEmZ; arc=none smtp.client-ip=209.85.214.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sifive.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="kAcxfEmZ" Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-1f4a5344ec7so16231045ad.1 for ; Fri, 14 Jun 2024 07:22:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1718374948; x=1718979748; darn=vger.kernel.org; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=icfonVjdOGiS4jBGMCDTwtNC3iVuGDFaRUw0zwHlFBc=; b=kAcxfEmZFcXn9bNNPjz/o5sQvj+qgNprGqYtPtC+YVn262DE2QSiHG+auIslF/yHxw 3jQvsiVbDU8ARY7EvoUt6ZUWZpMTHFyITSw6MXm0KU/JIhyGIiAL74fPIra30K1QxsOd D5s7gRYzueqC0H6inrRc/od+9CxA1NYSlXESvRUBbXa6MHwXIyGG8LF9eqJJK6D0Rz2l aQ3Q+hV9ejdSIKIGekA4b1WGmgreItduF2kz24kvp2UZ0vTkv6JLuWzbUoojaR2YrgN9 JET7MaBSggieIt97blMY8iVgcR1gQ5Eo7hZuRv+nYA5KzUDhSyWzAhutYHkG7KTVRZcW rX2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718374948; x=1718979748; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=icfonVjdOGiS4jBGMCDTwtNC3iVuGDFaRUw0zwHlFBc=; b=fNvBhMenYee4syZMWC52QSyjFjoM3iha4Ayhh46ppP8LPFq1PC7fTYmTaPMr9GEfsi rYGVPAMc1dLZb5afUdEA729CQvyjco8/iTAXqQAhnGBjYb5IJrTkQukf6Zq5FjrOZNwJ eK1uqjP7HCf7EaWCJrqdQ6ZH5X1Kd717vByRamSupTD7uom1uC4SmvNaHT/n356RRqsC QoWQkjehb1B6MhK/psjtIdEjNc0FCDhfxZMPrZE7RRs88tDJSH4AeThDtwhph1NBb0gH v+xTeOcBZlXQKurvFEWahMrFwoZSbNmuRB3zt/JMx/bcFGn5Q103RLvCwLTc3vTBFmjl f9nQ== X-Forwarded-Encrypted: i=1; AJvYcCW6Mwg0CvP3aDWXYkaLaXIhDxvpKHyLS1Frx84P8Bnc/cS+8SrWYhJSVMhIVrAF07PSUbQhtHiN0ht5ObT6osqOd8U6pWMeCQYN5da3 X-Gm-Message-State: AOJu0Yw3kL4mgqkcMW1/4b/fMJHGZ4f2Avo24yg3iJvCZO2t0IH8uwPm EvgsMLf8o26s5KTbMOBPPRK2A66mBGHTpy/7FXNaoflLE+L2QBlyay62W3FAwPM= X-Google-Smtp-Source: AGHT+IHErCU6r7FcqlhN+6YKmphLPYai+yLyt0MfvZY472OSF5t8ZLzOGhTYEkvrbQPRWasAsRJaag== X-Received: by 2002:a17:902:d2ca:b0:1f7:207f:7081 with SMTP id d9443c01a7336-1f84e1d39e7mr99383395ad.14.1718374947672; Fri, 14 Jun 2024 07:22:27 -0700 (PDT) Received: from hsinchu26.internal.sifive.com (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1f855e559d9sm32522005ad.35.2024.06.14.07.22.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Jun 2024 07:22:27 -0700 (PDT) From: Zong Li To: joro@8bytes.org, will@kernel.org, robin.murphy@arm.com, tjeznach@rivosinc.com, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, jgg@ziepe.ca, kevin.tian@intel.com, linux-kernel@vger.kernel.org, iommu@lists.linux.dev, linux-riscv@lists.infradead.org Cc: Zong Li Subject: [RFC PATCH v2 07/10] iommu/riscv: support nested iommu for creating domains owned by userspace Date: Fri, 14 Jun 2024 22:21:53 +0800 Message-Id: <20240614142156.29420-8-zong.li@sifive.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240614142156.29420-1-zong.li@sifive.com> References: <20240614142156.29420-1-zong.li@sifive.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This patch implements .domain_alloc_user operation for creating domains owend by userspace, e.g. through IOMMUFD. Add s2 domain for parent domain for second stage, s1 domain will be the first stage. Don't remove IOMMU private data of dev in blocked domain, because it holds the user data of device, which is used when attaching device into s1 domain. Signed-off-by: Zong Li --- drivers/iommu/riscv/iommu.c | 236 ++++++++++++++++++++++++++++++++++- include/uapi/linux/iommufd.h | 17 +++ 2 files changed, 252 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/riscv/iommu.c b/drivers/iommu/riscv/iommu.c index 2130106e421f..410b236e9b24 100644 --- a/drivers/iommu/riscv/iommu.c +++ b/drivers/iommu/riscv/iommu.c @@ -846,6 +846,8 @@ static int riscv_iommu_iodir_set_mode(struct riscv_iomm= u_device *iommu, =20 /* This struct contains protection domain specific IOMMU driver data. */ struct riscv_iommu_domain { + struct riscv_iommu_domain *s2; + struct riscv_iommu_device *iommu; struct iommu_domain domain; struct list_head bonds; spinlock_t lock; /* protect bonds list updates. */ @@ -863,6 +865,7 @@ struct riscv_iommu_domain { /* Private IOMMU data for managed devices, dev_iommu_priv_* */ struct riscv_iommu_info { struct riscv_iommu_domain *domain; + struct riscv_iommu_dc dc_user; }; =20 /* @@ -1532,7 +1535,6 @@ static int riscv_iommu_attach_blocking_domain(struct = iommu_domain *iommu_domain, /* Make device context invalid, translation requests will fault w/ #258 */ riscv_iommu_iodir_update(iommu, dev, &dc); riscv_iommu_bond_unlink(info->domain, dev); - info->domain =3D NULL; =20 return 0; } @@ -1568,6 +1570,237 @@ static struct iommu_domain riscv_iommu_identity_dom= ain =3D { } }; =20 +/** + * Nested IOMMU operations + */ + +static int riscv_iommu_attach_dev_nested(struct iommu_domain *domain, stru= ct device *dev) +{ + struct riscv_iommu_domain *riscv_domain =3D iommu_domain_to_riscv(domain); + struct riscv_iommu_device *iommu =3D dev_to_iommu(dev); + struct riscv_iommu_info *info =3D dev_iommu_priv_get(dev); + + /* + * Add bond to the new domain's list, but don't unlink in current domain. + * We need to flush entries in stage-2 page table by iterating the list. + */ + if (riscv_iommu_bond_link(riscv_domain, dev)) + return -ENOMEM; + + riscv_iommu_iotlb_inval(riscv_domain, 0, ULONG_MAX); + info->dc_user.ta |=3D RISCV_IOMMU_PC_TA_V; + riscv_iommu_iodir_update(iommu, dev, &info->dc_user); + + info->domain =3D riscv_domain; + + return 0; +} + +static void riscv_iommu_domain_free_nested(struct iommu_domain *domain) +{ + struct riscv_iommu_domain *riscv_domain =3D iommu_domain_to_riscv(domain); + struct riscv_iommu_bond *bond; + + /* Unlink bond in s2 domain, because we link bond both on s1 and s2 domai= n */ + list_for_each_entry_rcu(bond, &riscv_domain->s2->bonds, list) + riscv_iommu_bond_unlink(riscv_domain->s2, bond->dev); + + if ((int)riscv_domain->pscid > 0) + ida_free(&riscv_iommu_pscids, riscv_domain->pscid); + + kfree(riscv_domain); +} + +static const struct iommu_domain_ops riscv_iommu_nested_domain_ops =3D { + .attach_dev =3D riscv_iommu_attach_dev_nested, + .free =3D riscv_iommu_domain_free_nested, +}; + +static int +riscv_iommu_get_dc_user(struct device *dev, struct iommu_hwpt_riscv_iommu = *user_arg) +{ + struct iommu_fwspec *fwspec =3D dev_iommu_fwspec_get(dev); + struct riscv_iommu_device *iommu =3D dev_to_iommu(dev); + struct riscv_iommu_info *info =3D dev_iommu_priv_get(dev); + struct riscv_iommu_dc dc; + struct riscv_iommu_fq_record event; + u64 dc_len =3D sizeof(struct riscv_iommu_dc) >> + (!(iommu->caps & RISCV_IOMMU_CAPABILITIES_MSI_FLAT)); + u64 event_len =3D sizeof(struct riscv_iommu_fq_record); + void __user *event_user =3D NULL; + + for (int i =3D 0; i < fwspec->num_ids; i++) { + event.hdr =3D + FIELD_PREP(RISCV_IOMMU_FQ_HDR_CAUSE, RISCV_IOMMU_FQ_CAUSE_DDT_INVALID) | + FIELD_PREP(RISCV_IOMMU_FQ_HDR_DID, fwspec->ids[i]); + + /* Sanity check DC of stage-1 from user data */ + if (!user_arg->out_event_uptr || user_arg->event_len !=3D event_len) + return -EINVAL; + + event_user =3D u64_to_user_ptr(user_arg->out_event_uptr); + + if (!user_arg->dc_uptr || user_arg->dc_len !=3D dc_len) + return -EINVAL; + + if (copy_from_user(&dc, u64_to_user_ptr(user_arg->dc_uptr), dc_len)) + return -EFAULT; + + if (!(dc.tc & RISCV_IOMMU_DDTE_V)) { + dev_dbg(dev, "Invalid DDT from user data\n"); + if (copy_to_user(event_user, &event, event_len)) + return -EFAULT; + } + + if (!dc.fsc || dc.iohgatp) { + dev_dbg(dev, "Wrong page table from user data\n"); + if (copy_to_user(event_user, &event, event_len)) + return -EFAULT; + } + + /* Save DC of stage-1 from user data */ + memcpy(&info->dc_user, + riscv_iommu_get_dc(iommu, fwspec->ids[i]), + sizeof(struct riscv_iommu_dc)); + info->dc_user.fsc =3D dc.fsc; + } + + return 0; +} + +static struct iommu_domain * +riscv_iommu_domain_alloc_nested(struct device *dev, + struct iommu_domain *parent, + const struct iommu_user_data *user_data) +{ + struct riscv_iommu_domain *s2_domain =3D iommu_domain_to_riscv(parent); + struct riscv_iommu_domain *s1_domain; + struct riscv_iommu_device *iommu =3D dev_to_iommu(dev); + struct iommu_hwpt_riscv_iommu arg; + int ret, va_bits; + + if (user_data->type !=3D IOMMU_HWPT_DATA_RISCV_IOMMU) + return ERR_PTR(-EOPNOTSUPP); + + if (parent->type !=3D IOMMU_DOMAIN_UNMANAGED) + return ERR_PTR(-EINVAL); + + ret =3D iommu_copy_struct_from_user(&arg, + user_data, + IOMMU_HWPT_DATA_RISCV_IOMMU, + out_event_uptr); + if (ret) + return ERR_PTR(ret); + + s1_domain =3D kzalloc(sizeof(*s1_domain), GFP_KERNEL); + if (!s1_domain) + return ERR_PTR(-ENOMEM); + + spin_lock_init(&s1_domain->lock); + INIT_LIST_HEAD_RCU(&s1_domain->bonds); + + s1_domain->pscid =3D ida_alloc_range(&riscv_iommu_pscids, 1, + RISCV_IOMMU_MAX_PSCID, GFP_KERNEL); + if (s1_domain->pscid < 0) { + iommu_free_page(s1_domain->pgd_root); + kfree(s1_domain); + return ERR_PTR(-ENOMEM); + } + + /* Get device context of stage-1 from user*/ + ret =3D riscv_iommu_get_dc_user(dev, &arg); + if (ret) { + kfree(s1_domain); + return ERR_PTR(-EINVAL); + } + + if (!iommu) { + va_bits =3D VA_BITS; + } else if (iommu->caps & RISCV_IOMMU_CAPABILITIES_SV57) { + va_bits =3D 57; + } else if (iommu->caps & RISCV_IOMMU_CAPABILITIES_SV48) { + va_bits =3D 48; + } else if (iommu->caps & RISCV_IOMMU_CAPABILITIES_SV39) { + va_bits =3D 39; + } else { + dev_err(dev, "cannot find supported page table mode\n"); + return ERR_PTR(-ENODEV); + } + + /* + * The ops->domain_alloc_user could be directly called by the iommufd cor= e, + * instead of iommu core. So, this function need to set the default value= of + * following data member: + * - domain->pgsize_bitmap + * - domain->geometry + * - domain->type + * - domain->ops + */ + s1_domain->s2 =3D s2_domain; + s1_domain->iommu =3D iommu; + s1_domain->domain.type =3D IOMMU_DOMAIN_NESTED; + s1_domain->domain.ops =3D &riscv_iommu_nested_domain_ops; + s1_domain->domain.pgsize_bitmap =3D SZ_4K; + s1_domain->domain.geometry.aperture_start =3D 0; + s1_domain->domain.geometry.aperture_end =3D DMA_BIT_MASK(va_bits - 1); + s1_domain->domain.geometry.force_aperture =3D true; + + return &s1_domain->domain; +} + +static struct iommu_domain * +riscv_iommu_domain_alloc_user(struct device *dev, u32 flags, + struct iommu_domain *parent, + const struct iommu_user_data *user_data) +{ + struct iommu_domain *domain; + struct riscv_iommu_domain *riscv_domain; + + /* Allocate stage-1 domain if it has stage-2 parent domain */ + if (parent) + return riscv_iommu_domain_alloc_nested(dev, parent, user_data); + + if (flags & ~((IOMMU_HWPT_ALLOC_NEST_PARENT | IOMMU_HWPT_ALLOC_DIRTY_TRAC= KING))) + return ERR_PTR(-EOPNOTSUPP); + + if (user_data) + return ERR_PTR(-EINVAL); + + /* domain_alloc_user op needs to be fully initialized */ + domain =3D iommu_domain_alloc(dev->bus); + if (!domain) + return ERR_PTR(-ENOMEM); + + /* + * We assume that nest-parent or g-stage only will come here + * TODO: Shadow page table doesn't be supported now. + * We currently can't distinguish g-stage and shadow + * page table here. Shadow page table shouldn't be + * put at stage-2. + */ + riscv_domain =3D iommu_domain_to_riscv(domain); + + /* pgd_root may be allocated in .domain_alloc_paging */ + if (riscv_domain->pgd_root) + iommu_free_page(riscv_domain->pgd_root); + + riscv_domain->pgd_root =3D iommu_alloc_pages_node(riscv_domain->numa_node, + GFP_KERNEL_ACCOUNT, + 2); + if (!riscv_domain->pgd_root) + return ERR_PTR(-ENOMEM); + + riscv_domain->gscid =3D ida_alloc_range(&riscv_iommu_gscids, 1, + RISCV_IOMMU_MAX_GSCID, GFP_KERNEL); + if (riscv_domain->gscid < 0) { + iommu_free_pages(riscv_domain->pgd_root, 2); + kfree(riscv_domain); + return ERR_PTR(-ENOMEM); + } + + return domain; +} + static void *riscv_iommu_hw_info(struct device *dev, u32 *length, u32 *typ= e) { struct riscv_iommu_device *iommu =3D dev_to_iommu(dev); @@ -1668,6 +1901,7 @@ static const struct iommu_ops riscv_iommu_ops =3D { .blocked_domain =3D &riscv_iommu_blocking_domain, .release_domain =3D &riscv_iommu_blocking_domain, .domain_alloc_paging =3D riscv_iommu_alloc_paging_domain, + .domain_alloc_user =3D riscv_iommu_domain_alloc_user, .def_domain_type =3D riscv_iommu_device_domain_type, .device_group =3D riscv_iommu_device_group, .probe_device =3D riscv_iommu_probe_device, diff --git a/include/uapi/linux/iommufd.h b/include/uapi/linux/iommufd.h index 736f4408b5e0..514463fe85d3 100644 --- a/include/uapi/linux/iommufd.h +++ b/include/uapi/linux/iommufd.h @@ -390,14 +390,31 @@ struct iommu_hwpt_vtd_s1 { __u32 __reserved; }; =20 +/** + * struct iommu_hwpt_riscv_iommu - RISCV IOMMU stage-1 device context table + * info (IOMMU_HWPT_TYPE_RISCV_IOMMU) + * @dc_len: Length of device context + * @dc_uptr: User pointer to the address of device context + * @event_len: Length of an event record + * @out_event_uptr: User pointer to the address of event record + */ +struct iommu_hwpt_riscv_iommu { + __aligned_u64 dc_len; + __aligned_u64 dc_uptr; + __aligned_u64 event_len; + __aligned_u64 out_event_uptr; +}; + /** * enum iommu_hwpt_data_type - IOMMU HWPT Data Type * @IOMMU_HWPT_DATA_NONE: no data * @IOMMU_HWPT_DATA_VTD_S1: Intel VT-d stage-1 page table + * @IOMMU_HWPT_DATA_RISCV_IOMMU: RISC-V IOMMU device context table */ enum iommu_hwpt_data_type { IOMMU_HWPT_DATA_NONE, IOMMU_HWPT_DATA_VTD_S1, + IOMMU_HWPT_DATA_RISCV_IOMMU, }; =20 /** --=20 2.17.1 From nobody Wed Dec 17 13:54:13 2025 Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3F8683A260 for ; Fri, 14 Jun 2024 14:22:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718374952; cv=none; b=UmlGcmxmtsp5d5ss8xfCUXztXukE/HYmZLayUg2hRb84xPsxJEHmmieUHQmMPM83HTo8ZK9S+v6IqfvNVDW2rNOJUxCQU/8jqT9SMeJeLksPUO24OlzBzq4YET8iqCA/d+M26R3Y6lUDPKxrwqR4HDvELI4sgARjjSHknkVYBww= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718374952; c=relaxed/simple; bh=AMjRqApdC6JhSqc5cTc8FRbsioFIOI+eQK8ee2g7rO4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=opH2hIFE00gAKegeWLLvsnG6MFOGH2R78zudpJNpLDv2TDyRdg8mZn4zA4zdlNAXgrnse6/MlPLnB5CFVVbTuypsaYeMneRY7U5lx/T/ny1yZmqVV+JjaUmyyn5c/gH5wNDF3PCxlX27cgAXmaLOmEPThzDmZSb0vm9C3PjeR1E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com; spf=pass smtp.mailfrom=sifive.com; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b=DdioKCi6; arc=none smtp.client-ip=209.85.214.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sifive.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="DdioKCi6" Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-1f480624d0fso17252235ad.1 for ; Fri, 14 Jun 2024 07:22:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1718374950; x=1718979750; darn=vger.kernel.org; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=uqErywjP/CjLC2eaDMmEQxFN9AljnO4yCx1Hy/yfHnA=; b=DdioKCi63zUMgI2p3dSNdP/2KIdGsVvIWVw98IziwRv2rcw1W0AdfYQOunweqGGZil H4XJOHq866sdJnbtnYhj95PRHxzRnd4LRDaDP7TTNQeDh+KqpDOVZHEu8HL/62uaijGk 2XnEaeuHEb9ZbS7kvpLlZPqfZ5TTd9jYZMiZ3yzSa9AapMlTscOcTQFFwkaxP+btYGZv AVu47BEMdeF377bSEZCyCaMOoISQGfxv8UCGsjShP5O/F0QGqxtB8ipSYC1+n616xOmS 8fJCX2Vga3EvuIYN21V/muMFyYgtx/Wqw9xXe4xp2wFpphBd0Cj538RFusv/BEPL/2Rb vtcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718374950; x=1718979750; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=uqErywjP/CjLC2eaDMmEQxFN9AljnO4yCx1Hy/yfHnA=; b=srtZZ3bP3Is1xHEJtibrhiDqzuo4zWqqn2NmnyVzB8xZZZIIK36Up62nlJFEOZtkrE 4mjDpwY8EGY0Mify445R+iHeaNDBWlqZWP6XVtHnZlPb+xX6EuhL5YUQRZSuUcavuv6W pxCOlZhO4tr7YLOMqraBJ2mL+qgzmZp76rZPy/vrcPEuCC83qD0dtg4szBapwg0I8nqB ci/XzA+tUSa2Iw0XVlEVjHqFVcjIf8xBANavBuxaleGuHsfZFHlet+EluT9k3pVESLAC 9Uco2pqXT79AOaDR6k5A5dObVR9+fS8YJby93reqXdj2Jjag6ZSYXoy5Zf2EHOXaAk9N mxLA== X-Forwarded-Encrypted: i=1; AJvYcCU62hZmdCiT+en5fvTo+JfvvgsW2bAav4MiAXTb215VJXDNcfWg2a4QMQahVN5y3bBcQd/GjnDBPZyyKMtnt2m2NaYLEsqG070h82kA X-Gm-Message-State: AOJu0YwS5OGdIQuBwlcrQcNpX67iikxx14/nBAWSvHuA4MLS82OUSyP5 +5IZ5nTmP8yV/gNyC03ZHFf314bqW8wnRjwT4QXv6b3x4HcF+ctYY3atnF6OhUk= X-Google-Smtp-Source: AGHT+IHSez2BcKKbBZwd377qNguyZGbzCpuJqffZQYRK1qpXa16+hD0f3KlRelSKzLQGJUtI0sNeSg== X-Received: by 2002:a17:903:1c4:b0:1f7:1525:ddfc with SMTP id d9443c01a7336-1f8625d9e30mr34295185ad.20.1718374950324; Fri, 14 Jun 2024 07:22:30 -0700 (PDT) Received: from hsinchu26.internal.sifive.com (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1f855e559d9sm32522005ad.35.2024.06.14.07.22.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Jun 2024 07:22:29 -0700 (PDT) From: Zong Li To: joro@8bytes.org, will@kernel.org, robin.murphy@arm.com, tjeznach@rivosinc.com, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, jgg@ziepe.ca, kevin.tian@intel.com, linux-kernel@vger.kernel.org, iommu@lists.linux.dev, linux-riscv@lists.infradead.org Cc: Zong Li Subject: [RFC PATCH v2 08/10] iommu/riscv: support nested iommu for flushing cache Date: Fri, 14 Jun 2024 22:21:54 +0800 Message-Id: <20240614142156.29420-9-zong.li@sifive.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240614142156.29420-1-zong.li@sifive.com> References: <20240614142156.29420-1-zong.li@sifive.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This patch implements cache_invalidate_user operation for the userspace to flush the hardware caches for a nested domain through iommufd. Signed-off-by: Zong Li --- drivers/iommu/riscv/iommu.c | 90 ++++++++++++++++++++++++++++++++++-- include/uapi/linux/iommufd.h | 11 +++++ 2 files changed, 97 insertions(+), 4 deletions(-) diff --git a/drivers/iommu/riscv/iommu.c b/drivers/iommu/riscv/iommu.c index 410b236e9b24..d08eb0a2939e 100644 --- a/drivers/iommu/riscv/iommu.c +++ b/drivers/iommu/riscv/iommu.c @@ -1587,8 +1587,9 @@ static int riscv_iommu_attach_dev_nested(struct iommu= _domain *domain, struct dev if (riscv_iommu_bond_link(riscv_domain, dev)) return -ENOMEM; =20 - riscv_iommu_iotlb_inval(riscv_domain, 0, ULONG_MAX); - info->dc_user.ta |=3D RISCV_IOMMU_PC_TA_V; + if (riscv_iommu_bond_link(info->domain, dev)) + return -ENOMEM; + riscv_iommu_iodir_update(iommu, dev, &info->dc_user); =20 info->domain =3D riscv_domain; @@ -1611,13 +1612,92 @@ static void riscv_iommu_domain_free_nested(struct i= ommu_domain *domain) kfree(riscv_domain); } =20 +static int riscv_iommu_fix_user_cmd(struct riscv_iommu_command *cmd, + unsigned int pscid, unsigned int gscid) +{ + u32 opcode =3D FIELD_GET(RISCV_IOMMU_CMD_OPCODE, cmd->dword0); + + switch (opcode) { + case RISCV_IOMMU_CMD_IOTINVAL_OPCODE: + u32 func =3D FIELD_GET(RISCV_IOMMU_CMD_FUNC, cmd->dword0); + + if (func !=3D RISCV_IOMMU_CMD_IOTINVAL_FUNC_GVMA && + func !=3D RISCV_IOMMU_CMD_IOTINVAL_FUNC_VMA) { + pr_warn("The IOTINVAL function: 0x%x is not supported\n", + func); + return -EOPNOTSUPP; + } + + if (func =3D=3D RISCV_IOMMU_CMD_IOTINVAL_FUNC_GVMA) { + cmd->dword0 &=3D ~RISCV_IOMMU_CMD_FUNC; + cmd->dword0 |=3D FIELD_PREP(RISCV_IOMMU_CMD_FUNC, + RISCV_IOMMU_CMD_IOTINVAL_FUNC_VMA); + } + + cmd->dword0 &=3D ~(RISCV_IOMMU_CMD_IOTINVAL_PSCID | + RISCV_IOMMU_CMD_IOTINVAL_GSCID); + riscv_iommu_cmd_inval_set_pscid(cmd, pscid); + riscv_iommu_cmd_inval_set_gscid(cmd, gscid); + break; + case RISCV_IOMMU_CMD_IODIR_OPCODE: + /* + * Ensure the device ID is right. We expect that VMM has + * transferred the device ID to host's from guest's. + */ + break; + default: + return -EOPNOTSUPP; + } + + return 0; +} + +static int riscv_iommu_cache_invalidate_user(struct iommu_domain *domain, + struct iommu_user_data_array *array) +{ + struct riscv_iommu_domain *riscv_domain =3D iommu_domain_to_riscv(domain); + struct iommu_hwpt_riscv_iommu_invalidate inv_info; + int ret, index; + + if (array->type !=3D IOMMU_HWPT_INVALIDATE_DATA_RISCV_IOMMU) { + ret =3D -EINVAL; + goto out; + } + + for (index =3D 0; index < array->entry_num; index++) { + ret =3D iommu_copy_struct_from_user_array(&inv_info, array, + IOMMU_HWPT_INVALIDATE_DATA_RISCV_IOMMU, + index, cmd); + if (ret) + break; + + ret =3D riscv_iommu_fix_user_cmd((struct riscv_iommu_command *)inv_info.= cmd, + riscv_domain->pscid, + riscv_domain->s2->gscid); + if (ret =3D=3D -EOPNOTSUPP) + continue; + + riscv_iommu_cmd_send(riscv_domain->iommu, + (struct riscv_iommu_command *)inv_info.cmd); + riscv_iommu_cmd_sync(riscv_domain->iommu, + RISCV_IOMMU_IOTINVAL_TIMEOUT); + } + +out: + array->entry_num =3D index; + + return ret; +} + static const struct iommu_domain_ops riscv_iommu_nested_domain_ops =3D { .attach_dev =3D riscv_iommu_attach_dev_nested, .free =3D riscv_iommu_domain_free_nested, + .cache_invalidate_user =3D riscv_iommu_cache_invalidate_user, }; =20 static int -riscv_iommu_get_dc_user(struct device *dev, struct iommu_hwpt_riscv_iommu = *user_arg) +riscv_iommu_get_dc_user(struct device *dev, struct iommu_hwpt_riscv_iommu = *user_arg, + struct riscv_iommu_domain *s1_domain) { struct iommu_fwspec *fwspec =3D dev_iommu_fwspec_get(dev); struct riscv_iommu_device *iommu =3D dev_to_iommu(dev); @@ -1663,6 +1743,8 @@ riscv_iommu_get_dc_user(struct device *dev, struct io= mmu_hwpt_riscv_iommu *user_ riscv_iommu_get_dc(iommu, fwspec->ids[i]), sizeof(struct riscv_iommu_dc)); info->dc_user.fsc =3D dc.fsc; + info->dc_user.ta =3D FIELD_PREP(RISCV_IOMMU_PC_TA_PSCID, s1_domain->psci= d) | + RISCV_IOMMU_PC_TA_V; } =20 return 0; @@ -1708,7 +1790,7 @@ riscv_iommu_domain_alloc_nested(struct device *dev, } =20 /* Get device context of stage-1 from user*/ - ret =3D riscv_iommu_get_dc_user(dev, &arg); + ret =3D riscv_iommu_get_dc_user(dev, &arg, s1_domain); if (ret) { kfree(s1_domain); return ERR_PTR(-EINVAL); diff --git a/include/uapi/linux/iommufd.h b/include/uapi/linux/iommufd.h index 514463fe85d3..876cbe980a42 100644 --- a/include/uapi/linux/iommufd.h +++ b/include/uapi/linux/iommufd.h @@ -653,9 +653,11 @@ struct iommu_hwpt_get_dirty_bitmap { * enum iommu_hwpt_invalidate_data_type - IOMMU HWPT Cache Invalidation * Data Type * @IOMMU_HWPT_INVALIDATE_DATA_VTD_S1: Invalidation data for VTD_S1 + * @IOMMU_HWPT_INVALIDATE_DATA_RISCV_IOMMU: Invalidation data for RISCV_IO= MMU */ enum iommu_hwpt_invalidate_data_type { IOMMU_HWPT_INVALIDATE_DATA_VTD_S1, + IOMMU_HWPT_INVALIDATE_DATA_RISCV_IOMMU, }; =20 /** @@ -694,6 +696,15 @@ struct iommu_hwpt_vtd_s1_invalidate { __u32 __reserved; }; =20 +/** + * struct iommu_hwpt_riscv_iommu_invalidate - RISCV IOMMU cache invalidati= on + * (IOMMU_HWPT_TYPE_RISCV_IOMMU) + * @cmd: An array holds a command for cache invalidation + */ +struct iommu_hwpt_riscv_iommu_invalidate { + __aligned_u64 cmd[2]; +}; + /** * struct iommu_hwpt_invalidate - ioctl(IOMMU_HWPT_INVALIDATE) * @size: sizeof(struct iommu_hwpt_invalidate) --=20 2.17.1 From nobody Wed Dec 17 13:54:13 2025 Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6846245C1C for ; Fri, 14 Jun 2024 14:22:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718374954; cv=none; b=YBz4UD83+7niSp/I26vBRPoIsIHGNooazwjBrwzIrn16InfM9syyynvueval1bcaqYdVFSaeCw1P9fExuzkantt7yiFIuIKPZoWb6GGP0Cs4dgdcoEfFlohPPcobqVV/6luAH8E0xsJh3baXHRY8smuneXHRFcmgxtq0cwfO03s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718374954; c=relaxed/simple; bh=7Yb/SMMlZn1tJaUWC6e7+SVPbq9JvQRpG9Rr2Zf6yls=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=tSgocYckwoR79WRgrmnbuxNWFuradj0OH4VqqBpl47CeoFUllwRiUiKMq0xmgE+7CuMILydhEFmEAJuUFSWZXw43u1RgJjC7RW9wWmVRmQ0Rr3Z+IX4Og4YmUcWp6rJWflZEbaSM1tUefirRPiPhHRmEDhl3mAypSaY01/ePu+Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com; spf=pass smtp.mailfrom=sifive.com; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b=dDYAguRs; arc=none smtp.client-ip=209.85.214.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sifive.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="dDYAguRs" Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-1f717b3f2d8so24110005ad.1 for ; Fri, 14 Jun 2024 07:22:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1718374953; x=1718979753; darn=vger.kernel.org; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=BxtO7IOLUtPSQ+XdcXJ6pDuSrhddnSGqhgchDX3PwwI=; b=dDYAguRsw8WcaLPhDDOYTwPKPYmtkKlK2A+JfqrrJpzbeiMoaimHR7Jra1LJ9QI9eM ks6gfiAmHgv5bHqoA/trlNQn0Utxp69J8U4T7fuPH63pmca6pcBdDQ99kRM/5+AjjKgV msZmYp0xmu1OgyatdmIwnbf8Oo1WAmPRDp6tCqvuLCdEpk73dD76kTdPj75fdogK3IpB QTZTPl9AJ1QwrH95ta4S52swvFjLFBanAAjPkWw8uypbCB51s+VbUFTjj1YmfHVpo/Cc GY6IoiMvciKlcik8QZW8lAk2jiymWnh+n/+wUGdXKGCuQgWnvZa0QExiMhnBhLevGqZs TycQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718374953; x=1718979753; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=BxtO7IOLUtPSQ+XdcXJ6pDuSrhddnSGqhgchDX3PwwI=; b=cYam5ZlqYNOr9cLnqxXkDIOlqUgycH3RSX7zKxMwFzs3RNYtDrcs38TPgsr2r9INU1 MO13sJ/CDqZSoJT5v6HLzmZ+A9sO6yfeBBr4WfdLpwg0oZC7L9/Xr+mCKSQCJa3CIGfm ho5V8Xk2WaRjG3Cs8jF7ZWZaO+PTPowKgIdY1EPMUN4Wf13v1ZPT3vUtPUFyRpa4VRYE D0WtESMrGiiVViu3mZlGgzWkKFubfKgDPl86xJEU+7nip2lJiOmoQKvbXMc5SKLfsi6j TZXiGS+FAqaBO1b///rsWRkR9Ll6VHE8FTDoQGQfTXE2fRVK8RhX4FS35FxyJaNp/b9N bQiw== X-Forwarded-Encrypted: i=1; AJvYcCUxec5GIzaqeJ/SZKnh395QNDgydztICYVa6pZ8kT+qhjAmuLTnzD9QtG3oBXks5VU5847iOpBIjIy7jryfDRZF0iGOSxEYtu6db02Z X-Gm-Message-State: AOJu0Yx8gay2HJqN7SyChmPZxw9l+16LLaMFF8PSs1fJRe8zyVJrE34s 6ilFObRGQJB+m04xXtNeTeYD+wrONkaJ+n2PlBiDJoZucHLvBWcOElttnsSpEek= X-Google-Smtp-Source: AGHT+IG11s4ZHx3p3pdh0/8168TEMxbmtzHZ+b6vnWRUAkFpilwyNQyd4ZLsjbcQgl8XVHvD/vsepw== X-Received: by 2002:a17:902:cccc:b0:1f7:126:5ba7 with SMTP id d9443c01a7336-1f84e42e7bamr92287235ad.21.1718374952848; Fri, 14 Jun 2024 07:22:32 -0700 (PDT) Received: from hsinchu26.internal.sifive.com (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1f855e559d9sm32522005ad.35.2024.06.14.07.22.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Jun 2024 07:22:32 -0700 (PDT) From: Zong Li To: joro@8bytes.org, will@kernel.org, robin.murphy@arm.com, tjeznach@rivosinc.com, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, jgg@ziepe.ca, kevin.tian@intel.com, linux-kernel@vger.kernel.org, iommu@lists.linux.dev, linux-riscv@lists.infradead.org Cc: Nicolin Chen Subject: [RFC PATCH v2 09/10] iommu/dma: Support MSIs through nested domains Date: Fri, 14 Jun 2024 22:21:55 +0800 Message-Id: <20240614142156.29420-10-zong.li@sifive.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240614142156.29420-1-zong.li@sifive.com> References: <20240614142156.29420-1-zong.li@sifive.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Robin Murphy Currently, iommu-dma is the only place outside of IOMMUFD and drivers which might need to be aware of the stage 2 domain encapsulated within a nested domain. This would be in the legacy-VFIO-style case where we're using host-managed MSIs with an identity mapping at stage 1, where it is the underlying stage 2 domain which owns an MSI cookie and holds the corresponding dynamic mappings. Hook up the new op to resolve what we need from a nested domain. Signed-off-by: Robin Murphy Signed-off-by: Nicolin Chen --- drivers/iommu/dma-iommu.c | 18 ++++++++++++++++-- include/linux/iommu.h | 4 ++++ 2 files changed, 20 insertions(+), 2 deletions(-) diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index f731e4b2a417..d4235bb0a427 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -1806,6 +1806,20 @@ static struct iommu_dma_msi_page *iommu_dma_get_msi_= page(struct device *dev, return NULL; } =20 +/* + * Nested domains may not have an MSI cookie or accept mappings, but they = may + * be related to a domain which does, so we let them tell us what they nee= d. + */ +static struct iommu_domain *iommu_dma_get_msi_mapping_domain(struct device= *dev) +{ + struct iommu_domain *domain =3D iommu_get_domain_for_dev(dev); + + if (domain && domain->type =3D=3D IOMMU_DOMAIN_NESTED && + domain->ops->get_msi_mapping_domain) + domain =3D domain->ops->get_msi_mapping_domain(domain); + return domain; +} + /** * iommu_dma_prepare_msi() - Map the MSI page in the IOMMU domain * @desc: MSI descriptor, will store the MSI page @@ -1816,7 +1830,7 @@ static struct iommu_dma_msi_page *iommu_dma_get_msi_p= age(struct device *dev, int iommu_dma_prepare_msi(struct msi_desc *desc, phys_addr_t msi_addr) { struct device *dev =3D msi_desc_to_dev(desc); - struct iommu_domain *domain =3D iommu_get_domain_for_dev(dev); + struct iommu_domain *domain =3D iommu_dma_get_msi_mapping_domain(dev); struct iommu_dma_msi_page *msi_page; static DEFINE_MUTEX(msi_prepare_lock); /* see below */ =20 @@ -1849,7 +1863,7 @@ int iommu_dma_prepare_msi(struct msi_desc *desc, phys= _addr_t msi_addr) void iommu_dma_compose_msi_msg(struct msi_desc *desc, struct msi_msg *msg) { struct device *dev =3D msi_desc_to_dev(desc); - const struct iommu_domain *domain =3D iommu_get_domain_for_dev(dev); + const struct iommu_domain *domain =3D iommu_dma_get_msi_mapping_domain(de= v); const struct iommu_dma_msi_page *msi_page; =20 msi_page =3D msi_desc_get_iommu_cookie(desc); diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 7bc8dff7cf6d..400df9ae7012 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -629,6 +629,8 @@ struct iommu_ops { * @enable_nesting: Enable nesting * @set_pgtable_quirks: Set io page table quirks (IO_PGTABLE_QUIRK_*) * @free: Release the domain after use. + * @get_msi_mapping_domain: Return the related iommu_domain that should ho= ld the + * MSI cookie and accept mapping(s). */ struct iommu_domain_ops { int (*attach_dev)(struct iommu_domain *domain, struct device *dev); @@ -659,6 +661,8 @@ struct iommu_domain_ops { unsigned long quirks); =20 void (*free)(struct iommu_domain *domain); + struct iommu_domain * + (*get_msi_mapping_domain)(struct iommu_domain *domain); }; =20 /** --=20 2.17.1 From nobody Wed Dec 17 13:54:13 2025 Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 090D949630 for ; Fri, 14 Jun 2024 14:22:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718374957; cv=none; b=gP8y4nHFpb2oQiXokSwtDsXhUxQUNKWHKpROJ7tVsPAqFc0nwjksZFK8Q4BLQfSZ7qoFy561ts5S8cQug6B3AiR00U37etwrIeVI3WB+2mefDmDS3t50aWhGlzgGwUuebFkJKjhr7slXcVYNM3TAkunzmKf5IISEeVBXHNMOxx8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718374957; c=relaxed/simple; bh=Q7a++o7eiwUm8M0irLfARJWMrYHyrc7Ai/QKurMYIDk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=k7uoU/xDyx9rCKVmjS9pm2CIMukRqyO9pEWYR7KtdBfTx0QoTpo9U/GveT8u3ce8A24dcnoIzndc7OCCIWTb9V8A5wc+Xp9LL8/MIzN6V3EAeMN7Q6/8BZ8EYvLLJZ+KccVb0U6vDzWhrP8I6h9cdkEyVlH1C56QD4ISs/RT9w8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com; spf=pass smtp.mailfrom=sifive.com; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b=Sdrp3oKT; arc=none smtp.client-ip=209.85.214.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sifive.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="Sdrp3oKT" Received: by mail-pl1-f178.google.com with SMTP id d9443c01a7336-1f44b594deeso21678105ad.2 for ; Fri, 14 Jun 2024 07:22:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1718374955; x=1718979755; darn=vger.kernel.org; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=wDCzcwoY+7Urqf0z9zEneV8CU/qabdLYOlq0L2NAoPg=; b=Sdrp3oKT+qzuCM2pbbA7+MtRXJELPBufR3k0gYoVbxidM49WPLQTkiMebW6rlgQfDL zukX8zs+5evigM7FdXJU1k5VE86f4twOq7ZJnkaN5k4Dx9DTu8kX4f5FiNAWiSXUz/Rh sewN47NzCAoVu2tzEmoPKI4BmvPem34v/LnWxq4TrZ59Yd1u3jI52NEqnYvjNGRNRaRJ 8PjkVt3ezviUxHfZIgcp1l+AdNPELBfed0oAGkQnupj5144JRiKmT0oWPEZYl8pVohAP rNiUsLMBz9hptK7lFUWb6VqgROAUj0FlWic3fTEONxCcZ93BmvnMIYsIEArAEiI/D6F0 9jqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718374955; x=1718979755; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=wDCzcwoY+7Urqf0z9zEneV8CU/qabdLYOlq0L2NAoPg=; b=DPXXLtHUbRPTc77hnr+nn7MsexSubm8e+5/7IkQUiVPRCRnG00KCU6xZaIRd/QKYFE 9VPkqDqdv86iACGq6sXVmfq6J/wHaAUVWrR5l9ObJr1gNDwjlLZ7Dfy9KhIOz7Z3H4Hw SPyxWsFqCMGKZnNzosvhjIXT2b2hYqL2K900TU+9QuiCtwLs6XBPF49n0z2ED1dnYY3g kh/bQ8ymAWtgTmrzi3TfaleTE9aq2feYjRILqMIGc2YkceaU8KWPZIaQCtwYSEp07L04 9a/KQGo4ZHFens1jzsc8KZ4sqDSgXG0wsdZTTATaZ8IO7ABXdtKmSBcYY5rdOcFq3ZKf C+jw== X-Forwarded-Encrypted: i=1; AJvYcCXFSszaAa5uvZBLt7q15801bweNfDD70R382iMAug5EaV7caydVjh95lZAqMYCJMrA2itGcNqZH9tiPlJDFr3zlaPPVRe5rsHmBwOBO X-Gm-Message-State: AOJu0Yxx/OYVE0cAMc940A1S0LtsEd8sWyWYFL4OgGCnQmnBQ7taGoZV 6mlDbmI7NFf/LZrBlghQZhx/C1ircx6P/bofyYhVSfa0kkonZk93b6G1mg0p0FA= X-Google-Smtp-Source: AGHT+IFNWiib9Mbp3WUeDaTWc2pLd5fn9/fMVeRdDHjkyMdQAAUYNKmmTVlC+NReoIyOXuSV1VHXpQ== X-Received: by 2002:a17:902:f68e:b0:1f7:2849:183f with SMTP id d9443c01a7336-1f8625c243amr33675115ad.1.1718374955357; Fri, 14 Jun 2024 07:22:35 -0700 (PDT) Received: from hsinchu26.internal.sifive.com (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1f855e559d9sm32522005ad.35.2024.06.14.07.22.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Jun 2024 07:22:35 -0700 (PDT) From: Zong Li To: joro@8bytes.org, will@kernel.org, robin.murphy@arm.com, tjeznach@rivosinc.com, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, jgg@ziepe.ca, kevin.tian@intel.com, linux-kernel@vger.kernel.org, iommu@lists.linux.dev, linux-riscv@lists.infradead.org Cc: Zong Li Subject: [RFC PATCH v2 10/10] iommu:riscv: support nested iommu for get_msi_mapping_domain operation Date: Fri, 14 Jun 2024 22:21:56 +0800 Message-Id: <20240614142156.29420-11-zong.li@sifive.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240614142156.29420-1-zong.li@sifive.com> References: <20240614142156.29420-1-zong.li@sifive.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Return the iommu_domain that should hold the MSI cookie. Signed-off-by: Zong Li --- drivers/iommu/riscv/iommu.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/drivers/iommu/riscv/iommu.c b/drivers/iommu/riscv/iommu.c index d08eb0a2939e..969a0ba32c9e 100644 --- a/drivers/iommu/riscv/iommu.c +++ b/drivers/iommu/riscv/iommu.c @@ -1689,10 +1689,22 @@ static int riscv_iommu_cache_invalidate_user(struct= iommu_domain *domain, return ret; } =20 +static struct iommu_domain * +riscv_iommu_get_msi_mapping_domain(struct iommu_domain *domain) +{ + struct riscv_iommu_domain *riscv_domain =3D iommu_domain_to_riscv(domain); + + if (riscv_domain->s2) + return &riscv_domain->s2->domain; + + return domain; +} + static const struct iommu_domain_ops riscv_iommu_nested_domain_ops =3D { .attach_dev =3D riscv_iommu_attach_dev_nested, .free =3D riscv_iommu_domain_free_nested, .cache_invalidate_user =3D riscv_iommu_cache_invalidate_user, + .get_msi_mapping_domain =3D riscv_iommu_get_msi_mapping_domain, }; =20 static int --=20 2.17.1