To track DTE[GIOV] programming during IOMMU domain attach, also add logic
to determine if the GIOV is required, and set the variable accordinglly.

This is also a preparation for adding nested domain support, where the GIOV
setting is determined by the child domain.

Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
---
 drivers/iommu/amd/amd_iommu_types.h |  1 +
 drivers/iommu/amd/iommu.c           | 11 +++++++++--
 2 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/drivers/iommu/amd/amd_iommu_types.h b/drivers/iommu/amd/amd_iommu_types.h
index 3dc39bbc05fc..ff56c857f6ad 100644
--- a/drivers/iommu/amd/amd_iommu_types.h
+++ b/drivers/iommu/amd/amd_iommu_types.h
@@ -536,6 +536,7 @@ struct gcr3_tbl_info {
 	u64	*gcr3_tbl;	/* Guest CR3 table */
 	int	glx;		/* Number of levels for GCR3 table */
 	u32	pasid_cnt;	/* Track attached PASIDs */
+	bool	giov;		/* Track DTE[GIOV] */
 };
 
 struct amd_io_pgtable {
diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c
index 4e4ff1550cf3..b9759f6d8be2 100644
--- a/drivers/iommu/amd/iommu.c
+++ b/drivers/iommu/amd/iommu.c
@@ -1990,8 +1990,7 @@ static void set_dte_entry(struct amd_iommu *iommu,
 				((u64)GUEST_PGTABLE_5_LEVEL << DTE_GPT_LEVEL_SHIFT);
 		}
 
-		/* GIOV is supported with V2 page table mode only */
-		if (pdom_is_v2_pgtbl_mode(domain))
+		if (gcr3_info->giov)
 			pte_root |= DTE_FLAG_GIOV;
 	}
 
@@ -2067,6 +2066,14 @@ static int do_attach(struct iommu_dev_data *dev_data,
 			free_gcr3_table(dev_data);
 			return ret;
 		}
+
+		/*
+		 * GIOV is required for PD_MODE_V2 because we need
+		 * to support the case where the end-point device
+		 * does not have PASID in the TLP prefix when setting
+		 * up to use the v2 table.
+		 */
+		dev_data->gcr3_info.giov = true;
 	}
 
 	/* Update device table */
-- 
2.34.1

On Thu, Jan 11, 2024 at 06:06:40PM -0600, Suravee Suthikulpanit wrote:
> To track DTE[GIOV] programming during IOMMU domain attach, also add logic
> to determine if the GIOV is required, and set the variable accordinglly.
> 
> This is also a preparation for adding nested domain support, where the GIOV
> setting is determined by the child domain.
> 
> Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
> ---
>  drivers/iommu/amd/amd_iommu_types.h |  1 +
>  drivers/iommu/amd/iommu.c           | 11 +++++++++--
>  2 files changed, 10 insertions(+), 2 deletions(-)

I really think the DTE handling needs to be cleaned up before nesting
will be easy.

Various bits of the DTE should just flow directly through from the
VM's version of the DTE, it is going to be a mess to do that in this
manner

> @@ -2067,6 +2066,14 @@ static int do_attach(struct iommu_dev_data *dev_data,
>  			free_gcr3_table(dev_data);
>  			return ret;
>  		}
> +
> +		/*
> +		 * GIOV is required for PD_MODE_V2 because we need
> +		 * to support the case where the end-point device
> +		 * does not have PASID in the TLP prefix when setting
> +		 * up to use the v2 table.
> +		 */
> +		dev_data->gcr3_info.giov = true;

Ie who clears this once you set it ? :(

Jason