Affirm SMMU coherent pagetable walker capability on RPMh SoCs

[PATCH RFC 07/11] arm64: dts: qcom: sm6350: Affirm IDR0.CCTW on apps_smmu

Posted by Konrad Dybcio 2 months, 1 week ago

From: Konrad Dybcio <quic_kdybcio@quicinc.com>

On RPMh-based SoCs, the APPS SMMU advertizes support for cache-coherent
pagetable walk via the IDR0 register. This however is not respected by
the arm-smmu driver unless dma-coherent is set.

Mark the node as dma-coherent to ensure this (and other) implementations
take this coherency into account.

Signed-off-by: Konrad Dybcio <quic_kdybcio@quicinc.com>
---
 arch/arm64/boot/dts/qcom/sm6350.dtsi | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/arm64/boot/dts/qcom/sm6350.dtsi b/arch/arm64/boot/dts/qcom/sm6350.dtsi
index 7986ddb30f6e8ce6ceeb0f90772b0243aed6bffe..54cfe99006613f8ccc5bf6d83bcb4bf8e72f3cfe 100644
--- a/arch/arm64/boot/dts/qcom/sm6350.dtsi
+++ b/arch/arm64/boot/dts/qcom/sm6350.dtsi
@@ -2685,6 +2685,7 @@ apps_smmu: iommu@15000000 {
 				     <GIC_SPI 411 IRQ_TYPE_LEVEL_HIGH>,
 				     <GIC_SPI 412 IRQ_TYPE_LEVEL_HIGH>,
 				     <GIC_SPI 413 IRQ_TYPE_LEVEL_HIGH>;
+			dma-coherent;
 		};
 
 		intc: interrupt-controller@17a00000 {

-- 
2.46.1

Re: [PATCH RFC 07/11] arm64: dts: qcom: sm6350: Affirm IDR0.CCTW on apps_smmu

Posted by Luca Weiss 2 months ago

On Thu Sep 19, 2024 at 12:57 AM CEST, Konrad Dybcio wrote:
> From: Konrad Dybcio <quic_kdybcio@quicinc.com>
>
> On RPMh-based SoCs, the APPS SMMU advertizes support for cache-coherent
> pagetable walk via the IDR0 register. This however is not respected by
> the arm-smmu driver unless dma-coherent is set.
>
> Mark the node as dma-coherent to ensure this (and other) implementations
> take this coherency into account.

Hi Konrad!

Similar to [0] everything seems to look fine on SM7225 Fairphone 4.

[    0.190433] arm-smmu 15000000.iommu: probing hardware configuration...
[    0.190459] arm-smmu 15000000.iommu: SMMUv2 with:
[    0.190499] arm-smmu 15000000.iommu:         stage 1 translation
[    0.190515] arm-smmu 15000000.iommu:         coherent table walk
[    0.190531] arm-smmu 15000000.iommu:         stream matching with 71 register groups
[    0.190560] arm-smmu 15000000.iommu:         63 context banks (0 stage-2 only)
[    0.191097] arm-smmu 15000000.iommu:         Supported page sizes: 0x61311000
[    0.191114] arm-smmu 15000000.iommu:         Stage-1: 36-bit VA -> 36-bit IPA
[    0.191299] arm-smmu 15000000.iommu:         preserved 0 boot mappings

The Adreno SMMU still has non-coherent table walk.

[    1.141215] arm-smmu 3d40000.iommu: probing hardware configuration...
[    1.141243] arm-smmu 3d40000.iommu: SMMUv2 with:
[    1.141270] arm-smmu 3d40000.iommu:  stage 1 translation
[    1.141279] arm-smmu 3d40000.iommu:  address translation ops
[    1.141288] arm-smmu 3d40000.iommu:  non-coherent table walk
[    1.141296] arm-smmu 3d40000.iommu:  (IDR0.CTTW overridden by FW configuration)
[    1.141307] arm-smmu 3d40000.iommu:  stream matching with 5 register groups
[    1.141326] arm-smmu 3d40000.iommu:  5 context banks (0 stage-2 only)
[    1.141347] arm-smmu 3d40000.iommu:  Supported page sizes: 0x63315000
[    1.141356] arm-smmu 3d40000.iommu:  Stage-1: 48-bit VA -> 36-bit IPA
[    1.141568] arm-smmu 3d40000.iommu:  preserved 0 boot mappings


Tested-by: Luca Weiss <luca.weiss@fairphone.com> # sm7225-fairphone-fp4

[0] https://lore.kernel.org/linux-arm-msm/CAD=FV=Xrbe1NO+trk1SJ30gHm5jLFjd0bAeG3H46gD+vNFZa1w@mail.gmail.com/

Regards
Luca

>
> Signed-off-by: Konrad Dybcio <quic_kdybcio@quicinc.com>
> ---
>  arch/arm64/boot/dts/qcom/sm6350.dtsi | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/arch/arm64/boot/dts/qcom/sm6350.dtsi b/arch/arm64/boot/dts/qcom/sm6350.dtsi
> index 7986ddb30f6e8ce6ceeb0f90772b0243aed6bffe..54cfe99006613f8ccc5bf6d83bcb4bf8e72f3cfe 100644
> --- a/arch/arm64/boot/dts/qcom/sm6350.dtsi
> +++ b/arch/arm64/boot/dts/qcom/sm6350.dtsi
> @@ -2685,6 +2685,7 @@ apps_smmu: iommu@15000000 {
>  				     <GIC_SPI 411 IRQ_TYPE_LEVEL_HIGH>,
>  				     <GIC_SPI 412 IRQ_TYPE_LEVEL_HIGH>,
>  				     <GIC_SPI 413 IRQ_TYPE_LEVEL_HIGH>;
> +			dma-coherent;
>  		};
>  
>  		intc: interrupt-controller@17a00000 {

Re: [PATCH RFC 07/11] arm64: dts: qcom: sm6350: Affirm IDR0.CCTW on apps_smmu

Posted by Luca Weiss 2 months ago

On Tue Sep 24, 2024 at 9:15 AM CEST, Luca Weiss wrote:
> On Thu Sep 19, 2024 at 12:57 AM CEST, Konrad Dybcio wrote:
> > From: Konrad Dybcio <quic_kdybcio@quicinc.com>
> >
> > On RPMh-based SoCs, the APPS SMMU advertizes support for cache-coherent
> > pagetable walk via the IDR0 register. This however is not respected by
> > the arm-smmu driver unless dma-coherent is set.
> >
> > Mark the node as dma-coherent to ensure this (and other) implementations
> > take this coherency into account.
>
> Hi Konrad!
>
> Similar to [0] everything seems to look fine on SM7225 Fairphone 4.
>
> [    0.190433] arm-smmu 15000000.iommu: probing hardware configuration...
> [    0.190459] arm-smmu 15000000.iommu: SMMUv2 with:
> [    0.190499] arm-smmu 15000000.iommu:         stage 1 translation
> [    0.190515] arm-smmu 15000000.iommu:         coherent table walk
> [    0.190531] arm-smmu 15000000.iommu:         stream matching with 71 register groups
> [    0.190560] arm-smmu 15000000.iommu:         63 context banks (0 stage-2 only)
> [    0.191097] arm-smmu 15000000.iommu:         Supported page sizes: 0x61311000
> [    0.191114] arm-smmu 15000000.iommu:         Stage-1: 36-bit VA -> 36-bit IPA
> [    0.191299] arm-smmu 15000000.iommu:         preserved 0 boot mappings
>
> The Adreno SMMU still has non-coherent table walk.
>
> [    1.141215] arm-smmu 3d40000.iommu: probing hardware configuration...
> [    1.141243] arm-smmu 3d40000.iommu: SMMUv2 with:
> [    1.141270] arm-smmu 3d40000.iommu:  stage 1 translation
> [    1.141279] arm-smmu 3d40000.iommu:  address translation ops
> [    1.141288] arm-smmu 3d40000.iommu:  non-coherent table walk
> [    1.141296] arm-smmu 3d40000.iommu:  (IDR0.CTTW overridden by FW configuration)
> [    1.141307] arm-smmu 3d40000.iommu:  stream matching with 5 register groups
> [    1.141326] arm-smmu 3d40000.iommu:  5 context banks (0 stage-2 only)
> [    1.141347] arm-smmu 3d40000.iommu:  Supported page sizes: 0x63315000
> [    1.141356] arm-smmu 3d40000.iommu:  Stage-1: 48-bit VA -> 36-bit IPA
> [    1.141568] arm-smmu 3d40000.iommu:  preserved 0 boot mappings
>
>
> Tested-by: Luca Weiss <luca.weiss@fairphone.com> # sm7225-fairphone-fp4
>
> [0] https://lore.kernel.org/linux-arm-msm/CAD=FV=Xrbe1NO+trk1SJ30gHm5jLFjd0bAeG3H46gD+vNFZa1w@mail.gmail.com/

FWIW adding 'dma-coherent;' to &adreno_smmu also doesn't seem to
explode:

[    1.451965] arm-smmu 3d40000.iommu: probing hardware configuration...
[    1.455547] arm-smmu 3d40000.iommu: SMMUv2 with:
[    1.459041] arm-smmu 3d40000.iommu:  stage 1 translation
[    1.462446] arm-smmu 3d40000.iommu:  address translation ops
[    1.465843] arm-smmu 3d40000.iommu:  coherent table walk
[    1.469216] arm-smmu 3d40000.iommu:  stream matching with 5 register groups
[    1.472645] arm-smmu 3d40000.iommu:  5 context banks (0 stage-2 only)
[    1.476067] arm-smmu 3d40000.iommu:  Supported page sizes: 0x63315000
[    1.479458] arm-smmu 3d40000.iommu:  Stage-1: 48-bit VA -> 36-bit IPA
[    1.483152] arm-smmu 3d40000.iommu:  preserved 0 boot mappings

And kmscube still runs as expected.

Regards
Luca

>
> Regards
> Luca
>
> >
> > Signed-off-by: Konrad Dybcio <quic_kdybcio@quicinc.com>
> > ---
> >  arch/arm64/boot/dts/qcom/sm6350.dtsi | 1 +
> >  1 file changed, 1 insertion(+)
> >
> > diff --git a/arch/arm64/boot/dts/qcom/sm6350.dtsi b/arch/arm64/boot/dts/qcom/sm6350.dtsi
> > index 7986ddb30f6e8ce6ceeb0f90772b0243aed6bffe..54cfe99006613f8ccc5bf6d83bcb4bf8e72f3cfe 100644
> > --- a/arch/arm64/boot/dts/qcom/sm6350.dtsi
> > +++ b/arch/arm64/boot/dts/qcom/sm6350.dtsi
> > @@ -2685,6 +2685,7 @@ apps_smmu: iommu@15000000 {
> >  				     <GIC_SPI 411 IRQ_TYPE_LEVEL_HIGH>,
> >  				     <GIC_SPI 412 IRQ_TYPE_LEVEL_HIGH>,
> >  				     <GIC_SPI 413 IRQ_TYPE_LEVEL_HIGH>;
> > +			dma-coherent;
> >  		};
> >  
> >  		intc: interrupt-controller@17a00000 {

Re: [PATCH RFC 07/11] arm64: dts: qcom: sm6350: Affirm IDR0.CCTW on apps_smmu

Posted by Konrad Dybcio 2 months ago

On 24.09.2024 9:37 AM, Luca Weiss wrote:
> On Tue Sep 24, 2024 at 9:15 AM CEST, Luca Weiss wrote:
>> On Thu Sep 19, 2024 at 12:57 AM CEST, Konrad Dybcio wrote:
>>> From: Konrad Dybcio <quic_kdybcio@quicinc.com>
>>>
>>> On RPMh-based SoCs, the APPS SMMU advertizes support for cache-coherent
>>> pagetable walk via the IDR0 register. This however is not respected by
>>> the arm-smmu driver unless dma-coherent is set.
>>>
>>> Mark the node as dma-coherent to ensure this (and other) implementations
>>> take this coherency into account.
>>
>> Hi Konrad!
>>
>> Similar to [0] everything seems to look fine on SM7225 Fairphone 4.
>>
>> [    0.190433] arm-smmu 15000000.iommu: probing hardware configuration...
>> [    0.190459] arm-smmu 15000000.iommu: SMMUv2 with:
>> [    0.190499] arm-smmu 15000000.iommu:         stage 1 translation
>> [    0.190515] arm-smmu 15000000.iommu:         coherent table walk
>> [    0.190531] arm-smmu 15000000.iommu:         stream matching with 71 register groups
>> [    0.190560] arm-smmu 15000000.iommu:         63 context banks (0 stage-2 only)
>> [    0.191097] arm-smmu 15000000.iommu:         Supported page sizes: 0x61311000
>> [    0.191114] arm-smmu 15000000.iommu:         Stage-1: 36-bit VA -> 36-bit IPA
>> [    0.191299] arm-smmu 15000000.iommu:         preserved 0 boot mappings
>>
>> The Adreno SMMU still has non-coherent table walk.
>>
>> [    1.141215] arm-smmu 3d40000.iommu: probing hardware configuration...
>> [    1.141243] arm-smmu 3d40000.iommu: SMMUv2 with:
>> [    1.141270] arm-smmu 3d40000.iommu:  stage 1 translation
>> [    1.141279] arm-smmu 3d40000.iommu:  address translation ops
>> [    1.141288] arm-smmu 3d40000.iommu:  non-coherent table walk
>> [    1.141296] arm-smmu 3d40000.iommu:  (IDR0.CTTW overridden by FW configuration)
>> [    1.141307] arm-smmu 3d40000.iommu:  stream matching with 5 register groups
>> [    1.141326] arm-smmu 3d40000.iommu:  5 context banks (0 stage-2 only)
>> [    1.141347] arm-smmu 3d40000.iommu:  Supported page sizes: 0x63315000
>> [    1.141356] arm-smmu 3d40000.iommu:  Stage-1: 48-bit VA -> 36-bit IPA
>> [    1.141568] arm-smmu 3d40000.iommu:  preserved 0 boot mappings
>>
>>
>> Tested-by: Luca Weiss <luca.weiss@fairphone.com> # sm7225-fairphone-fp4
>>
>> [0] https://lore.kernel.org/linux-arm-msm/CAD=FV=Xrbe1NO+trk1SJ30gHm5jLFjd0bAeG3H46gD+vNFZa1w@mail.gmail.com/
> 
> FWIW adding 'dma-coherent;' to &adreno_smmu also doesn't seem to
> explode:
> 
> [    1.451965] arm-smmu 3d40000.iommu: probing hardware configuration...
> [    1.455547] arm-smmu 3d40000.iommu: SMMUv2 with:
> [    1.459041] arm-smmu 3d40000.iommu:  stage 1 translation
> [    1.462446] arm-smmu 3d40000.iommu:  address translation ops
> [    1.465843] arm-smmu 3d40000.iommu:  coherent table walk
> [    1.469216] arm-smmu 3d40000.iommu:  stream matching with 5 register groups
> [    1.472645] arm-smmu 3d40000.iommu:  5 context banks (0 stage-2 only)
> [    1.476067] arm-smmu 3d40000.iommu:  Supported page sizes: 0x63315000
> [    1.479458] arm-smmu 3d40000.iommu:  Stage-1: 48-bit VA -> 36-bit IPA
> [    1.483152] arm-smmu 3d40000.iommu:  preserved 0 boot mappings
> 
> And kmscube still runs as expected.

Thanks, I'll look into adreno separately

Konrad