[v3] PCI: cadence: Fix sending message with data or without data

Posted by Hans Zhang 1 year ago

View from cdns document cdn_pcie_gen4_hpa_axi_ips_ug_v1.04.pdf.
In section 9.1.7.1 AXI Subordinate to PCIe Address Translation
Registers below:

axi_s_awaddr bits 16 is 1 for MSG with data and 0 for MSG without data.

Signed-off-by: hans.zhang <18255117159@163.com>
---
Changes since v1-v2:
- Change email number and Signed-off-by
---
 drivers/pci/controller/cadence/pcie-cadence-ep.c | 3 +--
 drivers/pci/controller/cadence/pcie-cadence.h    | 2 +-
 2 files changed, 2 insertions(+), 3 deletions(-)

diff --git a/drivers/pci/controller/cadence/pcie-cadence-ep.c b/drivers/pci/controller/cadence/pcie-cadence-ep.c
index e0cc4560dfde..0bf4cde34f51 100644
--- a/drivers/pci/controller/cadence/pcie-cadence-ep.c
+++ b/drivers/pci/controller/cadence/pcie-cadence-ep.c
@@ -352,8 +352,7 @@ static void cdns_pcie_ep_assert_intx(struct cdns_pcie_ep *ep, u8 fn, u8 intx,
 	spin_unlock_irqrestore(&ep->lock, flags);
 
 	offset = CDNS_PCIE_NORMAL_MSG_ROUTING(MSG_ROUTING_LOCAL) |
-		 CDNS_PCIE_NORMAL_MSG_CODE(msg_code) |
-		 CDNS_PCIE_MSG_NO_DATA;
+		 CDNS_PCIE_NORMAL_MSG_CODE(msg_code);
 	writel(0, ep->irq_cpu_addr + offset);
 }
 
diff --git a/drivers/pci/controller/cadence/pcie-cadence.h b/drivers/pci/controller/cadence/pcie-cadence.h
index f5eeff834ec1..39ee9945c903 100644
--- a/drivers/pci/controller/cadence/pcie-cadence.h
+++ b/drivers/pci/controller/cadence/pcie-cadence.h
@@ -246,7 +246,7 @@ struct cdns_pcie_rp_ib_bar {
 #define CDNS_PCIE_NORMAL_MSG_CODE_MASK		GENMASK(15, 8)
 #define CDNS_PCIE_NORMAL_MSG_CODE(code) \
 	(((code) << 8) & CDNS_PCIE_NORMAL_MSG_CODE_MASK)
-#define CDNS_PCIE_MSG_NO_DATA			BIT(16)
+#define CDNS_PCIE_MSG_DATA			BIT(16)
 
 struct cdns_pcie;
 

base-commit: bb066fe812d6fb3a9d01c073d9f1e2fd5a63403b
-- 
2.47.1

Re: [v3] PCI: cadence: Fix sending message with data or without data

Posted by Manivannan Sadhasivam 11 months, 4 weeks ago

On Fri, Feb 07, 2025 at 06:39:23PM +0800, Hans Zhang wrote:

Subject should be changed as:

PCI: cadence-ep: Fix the driver to send MSG TLP for INTx without data payload

> View from cdns document cdn_pcie_gen4_hpa_axi_ips_ug_v1.04.pdf.
> In section 9.1.7.1 AXI Subordinate to PCIe Address Translation
> Registers below:
> 
> axi_s_awaddr bits 16 is 1 for MSG with data and 0 for MSG without data.
> 

How about,

Cadence reference manual cdn_pcie_gen4_hpa_axi_ips_ug_v1.04.pdf, section
9.1.7.1 'AXI Subordinate to PCIe Address Translation' mentions that
axi_s_awaddr bits 16 when set, corresponds to MSG with data and when not
set, MSG without data.

But the driver is doing the opposite and due to this, INTx is never
received on the host. So fix the driver to reflect the documentation and
also make INTx work.

> Signed-off-by: hans.zhang <18255117159@163.com>

Please add the Fixes tag also.

- Mani

> ---
> Changes since v1-v2:
> - Change email number and Signed-off-by
> ---
>  drivers/pci/controller/cadence/pcie-cadence-ep.c | 3 +--
>  drivers/pci/controller/cadence/pcie-cadence.h    | 2 +-
>  2 files changed, 2 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/pci/controller/cadence/pcie-cadence-ep.c b/drivers/pci/controller/cadence/pcie-cadence-ep.c
> index e0cc4560dfde..0bf4cde34f51 100644
> --- a/drivers/pci/controller/cadence/pcie-cadence-ep.c
> +++ b/drivers/pci/controller/cadence/pcie-cadence-ep.c
> @@ -352,8 +352,7 @@ static void cdns_pcie_ep_assert_intx(struct cdns_pcie_ep *ep, u8 fn, u8 intx,
>  	spin_unlock_irqrestore(&ep->lock, flags);
>  
>  	offset = CDNS_PCIE_NORMAL_MSG_ROUTING(MSG_ROUTING_LOCAL) |
> -		 CDNS_PCIE_NORMAL_MSG_CODE(msg_code) |
> -		 CDNS_PCIE_MSG_NO_DATA;
> +		 CDNS_PCIE_NORMAL_MSG_CODE(msg_code);
>  	writel(0, ep->irq_cpu_addr + offset);
>  }
>  
> diff --git a/drivers/pci/controller/cadence/pcie-cadence.h b/drivers/pci/controller/cadence/pcie-cadence.h
> index f5eeff834ec1..39ee9945c903 100644
> --- a/drivers/pci/controller/cadence/pcie-cadence.h
> +++ b/drivers/pci/controller/cadence/pcie-cadence.h
> @@ -246,7 +246,7 @@ struct cdns_pcie_rp_ib_bar {
>  #define CDNS_PCIE_NORMAL_MSG_CODE_MASK		GENMASK(15, 8)
>  #define CDNS_PCIE_NORMAL_MSG_CODE(code) \
>  	(((code) << 8) & CDNS_PCIE_NORMAL_MSG_CODE_MASK)
> -#define CDNS_PCIE_MSG_NO_DATA			BIT(16)
> +#define CDNS_PCIE_MSG_DATA			BIT(16)
>  
>  struct cdns_pcie;
>  
> 
> base-commit: bb066fe812d6fb3a9d01c073d9f1e2fd5a63403b
> -- 
> 2.47.1
> 

-- 
மணிவண்ணன் சதாசிவம்

Re: [v3] PCI: cadence: Fix sending message with data or without data

Posted by Hans Zhang 11 months, 4 weeks ago


On 2025/2/15 00:06, Manivannan Sadhasivam wrote:
> On Fri, Feb 07, 2025 at 06:39:23PM +0800, Hans Zhang wrote:
> 
> Subject should be changed as:
> 
> PCI: cadence-ep: Fix the driver to send MSG TLP for INTx without data payload

Okay.

>> View from cdns document cdn_pcie_gen4_hpa_axi_ips_ug_v1.04.pdf.
>> In section 9.1.7.1 AXI Subordinate to PCIe Address Translation
>> Registers below:
>>
>> axi_s_awaddr bits 16 is 1 for MSG with data and 0 for MSG without data.
>>
> 
> How about,
> 
> Cadence reference manual cdn_pcie_gen4_hpa_axi_ips_ug_v1.04.pdf, section
> 9.1.7.1 'AXI Subordinate to PCIe Address Translation' mentions that
> axi_s_awaddr bits 16 when set, corresponds to MSG with data and when not
> set, MSG without data.
> 
> But the driver is doing the opposite and due to this, INTx is never
> received on the host. So fix the driver to reflect the documentation and
> also make INTx work.

Thank you very much.

>> Signed-off-by: hans.zhang <18255117159@163.com>
> 
> Please add the Fixes tag also.
> 

Okay.

> 
>> ---
>> Changes since v1-v2:
>> - Change email number and Signed-off-by
>> ---
>>   drivers/pci/controller/cadence/pcie-cadence-ep.c | 3 +--
>>   drivers/pci/controller/cadence/pcie-cadence.h    | 2 +-
>>   2 files changed, 2 insertions(+), 3 deletions(-)
>>
>> diff --git a/drivers/pci/controller/cadence/pcie-cadence-ep.c b/drivers/pci/controller/cadence/pcie-cadence-ep.c
>> index e0cc4560dfde..0bf4cde34f51 100644
>> --- a/drivers/pci/controller/cadence/pcie-cadence-ep.c
>> +++ b/drivers/pci/controller/cadence/pcie-cadence-ep.c
>> @@ -352,8 +352,7 @@ static void cdns_pcie_ep_assert_intx(struct cdns_pcie_ep *ep, u8 fn, u8 intx,
>>   	spin_unlock_irqrestore(&ep->lock, flags);
>>   
>>   	offset = CDNS_PCIE_NORMAL_MSG_ROUTING(MSG_ROUTING_LOCAL) |
>> -		 CDNS_PCIE_NORMAL_MSG_CODE(msg_code) |
>> -		 CDNS_PCIE_MSG_NO_DATA;
>> +		 CDNS_PCIE_NORMAL_MSG_CODE(msg_code);
>>   	writel(0, ep->irq_cpu_addr + offset);
>>   }
>>   
>> diff --git a/drivers/pci/controller/cadence/pcie-cadence.h b/drivers/pci/controller/cadence/pcie-cadence.h
>> index f5eeff834ec1..39ee9945c903 100644
>> --- a/drivers/pci/controller/cadence/pcie-cadence.h
>> +++ b/drivers/pci/controller/cadence/pcie-cadence.h
>> @@ -246,7 +246,7 @@ struct cdns_pcie_rp_ib_bar {
>>   #define CDNS_PCIE_NORMAL_MSG_CODE_MASK		GENMASK(15, 8)
>>   #define CDNS_PCIE_NORMAL_MSG_CODE(code) \
>>   	(((code) << 8) & CDNS_PCIE_NORMAL_MSG_CODE_MASK)
>> -#define CDNS_PCIE_MSG_NO_DATA			BIT(16)
>> +#define CDNS_PCIE_MSG_DATA			BIT(16)
>>   
>>   struct cdns_pcie;
>>   
>>
>> base-commit: bb066fe812d6fb3a9d01c073d9f1e2fd5a63403b
>> -- 
>> 2.47.1
>>
>

Re: [v3] PCI: cadence: Fix sending message with data or without data

Posted by Manivannan Sadhasivam 11 months, 4 weeks ago

On Fri, Feb 07, 2025 at 06:39:23PM +0800, Hans Zhang wrote:
> View from cdns document cdn_pcie_gen4_hpa_axi_ips_ug_v1.04.pdf.
> In section 9.1.7.1 AXI Subordinate to PCIe Address Translation
> Registers below:
> 
> axi_s_awaddr bits 16 is 1 for MSG with data and 0 for MSG without data.
> 
> Signed-off-by: hans.zhang <18255117159@163.com>
> ---
> Changes since v1-v2:
> - Change email number and Signed-off-by
> ---
>  drivers/pci/controller/cadence/pcie-cadence-ep.c | 3 +--
>  drivers/pci/controller/cadence/pcie-cadence.h    | 2 +-
>  2 files changed, 2 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/pci/controller/cadence/pcie-cadence-ep.c b/drivers/pci/controller/cadence/pcie-cadence-ep.c
> index e0cc4560dfde..0bf4cde34f51 100644
> --- a/drivers/pci/controller/cadence/pcie-cadence-ep.c
> +++ b/drivers/pci/controller/cadence/pcie-cadence-ep.c
> @@ -352,8 +352,7 @@ static void cdns_pcie_ep_assert_intx(struct cdns_pcie_ep *ep, u8 fn, u8 intx,
>  	spin_unlock_irqrestore(&ep->lock, flags);
>  
>  	offset = CDNS_PCIE_NORMAL_MSG_ROUTING(MSG_ROUTING_LOCAL) |
> -		 CDNS_PCIE_NORMAL_MSG_CODE(msg_code) |
> -		 CDNS_PCIE_MSG_NO_DATA;
> +		 CDNS_PCIE_NORMAL_MSG_CODE(msg_code);
>  	writel(0, ep->irq_cpu_addr + offset);
>  }
>  
> diff --git a/drivers/pci/controller/cadence/pcie-cadence.h b/drivers/pci/controller/cadence/pcie-cadence.h
> index f5eeff834ec1..39ee9945c903 100644
> --- a/drivers/pci/controller/cadence/pcie-cadence.h
> +++ b/drivers/pci/controller/cadence/pcie-cadence.h
> @@ -246,7 +246,7 @@ struct cdns_pcie_rp_ib_bar {
>  #define CDNS_PCIE_NORMAL_MSG_CODE_MASK		GENMASK(15, 8)
>  #define CDNS_PCIE_NORMAL_MSG_CODE(code) \
>  	(((code) << 8) & CDNS_PCIE_NORMAL_MSG_CODE_MASK)
> -#define CDNS_PCIE_MSG_NO_DATA			BIT(16)
> +#define CDNS_PCIE_MSG_DATA			BIT(16)

Oops! So how did you spot the issue? Did INTx triggering ever worked? RC should
have reported it as malformed TLP isn't it?

- Mani

-- 
மணிவண்ணன் சதாசிவம்

Re: [v3] PCI: cadence: Fix sending message with data or without data

Posted by Hans Zhang 11 months, 4 weeks ago


On 2025/2/14 15:30, Manivannan Sadhasivam wrote:
> On Fri, Feb 07, 2025 at 06:39:23PM +0800, Hans Zhang wrote:
>> View from cdns document cdn_pcie_gen4_hpa_axi_ips_ug_v1.04.pdf.
>> In section 9.1.7.1 AXI Subordinate to PCIe Address Translation
>> Registers below:
>>
>> axi_s_awaddr bits 16 is 1 for MSG with data and 0 for MSG without data.
>>
>> Signed-off-by: hans.zhang <18255117159@163.com>
>> ---
>> Changes since v1-v2:
>> - Change email number and Signed-off-by
>> ---
>>   drivers/pci/controller/cadence/pcie-cadence-ep.c | 3 +--
>>   drivers/pci/controller/cadence/pcie-cadence.h    | 2 +-
>>   2 files changed, 2 insertions(+), 3 deletions(-)
>>
>> diff --git a/drivers/pci/controller/cadence/pcie-cadence-ep.c b/drivers/pci/controller/cadence/pcie-cadence-ep.c
>> index e0cc4560dfde..0bf4cde34f51 100644
>> --- a/drivers/pci/controller/cadence/pcie-cadence-ep.c
>> +++ b/drivers/pci/controller/cadence/pcie-cadence-ep.c
>> @@ -352,8 +352,7 @@ static void cdns_pcie_ep_assert_intx(struct cdns_pcie_ep *ep, u8 fn, u8 intx,
>>   	spin_unlock_irqrestore(&ep->lock, flags);
>>   
>>   	offset = CDNS_PCIE_NORMAL_MSG_ROUTING(MSG_ROUTING_LOCAL) |
>> -		 CDNS_PCIE_NORMAL_MSG_CODE(msg_code) |
>> -		 CDNS_PCIE_MSG_NO_DATA;
>> +		 CDNS_PCIE_NORMAL_MSG_CODE(msg_code);
>>   	writel(0, ep->irq_cpu_addr + offset);
>>   }
>>   
>> diff --git a/drivers/pci/controller/cadence/pcie-cadence.h b/drivers/pci/controller/cadence/pcie-cadence.h
>> index f5eeff834ec1..39ee9945c903 100644
>> --- a/drivers/pci/controller/cadence/pcie-cadence.h
>> +++ b/drivers/pci/controller/cadence/pcie-cadence.h
>> @@ -246,7 +246,7 @@ struct cdns_pcie_rp_ib_bar {
>>   #define CDNS_PCIE_NORMAL_MSG_CODE_MASK		GENMASK(15, 8)
>>   #define CDNS_PCIE_NORMAL_MSG_CODE(code) \
>>   	(((code) << 8) & CDNS_PCIE_NORMAL_MSG_CODE_MASK)
>> -#define CDNS_PCIE_MSG_NO_DATA			BIT(16)
>> +#define CDNS_PCIE_MSG_DATA			BIT(16)
> 
> Oops! So how did you spot the issue? Did INTx triggering ever worked? RC should
> have reported it as malformed TLP isn't it?
> 
In our first generation SOC, sending messages did not work, and the 
length of messages was all 1. Cadence fixed this problem in the second 
generation SOC. And I have verified in the EMU environment that it is OK 
to send various messages, including INTx.

And that's what Cadence's release documentation says:
axi_s_awaddr bits 16 is 1 for MSG with data and 0 for MSG without data.
The final verification results are also consistent with the 
documentation description.

Due to the RTL bug in Cadence IP of our SOC in the first generation, the 
AER Error reported at that time was Unsupported Request Error Status.

Best regards
Hans

Re: [v3] PCI: cadence: Fix sending message with data or without data

Posted by Manivannan Sadhasivam 11 months, 4 weeks ago

On Fri, Feb 14, 2025 at 04:23:33PM +0800, Hans Zhang wrote:
> 
> 
> On 2025/2/14 15:30, Manivannan Sadhasivam wrote:
> > On Fri, Feb 07, 2025 at 06:39:23PM +0800, Hans Zhang wrote:
> > > View from cdns document cdn_pcie_gen4_hpa_axi_ips_ug_v1.04.pdf.
> > > In section 9.1.7.1 AXI Subordinate to PCIe Address Translation
> > > Registers below:
> > > 
> > > axi_s_awaddr bits 16 is 1 for MSG with data and 0 for MSG without data.
> > > 
> > > Signed-off-by: hans.zhang <18255117159@163.com>
> > > ---
> > > Changes since v1-v2:
> > > - Change email number and Signed-off-by
> > > ---
> > >   drivers/pci/controller/cadence/pcie-cadence-ep.c | 3 +--
> > >   drivers/pci/controller/cadence/pcie-cadence.h    | 2 +-
> > >   2 files changed, 2 insertions(+), 3 deletions(-)
> > > 
> > > diff --git a/drivers/pci/controller/cadence/pcie-cadence-ep.c b/drivers/pci/controller/cadence/pcie-cadence-ep.c
> > > index e0cc4560dfde..0bf4cde34f51 100644
> > > --- a/drivers/pci/controller/cadence/pcie-cadence-ep.c
> > > +++ b/drivers/pci/controller/cadence/pcie-cadence-ep.c
> > > @@ -352,8 +352,7 @@ static void cdns_pcie_ep_assert_intx(struct cdns_pcie_ep *ep, u8 fn, u8 intx,
> > >   	spin_unlock_irqrestore(&ep->lock, flags);
> > >   	offset = CDNS_PCIE_NORMAL_MSG_ROUTING(MSG_ROUTING_LOCAL) |
> > > -		 CDNS_PCIE_NORMAL_MSG_CODE(msg_code) |
> > > -		 CDNS_PCIE_MSG_NO_DATA;
> > > +		 CDNS_PCIE_NORMAL_MSG_CODE(msg_code);
> > >   	writel(0, ep->irq_cpu_addr + offset);
> > >   }
> > > diff --git a/drivers/pci/controller/cadence/pcie-cadence.h b/drivers/pci/controller/cadence/pcie-cadence.h
> > > index f5eeff834ec1..39ee9945c903 100644
> > > --- a/drivers/pci/controller/cadence/pcie-cadence.h
> > > +++ b/drivers/pci/controller/cadence/pcie-cadence.h
> > > @@ -246,7 +246,7 @@ struct cdns_pcie_rp_ib_bar {
> > >   #define CDNS_PCIE_NORMAL_MSG_CODE_MASK		GENMASK(15, 8)
> > >   #define CDNS_PCIE_NORMAL_MSG_CODE(code) \
> > >   	(((code) << 8) & CDNS_PCIE_NORMAL_MSG_CODE_MASK)
> > > -#define CDNS_PCIE_MSG_NO_DATA			BIT(16)
> > > +#define CDNS_PCIE_MSG_DATA			BIT(16)
> > 
> > Oops! So how did you spot the issue? Did INTx triggering ever worked? RC should
> > have reported it as malformed TLP isn't it?
> > 
> In our first generation SOC, sending messages did not work, and the length
> of messages was all 1. Cadence fixed this problem in the second generation
> SOC. And I have verified in the EMU environment that it is OK to send
> various messages, including INTx.
> 
> And that's what Cadence's release documentation says:
> axi_s_awaddr bits 16 is 1 for MSG with data and 0 for MSG without data.

I'm confused now. So the change in axi_s_awaddr bit applies to second generation
SoCs only? What about the first ones?

Are you saying that the first generation SoCs can never send any message TLPs at
all? This sounds horrible.

- Mani

-- 
மணிவண்ணன் சதாசிவம்

Re: [v3] PCI: cadence: Fix sending message with data or without data

Posted by Hans Zhang 11 months, 4 weeks ago


On 2025/2/14 21:21, Manivannan Sadhasivam wrote:
> On Fri, Feb 14, 2025 at 04:23:33PM +0800, Hans Zhang wrote:
>>
>>
>> On 2025/2/14 15:30, Manivannan Sadhasivam wrote:
>>> On Fri, Feb 07, 2025 at 06:39:23PM +0800, Hans Zhang wrote:
>>>> View from cdns document cdn_pcie_gen4_hpa_axi_ips_ug_v1.04.pdf.
>>>> In section 9.1.7.1 AXI Subordinate to PCIe Address Translation
>>>> Registers below:
>>>>
>>>> axi_s_awaddr bits 16 is 1 for MSG with data and 0 for MSG without data.
>>>>
>>>> Signed-off-by: hans.zhang <18255117159@163.com>
>>>> ---
>>>> Changes since v1-v2:
>>>> - Change email number and Signed-off-by
>>>> ---
>>>>    drivers/pci/controller/cadence/pcie-cadence-ep.c | 3 +--
>>>>    drivers/pci/controller/cadence/pcie-cadence.h    | 2 +-
>>>>    2 files changed, 2 insertions(+), 3 deletions(-)
>>>>
>>>> diff --git a/drivers/pci/controller/cadence/pcie-cadence-ep.c b/drivers/pci/controller/cadence/pcie-cadence-ep.c
>>>> index e0cc4560dfde..0bf4cde34f51 100644
>>>> --- a/drivers/pci/controller/cadence/pcie-cadence-ep.c
>>>> +++ b/drivers/pci/controller/cadence/pcie-cadence-ep.c
>>>> @@ -352,8 +352,7 @@ static void cdns_pcie_ep_assert_intx(struct cdns_pcie_ep *ep, u8 fn, u8 intx,
>>>>    	spin_unlock_irqrestore(&ep->lock, flags);
>>>>    	offset = CDNS_PCIE_NORMAL_MSG_ROUTING(MSG_ROUTING_LOCAL) |
>>>> -		 CDNS_PCIE_NORMAL_MSG_CODE(msg_code) |
>>>> -		 CDNS_PCIE_MSG_NO_DATA;
>>>> +		 CDNS_PCIE_NORMAL_MSG_CODE(msg_code);
>>>>    	writel(0, ep->irq_cpu_addr + offset);
>>>>    }
>>>> diff --git a/drivers/pci/controller/cadence/pcie-cadence.h b/drivers/pci/controller/cadence/pcie-cadence.h
>>>> index f5eeff834ec1..39ee9945c903 100644
>>>> --- a/drivers/pci/controller/cadence/pcie-cadence.h
>>>> +++ b/drivers/pci/controller/cadence/pcie-cadence.h
>>>> @@ -246,7 +246,7 @@ struct cdns_pcie_rp_ib_bar {
>>>>    #define CDNS_PCIE_NORMAL_MSG_CODE_MASK		GENMASK(15, 8)
>>>>    #define CDNS_PCIE_NORMAL_MSG_CODE(code) \
>>>>    	(((code) << 8) & CDNS_PCIE_NORMAL_MSG_CODE_MASK)
>>>> -#define CDNS_PCIE_MSG_NO_DATA			BIT(16)
>>>> +#define CDNS_PCIE_MSG_DATA			BIT(16)
>>>
>>> Oops! So how did you spot the issue? Did INTx triggering ever worked? RC should
>>> have reported it as malformed TLP isn't it?
>>>
>> In our first generation SOC, sending messages did not work, and the length
>> of messages was all 1. Cadence fixed this problem in the second generation
>> SOC. And I have verified in the EMU environment that it is OK to send
>> various messages, including INTx.
>>
>> And that's what Cadence's release documentation says:
>> axi_s_awaddr bits 16 is 1 for MSG with data and 0 for MSG without data.
> 
> I'm confused now. So the change in axi_s_awaddr bit applies to second generation
> SoCs only? What about the first ones?
> 
> Are you saying that the first generation SoCs can never send any message TLPs at
> all? This sounds horrible.


Sorry Mani, I shouldn't have spread this SOC bug. This is a bug in RTL 
design, the WSTRB signal of AXI bus is not connected correctly, so the 
first generation SOC cannot send message, because we mainly use RC mode, 
and we cannot send PME_Turn_OFF, that is, our SOC does not support L2. I 
have no choice about this, I entered the company relatively late, and 
our SOC has already TO.


This patch is to solve the Cadence common code bug, and does not conform 
to Cadence documentation.

Best regards
Hans

Re: [v3] PCI: cadence: Fix sending message with data or without data

Posted by Manivannan Sadhasivam 11 months, 4 weeks ago

On Fri, Feb 14, 2025 at 10:28:11PM +0800, Hans Zhang wrote:
> 
> 
> On 2025/2/14 21:21, Manivannan Sadhasivam wrote:
> > On Fri, Feb 14, 2025 at 04:23:33PM +0800, Hans Zhang wrote:
> > > 
> > > 
> > > On 2025/2/14 15:30, Manivannan Sadhasivam wrote:
> > > > On Fri, Feb 07, 2025 at 06:39:23PM +0800, Hans Zhang wrote:
> > > > > View from cdns document cdn_pcie_gen4_hpa_axi_ips_ug_v1.04.pdf.
> > > > > In section 9.1.7.1 AXI Subordinate to PCIe Address Translation
> > > > > Registers below:
> > > > > 
> > > > > axi_s_awaddr bits 16 is 1 for MSG with data and 0 for MSG without data.
> > > > > 
> > > > > Signed-off-by: hans.zhang <18255117159@163.com>
> > > > > ---
> > > > > Changes since v1-v2:
> > > > > - Change email number and Signed-off-by
> > > > > ---
> > > > >    drivers/pci/controller/cadence/pcie-cadence-ep.c | 3 +--
> > > > >    drivers/pci/controller/cadence/pcie-cadence.h    | 2 +-
> > > > >    2 files changed, 2 insertions(+), 3 deletions(-)
> > > > > 
> > > > > diff --git a/drivers/pci/controller/cadence/pcie-cadence-ep.c b/drivers/pci/controller/cadence/pcie-cadence-ep.c
> > > > > index e0cc4560dfde..0bf4cde34f51 100644
> > > > > --- a/drivers/pci/controller/cadence/pcie-cadence-ep.c
> > > > > +++ b/drivers/pci/controller/cadence/pcie-cadence-ep.c
> > > > > @@ -352,8 +352,7 @@ static void cdns_pcie_ep_assert_intx(struct cdns_pcie_ep *ep, u8 fn, u8 intx,
> > > > >    	spin_unlock_irqrestore(&ep->lock, flags);
> > > > >    	offset = CDNS_PCIE_NORMAL_MSG_ROUTING(MSG_ROUTING_LOCAL) |
> > > > > -		 CDNS_PCIE_NORMAL_MSG_CODE(msg_code) |
> > > > > -		 CDNS_PCIE_MSG_NO_DATA;
> > > > > +		 CDNS_PCIE_NORMAL_MSG_CODE(msg_code);
> > > > >    	writel(0, ep->irq_cpu_addr + offset);
> > > > >    }
> > > > > diff --git a/drivers/pci/controller/cadence/pcie-cadence.h b/drivers/pci/controller/cadence/pcie-cadence.h
> > > > > index f5eeff834ec1..39ee9945c903 100644
> > > > > --- a/drivers/pci/controller/cadence/pcie-cadence.h
> > > > > +++ b/drivers/pci/controller/cadence/pcie-cadence.h
> > > > > @@ -246,7 +246,7 @@ struct cdns_pcie_rp_ib_bar {
> > > > >    #define CDNS_PCIE_NORMAL_MSG_CODE_MASK		GENMASK(15, 8)
> > > > >    #define CDNS_PCIE_NORMAL_MSG_CODE(code) \
> > > > >    	(((code) << 8) & CDNS_PCIE_NORMAL_MSG_CODE_MASK)
> > > > > -#define CDNS_PCIE_MSG_NO_DATA			BIT(16)
> > > > > +#define CDNS_PCIE_MSG_DATA			BIT(16)
> > > > 
> > > > Oops! So how did you spot the issue? Did INTx triggering ever worked? RC should
> > > > have reported it as malformed TLP isn't it?
> > > > 
> > > In our first generation SOC, sending messages did not work, and the length
> > > of messages was all 1. Cadence fixed this problem in the second generation
> > > SOC. And I have verified in the EMU environment that it is OK to send
> > > various messages, including INTx.
> > > 
> > > And that's what Cadence's release documentation says:
> > > axi_s_awaddr bits 16 is 1 for MSG with data and 0 for MSG without data.
> > 
> > I'm confused now. So the change in axi_s_awaddr bit applies to second generation
> > SoCs only? What about the first ones?
> > 
> > Are you saying that the first generation SoCs can never send any message TLPs at
> > all? This sounds horrible.
> 
> 
> Sorry Mani, I shouldn't have spread this SOC bug. This is a bug in RTL
> design, the WSTRB signal of AXI bus is not connected correctly, so the first
> generation SOC cannot send message, because we mainly use RC mode, and we
> cannot send PME_Turn_OFF, that is, our SOC does not support L2. I have no
> choice about this, I entered the company relatively late, and our SOC has
> already TO.

Ok. Just to clear my head, this patch is needed irrespective of the hw issue,
right? And with or without this patch, first revision hw cannot send any MSG
TLPs?

If so, it is fine. But is there a way we could detect those first generation IPs
and flag it to users about broken MSG TLP support? Atleast, that would make the
users aware of broken hw.

> 
> 
> This patch is to solve the Cadence common code bug, and does not conform to
> Cadence documentation.

you mean 'does'?

- Mani

-- 
மணிவண்ணன் சதாசிவம்

Re: [v3] PCI: cadence: Fix sending message with data or without data

Posted by Hans Zhang 11 months, 4 weeks ago


On 2025/2/14 23:31, Manivannan Sadhasivam wrote:
> On Fri, Feb 14, 2025 at 10:28:11PM +0800, Hans Zhang wrote:
>> Sorry Mani, I shouldn't have spread this SOC bug. This is a bug in RTL
>> design, the WSTRB signal of AXI bus is not connected correctly, so the first
>> generation SOC cannot send message, because we mainly use RC mode, and we
>> cannot send PME_Turn_OFF, that is, our SOC does not support L2. I have no
>> choice about this, I entered the company relatively late, and our SOC has
>> already TO.
> 
> Ok. Just to clear my head, this patch is needed irrespective of the hw issue,
> right? And with or without this patch, first revision hw cannot send any MSG
> TLPs?

Yes, that was a problem with our own SOC design, the Cadence RTL bug.	

> If so, it is fine. But is there a way we could detect those first generation IPs
> and flag it to users about broken MSG TLP support? Atleast, that would make the
> users aware of broken hw.

I don't know how to do it, but here are the questions that were actually 
tested.

>>
>> This patch is to solve the Cadence common code bug, and does not conform to
>> Cadence documentation.
> 
> you mean 'does'?
> 

What I mean is that common code bit16=1 is to send a message without 
data, while Cadence's development document says that bit16=0 is to send 
a message without data. This is not consistent with the documentation 
description, and the final verification results, the development 
documentation described is correct.

Best regards
Hans

Re: [v3] PCI: cadence: Fix sending message with data or without data

Posted by Manivannan Sadhasivam 11 months, 4 weeks ago

On Fri, Feb 14, 2025 at 11:48:16PM +0800, Hans Zhang wrote:
> 
> 
> On 2025/2/14 23:31, Manivannan Sadhasivam wrote:
> > On Fri, Feb 14, 2025 at 10:28:11PM +0800, Hans Zhang wrote:
> > > Sorry Mani, I shouldn't have spread this SOC bug. This is a bug in RTL
> > > design, the WSTRB signal of AXI bus is not connected correctly, so the first
> > > generation SOC cannot send message, because we mainly use RC mode, and we
> > > cannot send PME_Turn_OFF, that is, our SOC does not support L2. I have no
> > > choice about this, I entered the company relatively late, and our SOC has
> > > already TO.
> > 
> > Ok. Just to clear my head, this patch is needed irrespective of the hw issue,
> > right? And with or without this patch, first revision hw cannot send any MSG
> > TLPs?
> 
> Yes, that was a problem with our own SOC design, the Cadence RTL bug.	
> 
> > If so, it is fine. But is there a way we could detect those first generation IPs
> > and flag it to users about broken MSG TLP support? Atleast, that would make the
> > users aware of broken hw.
> 
> I don't know how to do it, but here are the questions that were actually
> tested.
> 

Not related to this patch, but please check if it is possible to detect those
controllers.

> > > 
> > > This patch is to solve the Cadence common code bug, and does not conform to
> > > Cadence documentation.
> > 
> > you mean 'does'?
> > 
> 
> What I mean is that common code bit16=1 is to send a message without data,
> while Cadence's development document says that bit16=0 is to send a message
> without data. This is not consistent with the documentation description, and
> the final verification results, the development documentation described is
> correct.
> 

Ok, makes sense now.

- Mani

-- 
மணிவண்ணன் சதாசிவம்

Re: [v3] PCI: cadence: Fix sending message with data or without data

Posted by Hans Zhang 11 months, 4 weeks ago


On 2025/2/14 23:57, Manivannan Sadhasivam wrote:
> On Fri, Feb 14, 2025 at 11:48:16PM +0800, Hans Zhang wrote:
>>> If so, it is fine. But is there a way we could detect those first generation IPs
>>> and flag it to users about broken MSG TLP support? Atleast, that would make the
>>> users aware of broken hw.
>>
>> I don't know how to do it, but here are the questions that were actually
>> tested.
>>
> 
> Not related to this patch, but please check if it is possible to detect those
> controllers.

If possible, I'll try to find an opportunity to check it out.

Best regards
Hans