From: Geetha sowjanya <gakula@marvell.com>
APR table contains the lmtst base address of PF/VFs.
These entries are updated by the PF/VF during the
device probe. Due to race condition while updating the
entries are getting corrupted. Hence secure the APR
table update with the lock.
Fixes: 893ae97214c3 ("octeontx2-af: cn10k: Support configurable LMTST regions")
Signed-off-by: Geetha sowjanya <gakula@marvell.com>
Signed-off-by: Sunil Kovvuri Goutham <sgoutham@marvell.com>
Signed-off-by: Sai Krishna <saikrishnag@marvell.com>
---
drivers/net/ethernet/marvell/octeontx2/af/rvu_cn10k.c | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_cn10k.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu_cn10k.c
index 4ad9ff025c96..8530250f6fba 100644
--- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_cn10k.c
+++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_cn10k.c
@@ -142,16 +142,17 @@ int rvu_mbox_handler_lmtst_tbl_setup(struct rvu *rvu,
* region, if so, convert that IOVA to physical address and
* populate LMT table with that address
*/
+ mutex_lock(&rvu->rsrc_lock);
if (req->use_local_lmt_region) {
err = rvu_get_lmtaddr(rvu, req->hdr.pcifunc,
req->lmt_iova, &lmt_addr);
if (err < 0)
- return err;
+ goto error;
/* Update the lmt addr for this PFFUNC in the LMT table */
err = rvu_update_lmtaddr(rvu, req->hdr.pcifunc, lmt_addr);
if (err)
- return err;
+ goto error;
}
/* Reconfiguring lmtst map table in lmt region shared mode i.e. make
@@ -181,7 +182,7 @@ int rvu_mbox_handler_lmtst_tbl_setup(struct rvu *rvu,
*/
err = rvu_update_lmtaddr(rvu, req->hdr.pcifunc, val);
if (err)
- return err;
+ goto error;
}
/* This mailbox can also be used to update word1 of APR_LMT_MAP_ENTRY_S
@@ -230,6 +231,7 @@ int rvu_mbox_handler_lmtst_tbl_setup(struct rvu *rvu,
}
error:
+ mutex_unlock(&rvu->rsrc_lock);
return err;
}
--
2.25.1
On Wed, Mar 29, 2023 at 10:36:13PM +0530, Sai Krishna wrote:
> From: Geetha sowjanya <gakula@marvell.com>
>
> APR table contains the lmtst base address of PF/VFs.
> These entries are updated by the PF/VF during the
> device probe. Due to race condition while updating the
> entries are getting corrupted. Hence secure the APR
> table update with the lock.
However, I don't see rsrc_lock in probe path.
otx2_probe()
-> cn10k_lmtst_init()
-> lmt_base/lmstst is updated with and without mbox.lock.
Where did you take rsrc_lock in probe flow?
Thanks
>
> Fixes: 893ae97214c3 ("octeontx2-af: cn10k: Support configurable LMTST regions")
> Signed-off-by: Geetha sowjanya <gakula@marvell.com>
> Signed-off-by: Sunil Kovvuri Goutham <sgoutham@marvell.com>
> Signed-off-by: Sai Krishna <saikrishnag@marvell.com>
> ---
> drivers/net/ethernet/marvell/octeontx2/af/rvu_cn10k.c | 8 +++++---
> 1 file changed, 5 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_cn10k.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu_cn10k.c
> index 4ad9ff025c96..8530250f6fba 100644
> --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_cn10k.c
> +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_cn10k.c
> @@ -142,16 +142,17 @@ int rvu_mbox_handler_lmtst_tbl_setup(struct rvu *rvu,
> * region, if so, convert that IOVA to physical address and
> * populate LMT table with that address
> */
> + mutex_lock(&rvu->rsrc_lock);
> if (req->use_local_lmt_region) {
> err = rvu_get_lmtaddr(rvu, req->hdr.pcifunc,
> req->lmt_iova, &lmt_addr);
> if (err < 0)
> - return err;
> + goto error;
>
> /* Update the lmt addr for this PFFUNC in the LMT table */
> err = rvu_update_lmtaddr(rvu, req->hdr.pcifunc, lmt_addr);
> if (err)
> - return err;
> + goto error;
> }
>
> /* Reconfiguring lmtst map table in lmt region shared mode i.e. make
> @@ -181,7 +182,7 @@ int rvu_mbox_handler_lmtst_tbl_setup(struct rvu *rvu,
> */
> err = rvu_update_lmtaddr(rvu, req->hdr.pcifunc, val);
> if (err)
> - return err;
> + goto error;
> }
>
> /* This mailbox can also be used to update word1 of APR_LMT_MAP_ENTRY_S
> @@ -230,6 +231,7 @@ int rvu_mbox_handler_lmtst_tbl_setup(struct rvu *rvu,
> }
>
> error:
> + mutex_unlock(&rvu->rsrc_lock);
> return err;
> }
>
> --
> 2.25.1
>
>-----Original Message-----
>From: Leon Romanovsky <leon@kernel.org>
>Sent: Thursday, March 30, 2023 11:26 AM
>To: Sai Krishna Gajula <saikrishnag@marvell.com>
>Cc: davem@davemloft.net; edumazet@google.com; kuba@kernel.org; pabeni@redhat.com; netdev@vger.kernel.org; linux-kernel@vger.kernel.org; Sunil Kovvuri Goutham <sgoutham@marvell.com>; >richardcochran@gmail.com; Geethasowjanya Akula <gakula@marvell.com>
>Subject: [EXT] Re: [net PATCH 1/7] octeontx2-af: Secure APR table update with the lock
>External Email
>----------------------------------------------------------------------
>On Wed, Mar 29, 2023 at 10:36:13PM +0530, Sai Krishna wrote:
>> From: Geetha sowjanya <gakula@marvell.com>
>>
>> APR table contains the lmtst base address of PF/VFs.
>> These entries are updated by the PF/VF during the device probe. Due to
>> race condition while updating the entries are getting corrupted. Hence
>> secure the APR table update with the lock.
>However, I don't see rsrc_lock in probe path.
>otx2_probe()
>-> cn10k_lmtst_init()
> -> lmt_base/lmstst is updated with and without mbox.lock.
>Where did you take rsrc_lock in probe flow?
rsrc_lock is initialized in AF driver. PF/VF driver in cn10k_lmtst_init() send a mbox request to AF to update the lmtst table.
mbox handler in AF takes rsrc_lock to update lmtst table.
Thanks,
Geetha.
>Thanks
>>
>> Fixes: 893ae97214c3 ("octeontx2-af: cn10k: Support configurable LMTST
>> regions")
>> Signed-off-by: Geetha sowjanya <gakula@marvell.com>
>> Signed-off-by: Sunil Kovvuri Goutham <sgoutham@marvell.com>
>> Signed-off-by: Sai Krishna <saikrishnag@marvell.com>
>> ---
>> drivers/net/ethernet/marvell/octeontx2/af/rvu_cn10k.c | 8 +++++---
>> 1 file changed, 5 insertions(+), 3 deletions(-)
>>
>> diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_cn10k.c
>> b/drivers/net/ethernet/marvell/octeontx2/af/rvu_cn10k.c
>> index 4ad9ff025c96..8530250f6fba 100644
>> --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_cn10k.c
>> +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_cn10k.c
>> @@ -142,16 +142,17 @@ int rvu_mbox_handler_lmtst_tbl_setup(struct rvu *rvu,
>> * region, if so, convert that IOVA to physical address and
>> * populate LMT table with that address
>> */
>> + mutex_lock(&rvu->rsrc_lock);
>> if (req->use_local_lmt_region) {
>> err = rvu_get_lmtaddr(rvu, req->hdr.pcifunc,
>> req->lmt_iova, &lmt_addr);
>> if (err < 0)
>> - return err;
>> + goto error;
>>
>> /* Update the lmt addr for this PFFUNC in the LMT table */
>> err = rvu_update_lmtaddr(rvu, req->hdr.pcifunc, lmt_addr);
>> if (err)
>> - return err;
>> + goto error;
>> }
>>
>> /* Reconfiguring lmtst map table in lmt region shared mode i.e. make
>> @@ -181,7 +182,7 @@ int rvu_mbox_handler_lmtst_tbl_setup(struct rvu *rvu,
>> */
>> err = rvu_update_lmtaddr(rvu, req->hdr.pcifunc, val);
>> if (err)
>> - return err;
>> + goto error;
>> }
>>
>> /* This mailbox can also be used to update word1 of
>> APR_LMT_MAP_ENTRY_S @@ -230,6 +231,7 @@ int rvu_mbox_handler_lmtst_tbl_setup(struct rvu *rvu,
>> }
>>
>> error:
>> + mutex_unlock(&rvu->rsrc_lock);
>> return err;
>> }
>>
>> --
>> 2.25.1
>>
On Thu, Mar 30, 2023 at 06:56:54AM +0000, Geethasowjanya Akula wrote:
>
> >-----Original Message-----
> >From: Leon Romanovsky <leon@kernel.org>
> >Sent: Thursday, March 30, 2023 11:26 AM
> >To: Sai Krishna Gajula <saikrishnag@marvell.com>
> >Cc: davem@davemloft.net; edumazet@google.com; kuba@kernel.org; pabeni@redhat.com; netdev@vger.kernel.org; linux-kernel@vger.kernel.org; Sunil Kovvuri Goutham <sgoutham@marvell.com>; >richardcochran@gmail.com; Geethasowjanya Akula <gakula@marvell.com>
> >Subject: [EXT] Re: [net PATCH 1/7] octeontx2-af: Secure APR table update with the lock
>
> >External Email
>
> >----------------------------------------------------------------------
> >On Wed, Mar 29, 2023 at 10:36:13PM +0530, Sai Krishna wrote:
> >> From: Geetha sowjanya <gakula@marvell.com>
> >>
> >> APR table contains the lmtst base address of PF/VFs.
> >> These entries are updated by the PF/VF during the device probe. Due to
> >> race condition while updating the entries are getting corrupted. Hence
> >> secure the APR table update with the lock.
>
> >However, I don't see rsrc_lock in probe path.
> >otx2_probe()
> >-> cn10k_lmtst_init()
> > -> lmt_base/lmstst is updated with and without mbox.lock.
>
> >Where did you take rsrc_lock in probe flow?
>
> rsrc_lock is initialized in AF driver. PF/VF driver in cn10k_lmtst_init() send a mbox request to AF to update the lmtst table.
> mbox handler in AF takes rsrc_lock to update lmtst table.
Can you please present the stack trace of such flow? What are the actual variables/struct rsrc_lock
is protecting?
Thanks
>
> Thanks,
> Geetha.
>
> >Thanks
>
> >>
> >> Fixes: 893ae97214c3 ("octeontx2-af: cn10k: Support configurable LMTST
> >> regions")
> >> Signed-off-by: Geetha sowjanya <gakula@marvell.com>
> >> Signed-off-by: Sunil Kovvuri Goutham <sgoutham@marvell.com>
> >> Signed-off-by: Sai Krishna <saikrishnag@marvell.com>
> >> ---
> >> drivers/net/ethernet/marvell/octeontx2/af/rvu_cn10k.c | 8 +++++---
> >> 1 file changed, 5 insertions(+), 3 deletions(-)
> >>
> >> diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_cn10k.c
> >> b/drivers/net/ethernet/marvell/octeontx2/af/rvu_cn10k.c
> >> index 4ad9ff025c96..8530250f6fba 100644
> >> --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_cn10k.c
> >> +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_cn10k.c
> >> @@ -142,16 +142,17 @@ int rvu_mbox_handler_lmtst_tbl_setup(struct rvu *rvu,
> >> * region, if so, convert that IOVA to physical address and
> >> * populate LMT table with that address
> >> */
> >> + mutex_lock(&rvu->rsrc_lock);
> >> if (req->use_local_lmt_region) {
> >> err = rvu_get_lmtaddr(rvu, req->hdr.pcifunc,
> >> req->lmt_iova, &lmt_addr);
> >> if (err < 0)
> >> - return err;
> >> + goto error;
> >>
> >> /* Update the lmt addr for this PFFUNC in the LMT table */
> >> err = rvu_update_lmtaddr(rvu, req->hdr.pcifunc, lmt_addr);
> >> if (err)
> >> - return err;
> >> + goto error;
> >> }
> >>
> >> /* Reconfiguring lmtst map table in lmt region shared mode i.e. make
> >> @@ -181,7 +182,7 @@ int rvu_mbox_handler_lmtst_tbl_setup(struct rvu *rvu,
> >> */
> >> err = rvu_update_lmtaddr(rvu, req->hdr.pcifunc, val);
> >> if (err)
> >> - return err;
> >> + goto error;
> >> }
> >>
> >> /* This mailbox can also be used to update word1 of
> >> APR_LMT_MAP_ENTRY_S @@ -230,6 +231,7 @@ int rvu_mbox_handler_lmtst_tbl_setup(struct rvu *rvu,
> >> }
> >>
> >> error:
> >> + mutex_unlock(&rvu->rsrc_lock);
> >> return err;
> >> }
> >>
> >> --
> >> 2.25.1
> >>
>-----Original Message-----
>From: Leon Romanovsky <leon@kernel.org>
>Sent: Thursday, March 30, 2023 12:46 PM
>To: Geethasowjanya Akula <gakula@marvell.com>
>Cc: Sai Krishna Gajula <saikrishnag@marvell.com>; davem@davemloft.net; edumazet@google.com; kuba@kernel.org; pabeni@redhat.com; netdev@vger.kernel.org; linux-kernel@vger.kernel.org; Sunil >Kovvuri Goutham <sgoutham@marvell.com>; richardcochran@gmail.com
>Subject: Re: [EXT] Re: [net PATCH 1/7] octeontx2-af: Secure APR table update with the lock
>
>On Thu, Mar 30, 2023 at 06:56:54AM +0000, Geethasowjanya Akula wrote:
>>
>> >-----Original Message-----
>> >From: Leon Romanovsky <leon@kernel.org>
>> >Sent: Thursday, March 30, 2023 11:26 AM
>> >To: Sai Krishna Gajula <saikrishnag@marvell.com>
>> >Cc: davem@davemloft.net; edumazet@google.com; kuba@kernel.org;
>> >pabeni@redhat.com; netdev@vger.kernel.org;
>> >linux-kernel@vger.kernel.org; Sunil Kovvuri Goutham
>> ><sgoutham@marvell.com>; >richardcochran@gmail.com; Geethasowjanya
>> >Akula <gakula@marvell.com>
>> >Subject: [EXT] Re: [net PATCH 1/7] octeontx2-af: Secure APR table
>> >update with the lock
>>
>> >External Email
>>
>> >---------------------------------------------------------------------
>> >- On Wed, Mar 29, 2023 at 10:36:13PM +0530, Sai Krishna wrote:
>> >> From: Geetha sowjanya <gakula@marvell.com>
>> >>
>> >> APR table contains the lmtst base address of PF/VFs.
>> >> These entries are updated by the PF/VF during the device probe. Due
>> >> to race condition while updating the entries are getting corrupted.
>> >> Hence secure the APR table update with the lock.
>>
>> >However, I don't see rsrc_lock in probe path.
>> >otx2_probe()
>> >-> cn10k_lmtst_init()
>> > -> lmt_base/lmstst is updated with and without mbox.lock.
>>
>> >Where did you take rsrc_lock in probe flow?
>>
>> rsrc_lock is initialized in AF driver. PF/VF driver in cn10k_lmtst_init() send a mbox request to AF to update the lmtst table.
>> mbox handler in AF takes rsrc_lock to update lmtst table.
>Can you please present the stack trace of such flow? What are the actual variables/struct rsrc_lock is protecting?
The lock tries to protect the request and response register at line#73 and line#83 in below function, from getting overwritten when
Multiple PFs invokes rvu_get_lmtaddr() simultaneously.
For example, if PF1 submit the request at line#73 and got permitted before it reads the response at line#80.
PF2 got scheduled submit the request then the response of PF1 is overwritten by the PF2 response.
When PF1 gets reschedule, it reads wrong data.
#static int rvu_get_lmtaddr(struct rvu *rvu, u16 pcifunc,
59 u64 iova, u64 *lmt_addr)
60 {
61 [...]
68
69 rvu_write64(rvu, BLKADDR_RVUM, RVU_AF_SMMU_ADDR_REQ, iova);
70 pf = rvu_get_pf(pcifunc) & 0x1F;
71 val = BIT_ULL(63) | BIT_ULL(14) | BIT_ULL(13) | pf << 8 |
72 ((pcifunc & RVU_PFVF_FUNC_MASK) & 0xFF);
73 rvu_write64(rvu, BLKADDR_RVUM, RVU_AF_SMMU_TXN_REQ, val);
74
75 err = rvu_poll_reg(rvu, BLKADDR_RVUM, RVU_AF_SMMU_ADDR_RSP_STS, BIT_ULL(0), false);
76 if (err) {
77 dev_err(rvu->dev, "%s LMTLINE iova transulation failed\n", __func__);
78 return err;
79 }
80 val = rvu_read64(rvu, BLKADDR_RVUM, RVU_AF_SMMU_ADDR_RSP_STS);
81 if (val & ~0x1ULL) {
82 dev_err(rvu->dev, "%s LMTLINE iova transulation failed err:%llx\n", __func__, val);
83 return -EIO;
84 }
85
Thanks.
>>Thanks
>>
>> Thanks,
>> Geetha.
>>
>> >Thanks
>>
>> >>
>> >> Fixes: 893ae97214c3 ("octeontx2-af: cn10k: Support configurable
>> >> LMTST
>> >> regions")
>> >> Signed-off-by: Geetha sowjanya <gakula@marvell.com>
>> >> Signed-off-by: Sunil Kovvuri Goutham <sgoutham@marvell.com>
>> >> Signed-off-by: Sai Krishna <saikrishnag@marvell.com>
>> >> ---
>> >> drivers/net/ethernet/marvell/octeontx2/af/rvu_cn10k.c | 8 +++++---
>> >> 1 file changed, 5 insertions(+), 3 deletions(-)
>> >>
>> >> diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_cn10k.c
>> >> b/drivers/net/ethernet/marvell/octeontx2/af/rvu_cn10k.c
>> >> index 4ad9ff025c96..8530250f6fba 100644
>> >> --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_cn10k.c
>> >> +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_cn10k.c
>> >> @@ -142,16 +142,17 @@ int rvu_mbox_handler_lmtst_tbl_setup(struct rvu *rvu,
>> >> * region, if so, convert that IOVA to physical address and
>> >> * populate LMT table with that address
>> >> */
>> >> + mutex_lock(&rvu->rsrc_lock);
>> >> if (req->use_local_lmt_region) {
>> >> err = rvu_get_lmtaddr(rvu, req->hdr.pcifunc,
>> >> req->lmt_iova, &lmt_addr);
>> >> if (err < 0)
>> >> - return err;
>> >> + goto error;
>> >>
>> >> /* Update the lmt addr for this PFFUNC in the LMT table */
>> >> err = rvu_update_lmtaddr(rvu, req->hdr.pcifunc, lmt_addr);
>> >> if (err)
>> >> - return err;
>> >> + goto error;
>> >> }
>> >>
>> >> /* Reconfiguring lmtst map table in lmt region shared mode i.e.
>> >> make @@ -181,7 +182,7 @@ int rvu_mbox_handler_lmtst_tbl_setup(struct rvu *rvu,
>> >> */
>> >> err = rvu_update_lmtaddr(rvu, req->hdr.pcifunc, val);
>> >> if (err)
>> >> - return err;
>> >> + goto error;
>> >> }
>> >>
>> >> /* This mailbox can also be used to update word1 of
>> >> APR_LMT_MAP_ENTRY_S @@ -230,6 +231,7 @@ int rvu_mbox_handler_lmtst_tbl_setup(struct rvu *rvu,
>> >> }
>> >>
>> >> error:
>> >> + mutex_unlock(&rvu->rsrc_lock);
>> >> return err;
>> >> }
>> >>
>> >> --
>> >> 2.25.1
>> >>
On Fri, Mar 31, 2023 at 08:56:59AM +0000, Geethasowjanya Akula wrote: > > > >-----Original Message----- > >From: Leon Romanovsky <leon@kernel.org> > >Sent: Thursday, March 30, 2023 12:46 PM > >To: Geethasowjanya Akula <gakula@marvell.com> > >Cc: Sai Krishna Gajula <saikrishnag@marvell.com>; davem@davemloft.net; edumazet@google.com; kuba@kernel.org; pabeni@redhat.com; netdev@vger.kernel.org; linux-kernel@vger.kernel.org; Sunil >Kovvuri Goutham <sgoutham@marvell.com>; richardcochran@gmail.com > >Subject: Re: [EXT] Re: [net PATCH 1/7] octeontx2-af: Secure APR table update with the lock > > > >On Thu, Mar 30, 2023 at 06:56:54AM +0000, Geethasowjanya Akula wrote: > >> > >> >-----Original Message----- > >> >From: Leon Romanovsky <leon@kernel.org> > >> >Sent: Thursday, March 30, 2023 11:26 AM > >> >To: Sai Krishna Gajula <saikrishnag@marvell.com> > >> >Cc: davem@davemloft.net; edumazet@google.com; kuba@kernel.org; > >> >pabeni@redhat.com; netdev@vger.kernel.org; > >> >linux-kernel@vger.kernel.org; Sunil Kovvuri Goutham > >> ><sgoutham@marvell.com>; >richardcochran@gmail.com; Geethasowjanya > >> >Akula <gakula@marvell.com> > >> >Subject: [EXT] Re: [net PATCH 1/7] octeontx2-af: Secure APR table > >> >update with the lock > >> > >> >External Email > >> > >> >--------------------------------------------------------------------- > >> >- On Wed, Mar 29, 2023 at 10:36:13PM +0530, Sai Krishna wrote: > >> >> From: Geetha sowjanya <gakula@marvell.com> > >> >> > >> >> APR table contains the lmtst base address of PF/VFs. > >> >> These entries are updated by the PF/VF during the device probe. Due > >> >> to race condition while updating the entries are getting corrupted. > >> >> Hence secure the APR table update with the lock. > >> > >> >However, I don't see rsrc_lock in probe path. > >> >otx2_probe() > >> >-> cn10k_lmtst_init() > >> > -> lmt_base/lmstst is updated with and without mbox.lock. > >> > >> >Where did you take rsrc_lock in probe flow? > >> > >> rsrc_lock is initialized in AF driver. PF/VF driver in cn10k_lmtst_init() send a mbox request to AF to update the lmtst table. > >> mbox handler in AF takes rsrc_lock to update lmtst table. > > >Can you please present the stack trace of such flow? What are the actual variables/struct rsrc_lock is protecting? > > The lock tries to protect the request and response register at line#73 and line#83 in below function, from getting overwritten when > Multiple PFs invokes rvu_get_lmtaddr() simultaneously. > For example, if PF1 submit the request at line#73 and got permitted before it reads the response at line#80. > PF2 got scheduled submit the request then the response of PF1 is overwritten by the PF2 response. > When PF1 gets reschedule, it reads wrong data. I see, thanks
© 2016 - 2026 Red Hat, Inc.