[v1] net/mlx5: support software TX timestamp

[PATCH net-next] net/mlx5: support software TX timestamp

Posted by Stanislav Fomichev 7 months, 1 week ago

Having a software timestamp (along with existing hardware one) is
useful to trace how the packets flow through the stack.
mlx5e_tx_skb_update_hwts_flags is called from tx paths
to setup HW timestamp; extend it to add software one as well.

Signed-off-by: Stanislav Fomichev <stfomichev@gmail.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c | 1 +
 drivers/net/ethernet/mellanox/mlx5/core/en_tx.c      | 1 +
 2 files changed, 2 insertions(+)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
index fdf9e9bb99ac..e399d7a3d6cb 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
@@ -1689,6 +1689,7 @@ int mlx5e_ethtool_get_ts_info(struct mlx5e_priv *priv,
 		return 0;
 
 	info->so_timestamping = SOF_TIMESTAMPING_TX_HARDWARE |
+				SOF_TIMESTAMPING_TX_SOFTWARE |
 				SOF_TIMESTAMPING_RX_HARDWARE |
 				SOF_TIMESTAMPING_RAW_HARDWARE;
 
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
index 4fd853d19e31..f6dd26ad29e5 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
@@ -341,6 +341,7 @@ static void mlx5e_tx_skb_update_hwts_flags(struct sk_buff *skb)
 {
 	if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP))
 		skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS;
+	skb_tx_timestamp(skb);
 }
 
 static void mlx5e_tx_check_stop(struct mlx5e_txqsq *sq)
-- 
2.49.0

Re: [PATCH net-next] net/mlx5: support software TX timestamp

Posted by Tariq Toukan 7 months, 1 week ago


On 07/05/2025 0:55, Stanislav Fomichev wrote:
> Having a software timestamp (along with existing hardware one) is
> useful to trace how the packets flow through the stack.
> mlx5e_tx_skb_update_hwts_flags is called from tx paths
> to setup HW timestamp; extend it to add software one as well.
> 
> Signed-off-by: Stanislav Fomichev <stfomichev@gmail.com>
> ---
>   drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c | 1 +
>   drivers/net/ethernet/mellanox/mlx5/core/en_tx.c      | 1 +
>   2 files changed, 2 insertions(+)
> 
> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
> index fdf9e9bb99ac..e399d7a3d6cb 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
> @@ -1689,6 +1689,7 @@ int mlx5e_ethtool_get_ts_info(struct mlx5e_priv *priv,
>   		return 0;
>   
>   	info->so_timestamping = SOF_TIMESTAMPING_TX_HARDWARE |
> +				SOF_TIMESTAMPING_TX_SOFTWARE |
>   				SOF_TIMESTAMPING_RX_HARDWARE |
>   				SOF_TIMESTAMPING_RAW_HARDWARE;
>   
> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
> index 4fd853d19e31..f6dd26ad29e5 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
> @@ -341,6 +341,7 @@ static void mlx5e_tx_skb_update_hwts_flags(struct sk_buff *skb)
>   {
>   	if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP))
>   		skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS;
> +	skb_tx_timestamp(skb);

Doesn't this interfere with skb_tstamp_tx call in the completion flow 
(mlx5e_consume_skb)?

What happens if both flags (SKBTX_SW_TSTAMP / SKBTX_HW_TSTAMP) are set 
Is it possible?

>   }
>   
>   static void mlx5e_tx_check_stop(struct mlx5e_txqsq *sq)

Re: [PATCH net-next] net/mlx5: support software TX timestamp

Posted by Jason Xing 7 months, 1 week ago

Hi Tariq,

On Thu, May 8, 2025 at 2:30 PM Tariq Toukan <ttoukan.linux@gmail.com> wrote:
>
>
>
> On 07/05/2025 0:55, Stanislav Fomichev wrote:
> > Having a software timestamp (along with existing hardware one) is
> > useful to trace how the packets flow through the stack.
> > mlx5e_tx_skb_update_hwts_flags is called from tx paths
> > to setup HW timestamp; extend it to add software one as well.
> >
> > Signed-off-by: Stanislav Fomichev <stfomichev@gmail.com>
> > ---
> >   drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c | 1 +
> >   drivers/net/ethernet/mellanox/mlx5/core/en_tx.c      | 1 +
> >   2 files changed, 2 insertions(+)
> >
> > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
> > index fdf9e9bb99ac..e399d7a3d6cb 100644
> > --- a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
> > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
> > @@ -1689,6 +1689,7 @@ int mlx5e_ethtool_get_ts_info(struct mlx5e_priv *priv,
> >               return 0;
> >
> >       info->so_timestamping = SOF_TIMESTAMPING_TX_HARDWARE |
> > +                             SOF_TIMESTAMPING_TX_SOFTWARE |
> >                               SOF_TIMESTAMPING_RX_HARDWARE |
> >                               SOF_TIMESTAMPING_RAW_HARDWARE;
> >
> > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
> > index 4fd853d19e31..f6dd26ad29e5 100644
> > --- a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
> > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
> > @@ -341,6 +341,7 @@ static void mlx5e_tx_skb_update_hwts_flags(struct sk_buff *skb)
> >   {
> >       if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP))
> >               skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS;
> > +     skb_tx_timestamp(skb);
>
> Doesn't this interfere with skb_tstamp_tx call in the completion flow
> (mlx5e_consume_skb)?

skb_tstamp_tx() only takes care of hardware timestamp in this case.

>
> What happens if both flags (SKBTX_SW_TSTAMP / SKBTX_HW_TSTAMP) are set
> Is it possible?

If only these two are set, only hardware timestamp will be passed to
the userspace because of the SOF_TIMESTAMPING_OPT_TX_SWHW limit in
__skb_tstamp_tx().

If users expect to see both timestamps, then
SOF_TIMESTAMPING_OPT_TX_SWHW has to be set.

Thanks,
Jason

>
> >   }
> >
> >   static void mlx5e_tx_check_stop(struct mlx5e_txqsq *sq)
>
>

Re: [PATCH net-next] net/mlx5: support software TX timestamp

Posted by Stanislav Fomichev 7 months, 1 week ago

On 05/08, Jason Xing wrote:
> Hi Tariq,
> 
> On Thu, May 8, 2025 at 2:30 PM Tariq Toukan <ttoukan.linux@gmail.com> wrote:
> >
> >
> >
> > On 07/05/2025 0:55, Stanislav Fomichev wrote:
> > > Having a software timestamp (along with existing hardware one) is
> > > useful to trace how the packets flow through the stack.
> > > mlx5e_tx_skb_update_hwts_flags is called from tx paths
> > > to setup HW timestamp; extend it to add software one as well.
> > >
> > > Signed-off-by: Stanislav Fomichev <stfomichev@gmail.com>
> > > ---
> > >   drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c | 1 +
> > >   drivers/net/ethernet/mellanox/mlx5/core/en_tx.c      | 1 +
> > >   2 files changed, 2 insertions(+)
> > >
> > > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
> > > index fdf9e9bb99ac..e399d7a3d6cb 100644
> > > --- a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
> > > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
> > > @@ -1689,6 +1689,7 @@ int mlx5e_ethtool_get_ts_info(struct mlx5e_priv *priv,
> > >               return 0;
> > >
> > >       info->so_timestamping = SOF_TIMESTAMPING_TX_HARDWARE |
> > > +                             SOF_TIMESTAMPING_TX_SOFTWARE |
> > >                               SOF_TIMESTAMPING_RX_HARDWARE |
> > >                               SOF_TIMESTAMPING_RAW_HARDWARE;
> > >
> > > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
> > > index 4fd853d19e31..f6dd26ad29e5 100644
> > > --- a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
> > > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
> > > @@ -341,6 +341,7 @@ static void mlx5e_tx_skb_update_hwts_flags(struct sk_buff *skb)
> > >   {
> > >       if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP))
> > >               skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS;
> > > +     skb_tx_timestamp(skb);
> >
> > Doesn't this interfere with skb_tstamp_tx call in the completion flow
> > (mlx5e_consume_skb)?
> 
> skb_tstamp_tx() only takes care of hardware timestamp in this case.
> 
> >
> > What happens if both flags (SKBTX_SW_TSTAMP / SKBTX_HW_TSTAMP) are set
> > Is it possible?
> 
> If only these two are set, only hardware timestamp will be passed to
> the userspace because of the SOF_TIMESTAMPING_OPT_TX_SWHW limit in
> __skb_tstamp_tx().
> 
> If users expect to see both timestamps, then
> SOF_TIMESTAMPING_OPT_TX_SWHW has to be set.

Right, skb_tx_timestamp does nothing and bails out if it detects
SKBTX_IN_PROGRESS. And skb_tstamp_tx in mlx5e_consume_skb handles
only (and will trigger only for) HW tstamp case.

Re: [PATCH net-next] net/mlx5: support software TX timestamp

Posted by Tariq Toukan 7 months, 1 week ago


On 08/05/2025 19:22, Stanislav Fomichev wrote:
> On 05/08, Jason Xing wrote:
>> Hi Tariq,
>>
>> On Thu, May 8, 2025 at 2:30 PM Tariq Toukan <ttoukan.linux@gmail.com> wrote:
>>>
>>>
>>>
>>> On 07/05/2025 0:55, Stanislav Fomichev wrote:
>>>> Having a software timestamp (along with existing hardware one) is
>>>> useful to trace how the packets flow through the stack.
>>>> mlx5e_tx_skb_update_hwts_flags is called from tx paths
>>>> to setup HW timestamp; extend it to add software one as well.
>>>>
>>>> Signed-off-by: Stanislav Fomichev <stfomichev@gmail.com>
>>>> ---
>>>>    drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c | 1 +
>>>>    drivers/net/ethernet/mellanox/mlx5/core/en_tx.c      | 1 +
>>>>    2 files changed, 2 insertions(+)
>>>>
>>>> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
>>>> index fdf9e9bb99ac..e399d7a3d6cb 100644
>>>> --- a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
>>>> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
>>>> @@ -1689,6 +1689,7 @@ int mlx5e_ethtool_get_ts_info(struct mlx5e_priv *priv,
>>>>                return 0;
>>>>
>>>>        info->so_timestamping = SOF_TIMESTAMPING_TX_HARDWARE |
>>>> +                             SOF_TIMESTAMPING_TX_SOFTWARE |
>>>>                                SOF_TIMESTAMPING_RX_HARDWARE |
>>>>                                SOF_TIMESTAMPING_RAW_HARDWARE;
>>>>
>>>> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
>>>> index 4fd853d19e31..f6dd26ad29e5 100644
>>>> --- a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
>>>> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
>>>> @@ -341,6 +341,7 @@ static void mlx5e_tx_skb_update_hwts_flags(struct sk_buff *skb)
>>>>    {
>>>>        if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP))
>>>>                skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS;
>>>> +     skb_tx_timestamp(skb);
>>>
>>> Doesn't this interfere with skb_tstamp_tx call in the completion flow
>>> (mlx5e_consume_skb)?
>>
>> skb_tstamp_tx() only takes care of hardware timestamp in this case.
>>
>>>
>>> What happens if both flags (SKBTX_SW_TSTAMP / SKBTX_HW_TSTAMP) are set
>>> Is it possible?
>>
>> If only these two are set, only hardware timestamp will be passed to
>> the userspace because of the SOF_TIMESTAMPING_OPT_TX_SWHW limit in
>> __skb_tstamp_tx().
>>
>> If users expect to see both timestamps, then
>> SOF_TIMESTAMPING_OPT_TX_SWHW has to be set.
> 
> Right, skb_tx_timestamp does nothing and bails out if it detects
> SKBTX_IN_PROGRESS. And skb_tstamp_tx in mlx5e_consume_skb handles
> only (and will trigger only for) HW tstamp case.

I see.
Patch LGTM, except for the function name nit, pointed out in an earlier 
comment.
We could remove the "hw" from function name mlx5e_tx_skb_update_hwts_flags.

Re: [PATCH net-next] net/mlx5: support software TX timestamp

Posted by Stanislav Fomichev 7 months, 1 week ago

On 05/09, Tariq Toukan wrote:
> 
> 
> On 08/05/2025 19:22, Stanislav Fomichev wrote:
> > On 05/08, Jason Xing wrote:
> > > Hi Tariq,
> > > 
> > > On Thu, May 8, 2025 at 2:30 PM Tariq Toukan <ttoukan.linux@gmail.com> wrote:
> > > > 
> > > > 
> > > > 
> > > > On 07/05/2025 0:55, Stanislav Fomichev wrote:
> > > > > Having a software timestamp (along with existing hardware one) is
> > > > > useful to trace how the packets flow through the stack.
> > > > > mlx5e_tx_skb_update_hwts_flags is called from tx paths
> > > > > to setup HW timestamp; extend it to add software one as well.
> > > > > 
> > > > > Signed-off-by: Stanislav Fomichev <stfomichev@gmail.com>
> > > > > ---
> > > > >    drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c | 1 +
> > > > >    drivers/net/ethernet/mellanox/mlx5/core/en_tx.c      | 1 +
> > > > >    2 files changed, 2 insertions(+)
> > > > > 
> > > > > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
> > > > > index fdf9e9bb99ac..e399d7a3d6cb 100644
> > > > > --- a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
> > > > > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
> > > > > @@ -1689,6 +1689,7 @@ int mlx5e_ethtool_get_ts_info(struct mlx5e_priv *priv,
> > > > >                return 0;
> > > > > 
> > > > >        info->so_timestamping = SOF_TIMESTAMPING_TX_HARDWARE |
> > > > > +                             SOF_TIMESTAMPING_TX_SOFTWARE |
> > > > >                                SOF_TIMESTAMPING_RX_HARDWARE |
> > > > >                                SOF_TIMESTAMPING_RAW_HARDWARE;
> > > > > 
> > > > > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
> > > > > index 4fd853d19e31..f6dd26ad29e5 100644
> > > > > --- a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
> > > > > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
> > > > > @@ -341,6 +341,7 @@ static void mlx5e_tx_skb_update_hwts_flags(struct sk_buff *skb)
> > > > >    {
> > > > >        if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP))
> > > > >                skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS;
> > > > > +     skb_tx_timestamp(skb);
> > > > 
> > > > Doesn't this interfere with skb_tstamp_tx call in the completion flow
> > > > (mlx5e_consume_skb)?
> > > 
> > > skb_tstamp_tx() only takes care of hardware timestamp in this case.
> > > 
> > > > 
> > > > What happens if both flags (SKBTX_SW_TSTAMP / SKBTX_HW_TSTAMP) are set
> > > > Is it possible?
> > > 
> > > If only these two are set, only hardware timestamp will be passed to
> > > the userspace because of the SOF_TIMESTAMPING_OPT_TX_SWHW limit in
> > > __skb_tstamp_tx().
> > > 
> > > If users expect to see both timestamps, then
> > > SOF_TIMESTAMPING_OPT_TX_SWHW has to be set.
> > 
> > Right, skb_tx_timestamp does nothing and bails out if it detects
> > SKBTX_IN_PROGRESS. And skb_tstamp_tx in mlx5e_consume_skb handles
> > only (and will trigger only for) HW tstamp case.
> 
> I see.
> Patch LGTM, except for the function name nit, pointed out in an earlier
> comment.
> We could remove the "hw" from function name mlx5e_tx_skb_update_hwts_flags.

SG, will repost with a rename!

Re: [PATCH net-next] net/mlx5: support software TX timestamp

Posted by Jason Xing 7 months, 1 week ago

On Wed, May 7, 2025 at 5:57 AM Stanislav Fomichev <stfomichev@gmail.com> wrote:
>
> Having a software timestamp (along with existing hardware one) is
> useful to trace how the packets flow through the stack.
> mlx5e_tx_skb_update_hwts_flags is called from tx paths
> to setup HW timestamp; extend it to add software one as well.
>
> Signed-off-by: Stanislav Fomichev <stfomichev@gmail.com>

Reviewed-by: Jason Xing <kerneljasonxing@gmail.com>

Only one nit as below.

> ---
>  drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c | 1 +
>  drivers/net/ethernet/mellanox/mlx5/core/en_tx.c      | 1 +
>  2 files changed, 2 insertions(+)
>
> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
> index fdf9e9bb99ac..e399d7a3d6cb 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
> @@ -1689,6 +1689,7 @@ int mlx5e_ethtool_get_ts_info(struct mlx5e_priv *priv,
>                 return 0;
>
>         info->so_timestamping = SOF_TIMESTAMPING_TX_HARDWARE |
> +                               SOF_TIMESTAMPING_TX_SOFTWARE |
>                                 SOF_TIMESTAMPING_RX_HARDWARE |
>                                 SOF_TIMESTAMPING_RAW_HARDWARE;
>
> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
> index 4fd853d19e31..f6dd26ad29e5 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c
> @@ -341,6 +341,7 @@ static void mlx5e_tx_skb_update_hwts_flags(struct sk_buff *skb)

nit: the function name including 'hwts' doesn't reflect the following
software behavior.

Thanks,
Jason

>  {
>         if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP))
>                 skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS;
> +       skb_tx_timestamp(skb);
>  }
>
>  static void mlx5e_tx_check_stop(struct mlx5e_txqsq *sq)
> --
> 2.49.0
>
>