[PATCH net-next v3 0/3] make skip_sw actually skip software

Asbjørn Sloth Tønnesen posted 3 patches 1 year, 9 months ago
There is a newer version of this series
include/net/pkt_cls.h     |  9 +++++++++
include/net/sch_generic.h |  4 ++++
net/core/dev.c            | 10 ++++++++++
net/sched/cls_api.c       | 41 +++++++++++++++++++++++++++++++++++++++
4 files changed, 64 insertions(+)
[PATCH net-next v3 0/3] make skip_sw actually skip software
Posted by Asbjørn Sloth Tønnesen 1 year, 9 months ago
Hi,

During development of flower-route[1], which I
recently presented at FOSDEM[2], I noticed that
CPU usage, would increase the more rules I installed
into the hardware for IP forwarding offloading.

Since we use TC flower offload for the hottest
prefixes, and leave the long tail to the normal (non-TC)
Linux network stack for slow-path IP forwarding.
We therefore need both the hardware and software
datapath to perform well.

I found that skip_sw rules, are quite expensive
in the kernel datapath, since they must be evaluated
and matched upon, before the kernel checks the
skip_sw flag.

This patchset optimizes the case where all rules
are skip_sw, by implementing a TC bypass for these
cases, where TC is only used as a control plane
for the hardware path.

CHanges from v2:
- Patch 3:
  - Fix source_inline
  - Fix build failure, when CONFIG_NET_CLS without CONFIG_NET_CLS_ACT.

Changes from v1:
- Patch 1:
  - Add Reviewed-By from Jiri Pirko
- Patch 2:
  - Move code, to avoid forward declaration (Jiri).
- Patch 3
  - Refactor to use a static key.
  - Add performance data for trapping, or sending
    a packet to a non-existent chain (as suggested by Marcelo).

[1] flower-route
    https://github.com/fiberby-dk/flower-route

[2] FOSDEM talk
    https://fosdem.org/2024/schedule/event/fosdem-2024-3337-flying-higher-hardware-offloading-with-bird/

Asbjørn Sloth Tønnesen (3):
  net: sched: cls_api: add skip_sw counter
  net: sched: cls_api: add filter counter
  net: sched: make skip_sw actually skip software

 include/net/pkt_cls.h     |  9 +++++++++
 include/net/sch_generic.h |  4 ++++
 net/core/dev.c            | 10 ++++++++++
 net/sched/cls_api.c       | 41 +++++++++++++++++++++++++++++++++++++++
 4 files changed, 64 insertions(+)

-- 
2.43.0

Re: [PATCH net-next v3 0/3] make skip_sw actually skip software
Posted by Jakub Kicinski 1 year, 9 months ago
On Wed,  6 Mar 2024 16:58:08 +0000 Asbjørn Sloth Tønnesen wrote:
> During development of flower-route[1], which I
> recently presented at FOSDEM[2], I noticed that
> CPU usage, would increase the more rules I installed
> into the hardware for IP forwarding offloading.
> 
> Since we use TC flower offload for the hottest
> prefixes, and leave the long tail to the normal (non-TC)
> Linux network stack for slow-path IP forwarding.
> We therefore need both the hardware and software
> datapath to perform well.
> 
> I found that skip_sw rules, are quite expensive
> in the kernel datapath, since they must be evaluated
> and matched upon, before the kernel checks the
> skip_sw flag.
> 
> This patchset optimizes the case where all rules
> are skip_sw, by implementing a TC bypass for these
> cases, where TC is only used as a control plane
> for the hardware path.

Linus tagged v6.8 and the merge window for v6.9 has started.
This feels a bit too risky for me to apply last minute,
could you repost in 2 weeks once the merge window is over?
-- 
pw-bot: defer
Re: [PATCH net-next v3 0/3] make skip_sw actually skip software
Posted by Asbjørn Sloth Tønnesen 1 year, 9 months ago
Hi Kuba,

On 3/11/24 8:44 PM, Jakub Kicinski wrote:
> On Wed,  6 Mar 2024 16:58:08 +0000 Asbjørn Sloth Tønnesen wrote:
>> During development of flower-route[1], which I
>> recently presented at FOSDEM[2], I noticed that
>> CPU usage, would increase the more rules I installed
>> into the hardware for IP forwarding offloading.
>>
>> Since we use TC flower offload for the hottest
>> prefixes, and leave the long tail to the normal (non-TC)
>> Linux network stack for slow-path IP forwarding.
>> We therefore need both the hardware and software
>> datapath to perform well.
>>
>> I found that skip_sw rules, are quite expensive
>> in the kernel datapath, since they must be evaluated
>> and matched upon, before the kernel checks the
>> skip_sw flag.
>>
>> This patchset optimizes the case where all rules
>> are skip_sw, by implementing a TC bypass for these
>> cases, where TC is only used as a control plane
>> for the hardware path.
> 
> Linus tagged v6.8 and the merge window for v6.9 has started.
> This feels a bit too risky for me to apply last minute,
> could you repost in 2 weeks once the merge window is over?

Sure, I will repost once net-next opens back up again.

-- 
Best regards
Asbjørn Sloth Tønnesen
Network Engineer
Fiberby - AS42541