[PATCH 0/4] network: firewalld: fix routed network

Eric Garver posted 4 patches 4 days, 21 hours ago
Failed in applying to current master (apply log)
src/network/bridge_driver_linux.c     |  6 +++++-
src/network/libvirt-nat-out.policy    | 12 ++++++++++++
src/network/libvirt-routed-in.policy  | 11 +++++++++++
src/network/libvirt-routed-out.policy | 12 ++++++++++++
src/network/libvirt-routed.zone       | 12 ++++++++++++
src/network/libvirt-to-host.policy    | 21 +++++++++++++++++++++
src/network/libvirt.zone              | 23 +++++------------------
src/network/meson.build               | 25 +++++++++++++++++++++++++
8 files changed, 103 insertions(+), 19 deletions(-)
create mode 100644 src/network/libvirt-nat-out.policy
create mode 100644 src/network/libvirt-routed-in.policy
create mode 100644 src/network/libvirt-routed-out.policy
create mode 100644 src/network/libvirt-routed.zone
create mode 100644 src/network/libvirt-to-host.policy
[PATCH 0/4] network: firewalld: fix routed network
Posted by Eric Garver 4 days, 21 hours ago
This series fixes routed networks when a newer firewalld (>= 1.0.0) is
present [1]. Firewalld 1.0.0 included a change that disallows implicit
forwarding between zones [2]. libvirt was relying on this behavior to
allow routed networks to function.

New firewalld policies are added. This is done to use common rules
between NAT and routed networks. Policies have been supported since
firewalld 0.9.0.

[1]: https://bugzilla.redhat.com/show_bug.cgi?id=2055706
[2]: https://github.com/firewalld/firewalld/issues/177

Eric Garver (4):
  network: firewalld: convert to policies
  network: firewalld: add zone for routed networks
  network: firewalld: add policies for routed networks
  network: firewalld: add support for routed networks

 src/network/bridge_driver_linux.c     |  6 +++++-
 src/network/libvirt-nat-out.policy    | 12 ++++++++++++
 src/network/libvirt-routed-in.policy  | 11 +++++++++++
 src/network/libvirt-routed-out.policy | 12 ++++++++++++
 src/network/libvirt-routed.zone       | 12 ++++++++++++
 src/network/libvirt-to-host.policy    | 21 +++++++++++++++++++++
 src/network/libvirt.zone              | 23 +++++------------------
 src/network/meson.build               | 25 +++++++++++++++++++++++++
 8 files changed, 103 insertions(+), 19 deletions(-)
 create mode 100644 src/network/libvirt-nat-out.policy
 create mode 100644 src/network/libvirt-routed-in.policy
 create mode 100644 src/network/libvirt-routed-out.policy
 create mode 100644 src/network/libvirt-routed.zone
 create mode 100644 src/network/libvirt-to-host.policy

-- 
2.33.0
Re: [PATCH 0/4] network: firewalld: fix routed network
Posted by Daniel P. Berrangé 3 days, 19 hours ago
On Wed, May 11, 2022 at 11:41:51AM -0400, Eric Garver wrote:
> This series fixes routed networks when a newer firewalld (>= 1.0.0) is
> present [1]. Firewalld 1.0.0 included a change that disallows implicit
> forwarding between zones [2]. libvirt was relying on this behavior to
> allow routed networks to function.
> 
> New firewalld policies are added. This is done to use common rules
> between NAT and routed networks. Policies have been supported since
> firewalld 0.9.0.

For those following along, there's a helpful description of policies
here, specifically explaining how its useful to the libvirt scenario:

  https://firewalld.org/2020/09/policy-objects-introduction

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
Re: [PATCH 0/4] network: firewalld: fix routed network
Posted by Daniel P. Berrangé 3 days, 17 hours ago
On Thu, May 12, 2022 at 07:00:09PM +0100, Daniel P. Berrangé wrote:
> On Wed, May 11, 2022 at 11:41:51AM -0400, Eric Garver wrote:
> > This series fixes routed networks when a newer firewalld (>= 1.0.0) is
> > present [1]. Firewalld 1.0.0 included a change that disallows implicit
> > forwarding between zones [2]. libvirt was relying on this behavior to
> > allow routed networks to function.
> > 
> > New firewalld policies are added. This is done to use common rules
> > between NAT and routed networks. Policies have been supported since
> > firewalld 0.9.0.
> 
> For those following along, there's a helpful description of policies
> here, specifically explaining how its useful to the libvirt scenario:
> 
>   https://firewalld.org/2020/09/policy-objects-introduction

In reviewing these patches I've come to realize I'm still not
confident I'm understanding the interaction between traffic
we're managing at the firewalld  zones/policies.

For illustration let me assume the following setup:
[
   * Remote host on LAN (remote host IP 10.0.0.2)

   * eth0 public facing ethernet on the LAN (local host IP 10.0.0.5)

   * virbr0 isolated bridge device (local host IP 192.168.122.1)

   * vnet0 TAP device for a guest (guest IP 192.168.122.5)


   Remote host            Local host

  +----------+    LAN   +----------+   IP forward  +---------------+
  | 10.0.0.2 | -------- | 10.0.0.5 | --------------| 192.168.122.1 |
  | eth0     |          |  eth0    |   w/ NAT      |  virbr0       |
  +----------+          +----------+               +---------------+
                                                         |
							 | bridge port
                                                         |
                                                   +---------------+
                                                   | 192.168.122.5 |
						   | host: vnet0   |
						   | guest: eth0   |
						   +---------------+

IIUC zones are

   * 'libvirt' containing 'virbr0'
   * 'FedoraWorkstation' containing 'eth0'

Is 'vnet0' in a zone or not ?



Traffic flows


   * LAN Remote host (10.0.0.2) -> local host (10.0.0.5)

     Normal traffic nothing to do with libvirt

     Rules in <zone> FedoraWorkstation apply


   * LAN Remote host (10.0.0.2) -> guest (192.168.122.5)

     IP layer forwarding via eth0 (with conntrack match for NAT zone)

     ingress=FedoraWorkstation
     egress=libvirt

     Rules in <policy> libvirt-host-in apply ?


   * Local host (192.168.122.1) -> guest (192.168.122.5)

     Rules in <zone> libvirt apply ?


   * Local host (10.0.0.5) -> guest (192.168.122.5)

     NB, shouldn't happen as traffic should have originated
     from 192.168.122.1 instead.

     ingress=FedoraWorkstation
     egress=libvirt

     Rules in <policy> libvirt-host-in apply ?


   * Guest (192.168.122.5) -> Local host (192.168.122.1)

     Rules in <zone> libvirt apply ?

     Need to allow dhcp, dns, ssh. Feels like this
     should still be rules in the <zone> ?


   * Guest (192.168.122.5) -> Local host (10.0.0.5)

     NB, shouldn't happen as guest generally won't be
     aware of host's eth0 IP address.

     ingress=libvirt
     egress=FedoraWorkstation

     Rules in <policy> libvirt-nat-out apply ?

     Should not allow anything special related to virt,
     as dhcp/dns stuff should only be serviced from virbr0.
     So the libvirt-nat-out policy feels wrong for this
     case.


   * Guest (192.168.122.5) -> LAN remote host (10.0.0.2)

     ingress=libvirt
     egress=FedoraWorkstation

     Rules in <policy> libvirt-nat-out apply ?

     Need to allow all traffic


Is the above right, or any I getting mixed up somewhere ?

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH 0/4] network: firewalld: fix routed network
Posted by Eric Garver 3 days, 16 hours ago
On Thu, May 12, 2022 at 08:04:28PM +0100, Daniel P. Berrangé wrote:
> On Thu, May 12, 2022 at 07:00:09PM +0100, Daniel P. Berrangé wrote:
> > On Wed, May 11, 2022 at 11:41:51AM -0400, Eric Garver wrote:
> > > This series fixes routed networks when a newer firewalld (>= 1.0.0) is
> > > present [1]. Firewalld 1.0.0 included a change that disallows implicit
> > > forwarding between zones [2]. libvirt was relying on this behavior to
> > > allow routed networks to function.
> > > 
> > > New firewalld policies are added. This is done to use common rules
> > > between NAT and routed networks. Policies have been supported since
> > > firewalld 0.9.0.
> > 
> > For those following along, there's a helpful description of policies
> > here, specifically explaining how its useful to the libvirt scenario:
> > 
> >   https://firewalld.org/2020/09/policy-objects-introduction
> 
> In reviewing these patches I've come to realize I'm still not
> confident I'm understanding the interaction between traffic
> we're managing at the firewalld  zones/policies.

It's confusing because it's a combination of iptables (libvirt) and
firewalld (nftables). And they filter independently. Think of it as
having to pass through two firewalls.

Hopefully I got it all correct below.

> For illustration let me assume the following setup:
> [
>    * Remote host on LAN (remote host IP 10.0.0.2)
> 
>    * eth0 public facing ethernet on the LAN (local host IP 10.0.0.5)
> 
>    * virbr0 isolated bridge device (local host IP 192.168.122.1)
> 
>    * vnet0 TAP device for a guest (guest IP 192.168.122.5)
> 
> 
>    Remote host            Local host
> 
>   +----------+    LAN   +----------+   IP forward  +---------------+
>   | 10.0.0.2 | -------- | 10.0.0.5 | --------------| 192.168.122.1 |
>   | eth0     |          |  eth0    |   w/ NAT      |  virbr0       |
>   +----------+          +----------+               +---------------+
>                                                          |
> 							 | bridge port
>                                                          |
>                                                    +---------------+
>                                                    | 192.168.122.5 |
> 						   | host: vnet0   |
> 						   | guest: eth0   |
> 						   +---------------+
> 
> IIUC zones are
> 
>    * 'libvirt' containing 'virbr0'
>    * 'FedoraWorkstation' containing 'eth0'
> 
> Is 'vnet0' in a zone or not ?

No. Only the bridge interface is added to the zone. The vnet* interfaces
don't have addresses.

> 
> 
> Traffic flows
> 
> 
>    * LAN Remote host (10.0.0.2) -> local host (10.0.0.5)
> 
>      Normal traffic nothing to do with libvirt
> 
>      Rules in <zone> FedoraWorkstation apply

True.

> 
>    * LAN Remote host (10.0.0.2) -> guest (192.168.122.5)
> 
>      IP layer forwarding via eth0 (with conntrack match for NAT zone)
> 
>      ingress=FedoraWorkstation
>      egress=libvirt
> 
>      Rules in <policy> libvirt-host-in apply ?

False. There are no explicit firewalld rules for this.

Existing connections would be implicitly allowed by a top-level "ct
state" match in FORWARD.

> 
>    * Local host (192.168.122.1) -> guest (192.168.122.5)
> 
>      Rules in <zone> libvirt apply ?

False. No rules explicit rules apply.

Firewalld allows outbound by default.

> 
>    * Local host (10.0.0.5) -> guest (192.168.122.5)
> 
>      NB, shouldn't happen as traffic should have originated
>      from 192.168.122.1 instead.
> 
>      ingress=FedoraWorkstation
>      egress=libvirt
> 
>      Rules in <policy> libvirt-host-in apply ?

False. There are no explicit firewalld rules for this.

New connections would be denied. Existing (originating from VM) would be
allowed.

> 
>    * Guest (192.168.122.5) -> Local host (192.168.122.1)
> 
>      Rules in <zone> libvirt apply ?
> 
>      Need to allow dhcp, dns, ssh. Feels like this
>      should still be rules in the <zone> ?

True. This is handled by the current zone definition.

This series moves them into libvirt-to-host. You used the name
libvirt-host-in, which may be a better name for the policy. :)

> 
>    * Guest (192.168.122.5) -> Local host (10.0.0.5)
> 
>      NB, shouldn't happen as guest generally won't be
>      aware of host's eth0 IP address.
> 
>      ingress=libvirt
>      egress=FedoraWorkstation
> 
>      Rules in <policy> libvirt-nat-out apply ?
> 
>      Should not allow anything special related to virt,
>      as dhcp/dns stuff should only be serviced from virbr0.
>      So the libvirt-nat-out policy feels wrong for this
>      case.

False. I think this is still considered INPUT traffic since it's going
to the local network stack.

So the "libvirt" zone and libvirt-to-host would apply.

Would be

        ingress=libvirt
        egress=HOST

> 
>    * Guest (192.168.122.5) -> LAN remote host (10.0.0.2)
> 
>      ingress=libvirt
>      egress=FedoraWorkstation
> 
>      Rules in <policy> libvirt-nat-out apply ?
> 
>      Need to allow all traffic

True.

> 
> Is the above right, or any I getting mixed up somewhere ?

Answered all inline.
Re: [PATCH 0/4] network: firewalld: fix routed network
Posted by Laine Stump 3 days, 18 hours ago
On 5/12/22 2:00 PM, Daniel P. Berrangé wrote:
> On Wed, May 11, 2022 at 11:41:51AM -0400, Eric Garver wrote:
>> This series fixes routed networks when a newer firewalld (>= 1.0.0) is
>> present [1]. Firewalld 1.0.0 included a change that disallows implicit
>> forwarding between zones [2]. libvirt was relying on this behavior to
>> allow routed networks to function.
>>
>> New firewalld policies are added. This is done to use common rules
>> between NAT and routed networks. Policies have been supported since
>> firewalld 0.9.0.
> 
> For those following along, there's a helpful description of policies
> here, specifically explaining how its useful to the libvirt scenario:
> 
>    https://firewalld.org/2020/09/policy-objects-introduction

...and for some further context that is probably only documented in the 
discussions that we had with Eric and some other people back in 2018 or so:

Once firewalld switches to its native-nftables backend, all of its own 
rules go into a separate nftables table, while libvirt's rules go into 
the iptables-compatibility table called "filter". In order for a packet 
to be accepted and forwarded, it must be accepted by *all* tables. (with 
iptables, both firewalld and libvirt use the "filter" table, and it is 
enough for the rules of one or the other to accept a packet).

At the time libvirt added support for the firewalld nftables backend, 
there was no way to explicitly specify "allow forwarded traffic" in a 
zone, and if the zone was "default REJECT" then all forwarded traffic 
would be rejected. In order for our traffic to be accepted, we had to 
make the "libvirt zone" (which is itself a part of *firewalld's* rules, 
not libvirt's rules!) "default ACCEPT", and then use an at-the-time new 
feature of firewalld that allowed us to specify higher priority ACCEPT 
rules for the traffic we wanted accepted, then a lower priority "REJECT 
ALL" rule (which would reject all traffic on the *INPUT* chain, but not 
on the FORWARD chain), and then the "default ACCEPT" rule would 
implicitly add rules that accepted any forwarded traffic.

Yes, in restrospect it sounds fragile. And at the time it sounded 
fragile as well. Unfortunately it was the only way to make things work.

In the ensuing years, firewalld has added explicit support for 
accepting/rejecting traffic on the FORWARD and OUTPUT chains, but as a 
part of this, that implicit "default ACCEPT" of forwarded traffic has 
been removed. And *that* is what necessitates Eric's new zone/policy 
files! Whew!