src/network/bridge_driver_linux.c | 6 +++++- src/network/libvirt-nat-out.policy | 12 ++++++++++++ src/network/libvirt-routed-in.policy | 11 +++++++++++ src/network/libvirt-routed-out.policy | 12 ++++++++++++ src/network/libvirt-routed.zone | 12 ++++++++++++ src/network/libvirt-to-host.policy | 21 +++++++++++++++++++++ src/network/libvirt.zone | 23 +++++------------------ src/network/meson.build | 25 +++++++++++++++++++++++++ 8 files changed, 103 insertions(+), 19 deletions(-) create mode 100644 src/network/libvirt-nat-out.policy create mode 100644 src/network/libvirt-routed-in.policy create mode 100644 src/network/libvirt-routed-out.policy create mode 100644 src/network/libvirt-routed.zone create mode 100644 src/network/libvirt-to-host.policy
This series fixes routed networks when a newer firewalld (>= 1.0.0) is present [1]. Firewalld 1.0.0 included a change that disallows implicit forwarding between zones [2]. libvirt was relying on this behavior to allow routed networks to function. New firewalld policies are added. This is done to use common rules between NAT and routed networks. Policies have been supported since firewalld 0.9.0. [1]: https://bugzilla.redhat.com/show_bug.cgi?id=2055706 [2]: https://github.com/firewalld/firewalld/issues/177 Eric Garver (4): network: firewalld: convert to policies network: firewalld: add zone for routed networks network: firewalld: add policies for routed networks network: firewalld: add support for routed networks src/network/bridge_driver_linux.c | 6 +++++- src/network/libvirt-nat-out.policy | 12 ++++++++++++ src/network/libvirt-routed-in.policy | 11 +++++++++++ src/network/libvirt-routed-out.policy | 12 ++++++++++++ src/network/libvirt-routed.zone | 12 ++++++++++++ src/network/libvirt-to-host.policy | 21 +++++++++++++++++++++ src/network/libvirt.zone | 23 +++++------------------ src/network/meson.build | 25 +++++++++++++++++++++++++ 8 files changed, 103 insertions(+), 19 deletions(-) create mode 100644 src/network/libvirt-nat-out.policy create mode 100644 src/network/libvirt-routed-in.policy create mode 100644 src/network/libvirt-routed-out.policy create mode 100644 src/network/libvirt-routed.zone create mode 100644 src/network/libvirt-to-host.policy -- 2.33.0
On Wed, May 11, 2022 at 11:41:51AM -0400, Eric Garver wrote: > This series fixes routed networks when a newer firewalld (>= 1.0.0) is > present [1]. Firewalld 1.0.0 included a change that disallows implicit > forwarding between zones [2]. libvirt was relying on this behavior to > allow routed networks to function. > > New firewalld policies are added. This is done to use common rules > between NAT and routed networks. Policies have been supported since > firewalld 0.9.0. For those following along, there's a helpful description of policies here, specifically explaining how its useful to the libvirt scenario: https://firewalld.org/2020/09/policy-objects-introduction With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Thu, May 12, 2022 at 07:00:09PM +0100, Daniel P. Berrangé wrote:
> On Wed, May 11, 2022 at 11:41:51AM -0400, Eric Garver wrote:
> > This series fixes routed networks when a newer firewalld (>= 1.0.0) is
> > present [1]. Firewalld 1.0.0 included a change that disallows implicit
> > forwarding between zones [2]. libvirt was relying on this behavior to
> > allow routed networks to function.
> >
> > New firewalld policies are added. This is done to use common rules
> > between NAT and routed networks. Policies have been supported since
> > firewalld 0.9.0.
>
> For those following along, there's a helpful description of policies
> here, specifically explaining how its useful to the libvirt scenario:
>
> https://firewalld.org/2020/09/policy-objects-introduction
In reviewing these patches I've come to realize I'm still not
confident I'm understanding the interaction between traffic
we're managing at the firewalld zones/policies.
For illustration let me assume the following setup:
[
* Remote host on LAN (remote host IP 10.0.0.2)
* eth0 public facing ethernet on the LAN (local host IP 10.0.0.5)
* virbr0 isolated bridge device (local host IP 192.168.122.1)
* vnet0 TAP device for a guest (guest IP 192.168.122.5)
Remote host Local host
+----------+ LAN +----------+ IP forward +---------------+
| 10.0.0.2 | -------- | 10.0.0.5 | --------------| 192.168.122.1 |
| eth0 | | eth0 | w/ NAT | virbr0 |
+----------+ +----------+ +---------------+
|
| bridge port
|
+---------------+
| 192.168.122.5 |
| host: vnet0 |
| guest: eth0 |
+---------------+
IIUC zones are
* 'libvirt' containing 'virbr0'
* 'FedoraWorkstation' containing 'eth0'
Is 'vnet0' in a zone or not ?
Traffic flows
* LAN Remote host (10.0.0.2) -> local host (10.0.0.5)
Normal traffic nothing to do with libvirt
Rules in <zone> FedoraWorkstation apply
* LAN Remote host (10.0.0.2) -> guest (192.168.122.5)
IP layer forwarding via eth0 (with conntrack match for NAT zone)
ingress=FedoraWorkstation
egress=libvirt
Rules in <policy> libvirt-host-in apply ?
* Local host (192.168.122.1) -> guest (192.168.122.5)
Rules in <zone> libvirt apply ?
* Local host (10.0.0.5) -> guest (192.168.122.5)
NB, shouldn't happen as traffic should have originated
from 192.168.122.1 instead.
ingress=FedoraWorkstation
egress=libvirt
Rules in <policy> libvirt-host-in apply ?
* Guest (192.168.122.5) -> Local host (192.168.122.1)
Rules in <zone> libvirt apply ?
Need to allow dhcp, dns, ssh. Feels like this
should still be rules in the <zone> ?
* Guest (192.168.122.5) -> Local host (10.0.0.5)
NB, shouldn't happen as guest generally won't be
aware of host's eth0 IP address.
ingress=libvirt
egress=FedoraWorkstation
Rules in <policy> libvirt-nat-out apply ?
Should not allow anything special related to virt,
as dhcp/dns stuff should only be serviced from virbr0.
So the libvirt-nat-out policy feels wrong for this
case.
* Guest (192.168.122.5) -> LAN remote host (10.0.0.2)
ingress=libvirt
egress=FedoraWorkstation
Rules in <policy> libvirt-nat-out apply ?
Need to allow all traffic
Is the above right, or any I getting mixed up somewhere ?
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Thu, May 12, 2022 at 08:04:28PM +0100, Daniel P. Berrangé wrote:
> On Thu, May 12, 2022 at 07:00:09PM +0100, Daniel P. Berrangé wrote:
> > On Wed, May 11, 2022 at 11:41:51AM -0400, Eric Garver wrote:
> > > This series fixes routed networks when a newer firewalld (>= 1.0.0) is
> > > present [1]. Firewalld 1.0.0 included a change that disallows implicit
> > > forwarding between zones [2]. libvirt was relying on this behavior to
> > > allow routed networks to function.
> > >
> > > New firewalld policies are added. This is done to use common rules
> > > between NAT and routed networks. Policies have been supported since
> > > firewalld 0.9.0.
> >
> > For those following along, there's a helpful description of policies
> > here, specifically explaining how its useful to the libvirt scenario:
> >
> > https://firewalld.org/2020/09/policy-objects-introduction
>
> In reviewing these patches I've come to realize I'm still not
> confident I'm understanding the interaction between traffic
> we're managing at the firewalld zones/policies.
It's confusing because it's a combination of iptables (libvirt) and
firewalld (nftables). And they filter independently. Think of it as
having to pass through two firewalls.
Hopefully I got it all correct below.
> For illustration let me assume the following setup:
> [
> * Remote host on LAN (remote host IP 10.0.0.2)
>
> * eth0 public facing ethernet on the LAN (local host IP 10.0.0.5)
>
> * virbr0 isolated bridge device (local host IP 192.168.122.1)
>
> * vnet0 TAP device for a guest (guest IP 192.168.122.5)
>
>
> Remote host Local host
>
> +----------+ LAN +----------+ IP forward +---------------+
> | 10.0.0.2 | -------- | 10.0.0.5 | --------------| 192.168.122.1 |
> | eth0 | | eth0 | w/ NAT | virbr0 |
> +----------+ +----------+ +---------------+
> |
> | bridge port
> |
> +---------------+
> | 192.168.122.5 |
> | host: vnet0 |
> | guest: eth0 |
> +---------------+
>
> IIUC zones are
>
> * 'libvirt' containing 'virbr0'
> * 'FedoraWorkstation' containing 'eth0'
>
> Is 'vnet0' in a zone or not ?
No. Only the bridge interface is added to the zone. The vnet* interfaces
don't have addresses.
>
>
> Traffic flows
>
>
> * LAN Remote host (10.0.0.2) -> local host (10.0.0.5)
>
> Normal traffic nothing to do with libvirt
>
> Rules in <zone> FedoraWorkstation apply
True.
>
> * LAN Remote host (10.0.0.2) -> guest (192.168.122.5)
>
> IP layer forwarding via eth0 (with conntrack match for NAT zone)
>
> ingress=FedoraWorkstation
> egress=libvirt
>
> Rules in <policy> libvirt-host-in apply ?
False. There are no explicit firewalld rules for this.
Existing connections would be implicitly allowed by a top-level "ct
state" match in FORWARD.
>
> * Local host (192.168.122.1) -> guest (192.168.122.5)
>
> Rules in <zone> libvirt apply ?
False. No rules explicit rules apply.
Firewalld allows outbound by default.
>
> * Local host (10.0.0.5) -> guest (192.168.122.5)
>
> NB, shouldn't happen as traffic should have originated
> from 192.168.122.1 instead.
>
> ingress=FedoraWorkstation
> egress=libvirt
>
> Rules in <policy> libvirt-host-in apply ?
False. There are no explicit firewalld rules for this.
New connections would be denied. Existing (originating from VM) would be
allowed.
>
> * Guest (192.168.122.5) -> Local host (192.168.122.1)
>
> Rules in <zone> libvirt apply ?
>
> Need to allow dhcp, dns, ssh. Feels like this
> should still be rules in the <zone> ?
True. This is handled by the current zone definition.
This series moves them into libvirt-to-host. You used the name
libvirt-host-in, which may be a better name for the policy. :)
>
> * Guest (192.168.122.5) -> Local host (10.0.0.5)
>
> NB, shouldn't happen as guest generally won't be
> aware of host's eth0 IP address.
>
> ingress=libvirt
> egress=FedoraWorkstation
>
> Rules in <policy> libvirt-nat-out apply ?
>
> Should not allow anything special related to virt,
> as dhcp/dns stuff should only be serviced from virbr0.
> So the libvirt-nat-out policy feels wrong for this
> case.
False. I think this is still considered INPUT traffic since it's going
to the local network stack.
So the "libvirt" zone and libvirt-to-host would apply.
Would be
ingress=libvirt
egress=HOST
>
> * Guest (192.168.122.5) -> LAN remote host (10.0.0.2)
>
> ingress=libvirt
> egress=FedoraWorkstation
>
> Rules in <policy> libvirt-nat-out apply ?
>
> Need to allow all traffic
True.
>
> Is the above right, or any I getting mixed up somewhere ?
Answered all inline.
On 5/12/22 2:00 PM, Daniel P. Berrangé wrote: > On Wed, May 11, 2022 at 11:41:51AM -0400, Eric Garver wrote: >> This series fixes routed networks when a newer firewalld (>= 1.0.0) is >> present [1]. Firewalld 1.0.0 included a change that disallows implicit >> forwarding between zones [2]. libvirt was relying on this behavior to >> allow routed networks to function. >> >> New firewalld policies are added. This is done to use common rules >> between NAT and routed networks. Policies have been supported since >> firewalld 0.9.0. > > For those following along, there's a helpful description of policies > here, specifically explaining how its useful to the libvirt scenario: > > https://firewalld.org/2020/09/policy-objects-introduction ...and for some further context that is probably only documented in the discussions that we had with Eric and some other people back in 2018 or so: Once firewalld switches to its native-nftables backend, all of its own rules go into a separate nftables table, while libvirt's rules go into the iptables-compatibility table called "filter". In order for a packet to be accepted and forwarded, it must be accepted by *all* tables. (with iptables, both firewalld and libvirt use the "filter" table, and it is enough for the rules of one or the other to accept a packet). At the time libvirt added support for the firewalld nftables backend, there was no way to explicitly specify "allow forwarded traffic" in a zone, and if the zone was "default REJECT" then all forwarded traffic would be rejected. In order for our traffic to be accepted, we had to make the "libvirt zone" (which is itself a part of *firewalld's* rules, not libvirt's rules!) "default ACCEPT", and then use an at-the-time new feature of firewalld that allowed us to specify higher priority ACCEPT rules for the traffic we wanted accepted, then a lower priority "REJECT ALL" rule (which would reject all traffic on the *INPUT* chain, but not on the FORWARD chain), and then the "default ACCEPT" rule would implicitly add rules that accepted any forwarded traffic. Yes, in restrospect it sounds fragile. And at the time it sounded fragile as well. Unfortunately it was the only way to make things work. In the ensuing years, firewalld has added explicit support for accepting/rejecting traffic on the FORWARD and OUTPUT chains, but as a part of this, that implicit "default ACCEPT" of forwarded traffic has been removed. And *that* is what necessitates Eric's new zone/policy files! Whew!
© 2016 - 2024 Red Hat, Inc.