From nobody Tue Apr 23 07:58:55 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) client-ip=207.211.31.81; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1599824245; cv=none; d=zohomail.com; s=zohoarc; b=jmPtoRH2wCt0a6qunLRzPzMVif8zsCjCIDvuWi3gnfs2VB/TUCGDPOHHiOJEwphrzv3gnAlASAvmqtFBi3sXAaR/Pl8BCMjkVe64AAqcf0gWzOSJp3gdIZlExpn6bLnbjDgfmxBWaHkOgx6NmVbaL6sCtlyv6xrjuAWWLLhLlFg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1599824245; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:To; bh=4G+11vZWNUOzOMf7Nxhtf6LhFH41FiVfPy8Bd0D1RYM=; b=cxDm8mmSHf3FcKJWPSkwR6GvjOXauNkoFKrqTV629cJQHy83HktSv/lNkgaOyDhEAtHAodvzThfZzp3xEZtVYAEG7zf4L+UeTRzMdWa25TcX9SBNJBPqGfAtKNyUsFpQXF+vvOghZIy4bzLVrzQfm7Lz8Ca29lMoOl4YvQsC6i8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.81]) by mx.zohomail.com with SMTPS id 159982424554484.948574923852; Fri, 11 Sep 2020 04:37:25 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-335-QMpfEqqjPUWiUqPxiOwmag-1; Fri, 11 Sep 2020 07:37:20 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D03311006714; Fri, 11 Sep 2020 11:37:14 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 5925681C4F; Fri, 11 Sep 2020 11:37:14 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 68521181A06B; Fri, 11 Sep 2020 11:37:13 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 08BBYct0009506 for ; Fri, 11 Sep 2020 07:34:38 -0400 Received: by smtp.corp.redhat.com (Postfix) id E4FA71006E0A; Fri, 11 Sep 2020 11:34:37 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast05.extmail.prod.ext.rdu2.redhat.com [10.11.55.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E08611000DB0 for ; Fri, 11 Sep 2020 11:34:35 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 53415800BED for ; Fri, 11 Sep 2020 11:34:35 +0000 (UTC) Received: from mail-pg1-f200.google.com (mail-pg1-f200.google.com [209.85.215.200]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-41-UWSt3yFzMtWxJOfLVqQMpA-1; Fri, 11 Sep 2020 07:34:33 -0400 Received: by mail-pg1-f200.google.com with SMTP id r22so5865198pgk.14 for ; Fri, 11 Sep 2020 04:34:33 -0700 (PDT) Received: from fedora19.redhat.com (2001-44b8-4132-5a00-e6a4-71ff-fe56-570c.static.ipv6.internode.on.net. [2001:44b8:4132:5a00:e6a4:71ff:fe56:570c]) by smtp.gmail.com with ESMTPSA id h12sm2126113pfo.68.2020.09.11.04.34.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Sep 2020 04:34:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1599824242; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=4G+11vZWNUOzOMf7Nxhtf6LhFH41FiVfPy8Bd0D1RYM=; b=Zvf8xfpjCaKpAX7ybeST+4XbbGkoAOfFiqJREIUB1HH/fLwlZ+rOk1YgpThSMCslhbItb1 FTGCp26yQ4MmYAhDmxCs/v02EDdtgfwWEIW7bMESwJkpTm57p1jQo0M4LlMFCsUl9m59zL /FpshLqhffZ6vy/0YvjcJQ4pS+XLpwM= X-MC-Unique: QMpfEqqjPUWiUqPxiOwmag-1 X-MC-Unique: UWSt3yFzMtWxJOfLVqQMpA-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=4G+11vZWNUOzOMf7Nxhtf6LhFH41FiVfPy8Bd0D1RYM=; b=iqGp+6W4rvAtOhoVQLUGY2jwWRDHbbCoHHDWcTZMjJDp5JCZlbuz57cPIUugUl1VXq OLjqd5wRrPO23ybv2xswMW3F/8Oy5UOtlTZLYI3G/0IdJgQ7sSYaozcOY8RkLRctQBMS fS39oBJ3uehU8sFWVBPUBVC7txVREQael66FswJlgAcVeXFLCN7CJjmUyTlliFBxa3tk YPdzqVRUn5BmcYcu7JDhPEMI57CsewgNHI7Vp5TkaAr0UpugyGB5WE7zX+fPcCmFle39 GgBPLMWjMaUdOtonbYH/UPX1Zp3ULsOEaMlckNCfh6s6prriVzL2sAfFVVSguW6XHiXd RlFg== X-Gm-Message-State: AOAM530W87C6SQmNGl5zi+Rqq1yT8jjTxs/e37URBIvHIHrrmdvvOrX7 5E3KuCX8h4fmtHdIuHrXDyO+xXlKgkz/oc+jlq6NHSQdci1/yiqvPGSLgWb75M/qxyZXyeaQUkK mPcE/tv9U2DncPfFfVu8= X-Received: by 2002:a17:90b:1256:: with SMTP id gx22mr1928979pjb.47.1599824072151; Fri, 11 Sep 2020 04:34:32 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyEFvulanezI3RpGepz08MF9mrgxKKuasKrKbA4mDkeF/G72MHDbQaCABgu9bxmjr2vPsoY4A== X-Received: by 2002:a17:90b:1256:: with SMTP id gx22mr1928949pjb.47.1599824071733; Fri, 11 Sep 2020 04:34:31 -0700 (PDT) From: Ian Wienand To: libvir-list@redhat.com Subject: [PATCH v2] network: Only check kernel added routes in virNetDevIPCheckIPv6Forwarding Date: Fri, 11 Sep 2020 21:34:17 +1000 Message-Id: <20200911113417.915677-1-iwienand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-loop: libvir-list@redhat.com Cc: Cedric Bosdonnat , Ian Wienand X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0.003 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" The original motivation for adding virNetDevIPCheckIPv6Forwarding (00d28a78b5d1f6eaf79f06ac59e31c568af9da37) was that networking routes would disappear when ipv6 forwarding was enabled for an interface. This is a fairly undocumented side-effect of the "accept_ra" sysctl for an interface. 1 means the interface will accept_ra's if not forwarding, 2 means always accept_RAs; but it is not explained that enabling forwarding when accept_ra=3D=3D1 will also clear any kernel RA assigned routes, very likely breaking your networking. The check to warn about this currently uses netlink to go through all the routes and then look at the accept_ra status of the interfaces. However, it has been noticed that this problem does not affect systems where IPv6 RA configuration is handled in userspace, e.g. via tools such as NetworkManager. In this case, the error message from libvirt is spurious, and modifying the forwarding state will not affect the RA state or disable your networking. If you refer to the function rt6_purge_dflt_routers() in the kernel, we can see that the routes being purged are only those with the kernel's RTF_ADDRCONF flag set; that is, routes added by the kernel's RA handling. Why does it do this? I think this is a Linux implementation decision; it has always been like that and there are some comments suggesting that it is because a router should be statically configured, rather than accepting external configurations. The solution implemented here is to convert the existing check into a walk of /proc/net/ipv6_route and look for routes with this flag set. We then check the accept_ra status for the interface, and if enabling forwarding would break things raise an error. This should hopefully avoid "interactive" users, who are likely to be using NetworkManager and the like, having false warnings when enabling IPv6, but retain the error check for users relying on kernel-based IPv6 interface auto-configuration. Signed-off-by: Ian Wienand Reviewed-by: Laine Stump --- src/util/virnetdevip.c | 323 ++++++++++++++++------------------------- 1 file changed, 124 insertions(+), 199 deletions(-) diff --git a/src/util/virnetdevip.c b/src/util/virnetdevip.c index 7bd5a75f85..e208089bd6 100644 --- a/src/util/virnetdevip.c +++ b/src/util/virnetdevip.c @@ -43,8 +43,11 @@ #ifdef __linux__ # include # include +# include #endif =20 +#define PROC_NET_IPV6_ROUTE "/proc/net/ipv6_route" + #define VIR_FROM_THIS VIR_FROM_NONE =20 VIR_LOG_INIT("util.netdevip"); @@ -370,203 +373,6 @@ virNetDevIPRouteAdd(const char *ifname, } =20 =20 -static int -virNetDevIPGetAcceptRA(const char *ifname) -{ - g_autofree char *path =3D NULL; - g_autofree char *buf =3D NULL; - char *suffix; - int accept_ra =3D -1; - - path =3D g_strdup_printf("/proc/sys/net/ipv6/conf/%s/accept_ra", - ifname ? ifname : "all"); - - if ((virFileReadAll(path, 512, &buf) < 0) || - (virStrToLong_i(buf, &suffix, 10, &accept_ra) < 0)) - return -1; - - return accept_ra; -} - -struct virNetDevIPCheckIPv6ForwardingData { - bool hasRARoutes; - - /* Devices with conflicting accept_ra */ - char **devices; - size_t ndevices; -}; - - -static int -virNetDevIPCheckIPv6ForwardingAddIF(struct virNetDevIPCheckIPv6ForwardingD= ata *data, - char **ifname) -{ - size_t i; - - /* add ifname to the array if it's not already there - * (ifname is char** so VIR_APPEND_ELEMENT() will move the - * original pointer out of the way and avoid having it freed) - */ - for (i =3D 0; i < data->ndevices; i++) { - if (STREQ(data->devices[i], *ifname)) - return 0; - } - return VIR_APPEND_ELEMENT(data->devices, data->ndevices, *ifname); -} - - -static int -virNetDevIPCheckIPv6ForwardingCallback(struct nlmsghdr *resp, - void *opaque) -{ - struct rtmsg *rtmsg =3D NLMSG_DATA(resp); - struct virNetDevIPCheckIPv6ForwardingData *data =3D opaque; - struct rtattr *rta_attr; - int accept_ra =3D -1; - int ifindex =3D -1; - g_autofree char *ifname =3D NULL; - - /* Ignore messages other than route ones */ - if (resp->nlmsg_type !=3D RTM_NEWROUTE) - return 0; - - /* No need to do anything else for non RA routes */ - if (rtmsg->rtm_protocol !=3D RTPROT_RA) - return 0; - - rta_attr =3D (struct rtattr *)nlmsg_find_attr(resp, sizeof(struct rtms= g), RTA_OIF); - if (rta_attr) { - /* This is a single path route, with interface used to reach - * nexthop in the RTA_OIF attribute. - */ - ifindex =3D *(int *)RTA_DATA(rta_attr); - ifname =3D virNetDevGetName(ifindex); - - if (!ifname) - return -1; - - accept_ra =3D virNetDevIPGetAcceptRA(ifname); - - VIR_DEBUG("Checking route for device %s (%d), accept_ra: %d", - ifname, ifindex, accept_ra); - - if (accept_ra !=3D 2 && virNetDevIPCheckIPv6ForwardingAddIF(data, = &ifname) < 0) - return -1; - - data->hasRARoutes =3D true; - return 0; - } - - /* if no RTA_OIF was found, see if this is a multipath route (one - * which has an array of nexthops, each with its own interface) - */ - - rta_attr =3D (struct rtattr *)nlmsg_find_attr(resp, sizeof(struct rtms= g), RTA_MULTIPATH); - if (rta_attr) { - /* The data of the attribute is an array of rtnexthop */ - struct rtnexthop *nh =3D RTA_DATA(rta_attr); - size_t len =3D RTA_PAYLOAD(rta_attr); - - /* validate the attribute array length */ - len =3D MIN(len, ((char *)resp + NLMSG_PAYLOAD(resp, 0) - (char *)= rta_attr)); - - while (len >=3D sizeof(*nh) && len >=3D nh->rtnh_len) { - /* check accept_ra for the interface of each nexthop */ - - ifname =3D virNetDevGetName(nh->rtnh_ifindex); - - if (!ifname) - return -1; - - accept_ra =3D virNetDevIPGetAcceptRA(ifname); - - VIR_DEBUG("Checking multipath route nexthop device %s (%d), ac= cept_ra: %d", - ifname, nh->rtnh_ifindex, accept_ra); - - if (accept_ra !=3D 2 && virNetDevIPCheckIPv6ForwardingAddIF(da= ta, &ifname) < 0) - return -1; - - VIR_FREE(ifname); - data->hasRARoutes =3D true; - - len -=3D NLMSG_ALIGN(nh->rtnh_len); - VIR_WARNINGS_NO_CAST_ALIGN - nh =3D RTNH_NEXT(nh); - VIR_WARNINGS_RESET - } - } - - return 0; -} - -bool -virNetDevIPCheckIPv6Forwarding(void) -{ - bool valid =3D false; - struct rtgenmsg genmsg; - size_t i; - struct virNetDevIPCheckIPv6ForwardingData data =3D { - .hasRARoutes =3D false, - .devices =3D NULL, - .ndevices =3D 0 - }; - g_autoptr(virNetlinkMsg) nlmsg =3D NULL; - - - /* Prepare the request message */ - if (!(nlmsg =3D nlmsg_alloc_simple(RTM_GETROUTE, - NLM_F_REQUEST | NLM_F_DUMP))) { - virReportOOMError(); - goto cleanup; - } - - memset(&genmsg, 0, sizeof(genmsg)); - genmsg.rtgen_family =3D AF_INET6; - - if (nlmsg_append(nlmsg, &genmsg, sizeof(genmsg), NLMSG_ALIGNTO) < 0) { - virReportError(VIR_ERR_INTERNAL_ERROR, "%s", - _("allocated netlink buffer is too small")); - goto cleanup; - } - - /* Send the request and loop over the responses */ - if (virNetlinkDumpCommand(nlmsg, virNetDevIPCheckIPv6ForwardingCallbac= k, - 0, 0, NETLINK_ROUTE, 0, &data) < 0) { - virReportError(VIR_ERR_INTERNAL_ERROR, "%s", - _("Failed to loop over IPv6 routes")); - goto cleanup; - } - - valid =3D !data.hasRARoutes || data.ndevices =3D=3D 0; - - /* Check the global accept_ra if at least one isn't set on a - per-device basis */ - if (!valid && data.hasRARoutes) { - int accept_ra =3D virNetDevIPGetAcceptRA(NULL); - valid =3D accept_ra =3D=3D 2; - VIR_DEBUG("Checked global accept_ra: %d", accept_ra); - } - - if (!valid) { - g_auto(virBuffer) buf =3D VIR_BUFFER_INITIALIZER; - for (i =3D 0; i < data.ndevices; i++) { - virBufferAdd(&buf, data.devices[i], -1); - if (i < data.ndevices - 1) - virBufferAddLit(&buf, ", "); - } - - virReportError(VIR_ERR_INTERNAL_ERROR, - _("Check the host setup: enabling IPv6 forwarding w= ith " - "RA routes without accept_ra set to 2 is likely t= o cause " - "routes loss. Interfaces to look at: %s"), - virBufferCurrentContent(&buf)); - } - - cleanup: - virStringListFreeCount(data.devices, data.ndevices); - return valid; -} - #else /* defined(__linux__) && defined(WITH_LIBNL) */ =20 =20 @@ -688,15 +494,134 @@ virNetDevIPRouteAdd(const char *ifname, return 0; } =20 +#endif /* defined(__linux__) && defined(HAVE_LIBNL) */ + + +#if defined(__linux__) + +static int +virNetDevIPGetAcceptRA(const char *ifname) +{ + g_autofree char *path =3D NULL; + g_autofree char *buf =3D NULL; + char *suffix; + int accept_ra =3D -1; + + path =3D g_strdup_printf("/proc/sys/net/ipv6/conf/%s/accept_ra", + ifname ? ifname : "all"); + + if ((virFileReadAll(path, 512, &buf) < 0) || + (virStrToLong_i(buf, &suffix, 10, &accept_ra) < 0)) + return -1; + + return accept_ra; +} + +/** + * virNetDevIPCheckIPv6Forwarding + * + * This function checks if IPv6 routes have the RTF_ADDRCONF flag set, + * indicating they have been created by the kernel's RA configuration + * handling. These routes are subject to being flushed when ipv6 + * forwarding is enabled unless accept_ra is explicitly set to "2". + * This will most likely result in ipv6 networking being broken. + * + * Returns: true if it is safe to enable forwarding, or false if + * breakable routes are found. + * + **/ +bool +virNetDevIPCheckIPv6Forwarding(void) +{ + int len; + char *cur; + g_autofree char *buf =3D NULL; + /* lines are 150 chars */ + enum {MAX_ROUTE_SIZE =3D 150*100000}; + + /* This is /proc/sys/net/ipv6/conf/all/accept_ra */ + int all_accept_ra =3D virNetDevIPGetAcceptRA(NULL); + + /* Read ipv6 routes */ + if ((len =3D virFileReadAll(PROC_NET_IPV6_ROUTE, + MAX_ROUTE_SIZE, &buf)) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Unable to read %s " + "for ipv6 forwarding checks"), PROC_NET_IPV6_ROUT= E); + return false; + } + + /* VIR_DEBUG("%s output:\n%s", PROC_NET_IPV6_ROUTE, buf); */ + + /* Dropping the last character to stop the loop */ + if (len > 0) + buf[len-1] =3D '\0'; + + cur =3D buf; + while (cur) { + char route[33], flags[9], iface[9]; + unsigned int flags_val; + char *iface_val; + int num; + char *nl =3D strchr(cur, '\n'); + + if (nl) + *nl++ =3D '\0'; + + num =3D sscanf(cur, "%32s %*s %*s %*s %*s %*s %*s %*s %8s %8s", + route, flags, iface); + + cur =3D nl; + if (num !=3D 3) { + VIR_DEBUG("Failed to parse route line: %s", cur); + continue; + } + + if (virStrToLong_ui(flags, NULL, 16, &flags_val)) { + VIR_DEBUG("Failed to parse flags: %s", flags); + continue; + } + + /* This is right justified, strip leading spaces */ + iface_val =3D &iface[0]; + while (*iface_val && g_ascii_isspace(*iface_val)) + iface_val++; + + VIR_DEBUG("%s iface %s flags %s : RTF_ADDRCONF %sset", + route, iface_val, flags, + (flags_val & RTF_ADDRCONF ? "" : "not ")); + + if (flags_val & RTF_ADDRCONF) { + int ret =3D virNetDevIPGetAcceptRA(iface_val); + VIR_DEBUG("%s reports accept_ra of %d", + iface_val, ret); + /* If the interface for this autoconfigured route + * has accept_ra =3D=3D 1, or it is default and the "all" + * value of accept_ra =3D=3D 1, it will be subject to + * flushing if forwarding is enabled. + */ + if (ret =3D=3D 1 || (ret =3D=3D 0 && all_accept_ra =3D=3D 1)) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Check the host setup: interface %s has k= ernel " + "autoconfigured IPv6 routes and enabling = forwarding " + " without accept_ra set to 2 will cause t= he kernel " + "to flush them, breaking networking."), i= face_val); + return false; + } + } + } + return true; +} +#else =20 bool virNetDevIPCheckIPv6Forwarding(void) { - VIR_WARN("built without libnl: unable to check if IPv6 forwarding can = be safely enabled"); + VIR_DEBUG("No checks for IPv6 forwarding issues on non-Linux systems",= flags); return true; } =20 -#endif /* defined(__linux__) && defined(WITH_LIBNL) */ +#endif /* defined(__linux__) */ =20 =20 /** --=20 2.26.2