From nobody Mon Feb 9 19:53:19 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) client-ip=216.205.24.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=fail(p=none dis=none) header.from=126.com Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by mx.zohomail.com with SMTPS id 1625128980895631.3370581458572; Thu, 1 Jul 2021 01:43:00 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-149-Iu6NJQvcP9-dU4YDj5p2hQ-1; Thu, 01 Jul 2021 04:42:58 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 83F84343CE; Thu, 1 Jul 2021 08:42:52 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 595325C25D; Thu, 1 Jul 2021 08:42:52 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 95E581809C9A; Thu, 1 Jul 2021 08:42:50 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 1618gnJa006266 for ; Thu, 1 Jul 2021 04:42:49 -0400 Received: by smtp.corp.redhat.com (Postfix) id B5589FED2A; Thu, 1 Jul 2021 08:42:49 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast06.extmail.prod.ext.rdu2.redhat.com [10.11.55.22]) by smtp.corp.redhat.com (Postfix) with ESMTPS id AFF2E1032BB for ; Thu, 1 Jul 2021 08:42:47 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5FA9818A01A0 for ; Thu, 1 Jul 2021 08:42:47 +0000 (UTC) Received: from m15112.mail.126.com (m15112.mail.126.com [220.181.15.112]) by relay.mimecast.com with ESMTP id us-mta-458-zxE1x5IfPRe5TAiqs0sHTQ-1; Thu, 01 Jul 2021 04:42:43 -0400 Received: from zhangjl0200.home.langchao.com (unknown [58.56.96.28]) by smtp2 (Coremail) with SMTP id DMmowAB3LRr6f91ggnb6Cg--.12121S4; Thu, 01 Jul 2021 16:42:38 +0800 (CST) X-MC-Unique: Iu6NJQvcP9-dU4YDj5p2hQ-1 X-MC-Unique: zxE1x5IfPRe5TAiqs0sHTQ-1 From: zhangjl02 To: libvir-list@redhat.com Subject: [PATCHv2 2/4] virDomain: interface: add virNetDevOpenvswitchInterfaceSetQos and virNetDevOpenvswitchInterfaceClearQos Date: Thu, 1 Jul 2021 16:42:04 +0800 Message-Id: <20210701084206.621-3-jx8zjs@126.com> In-Reply-To: <20210701084206.621-1-jx8zjs@126.com> References: <20210701084206.621-1-jx8zjs@126.com> MIME-Version: 1.0 X-CM-TRANSID: DMmowAB3LRr6f91ggnb6Cg--.12121S4 X-Coremail-Antispam: 1Uf129KBjvAXoW3Zr1UGrWDZw4xtw43AF1UWrg_yoW8JFy7Ko W3Gr47Kr1kG34Uu3Z0ya17G3yUAr1rKrsxur1Y9a98Jry8GrZYv34kAan8AFsaqFW8Ar17 WFySqrW3GFsrCryfn29KB7ZKAUJUUUUU529EdanIXcx71UUUUU7v73VFW2AGmfu7bjvjm3 AaLaJ3UbIYCTnIWIevJa73UjIFyTuYvjxUSUDGUUUUU X-Originating-IP: [58.56.96.28] X-CM-SenderInfo: dm0y6ybv6rjloofrz/1tbi2R7CSFpEB2ML7wABsS X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-loop: libvir-list@redhat.com Cc: shenjiatong@inspur.com, zhangjl02@inspur.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: zhangjl02 Introduce qos setting and cleaning method. Use ovs command to set qos parameters on specific interface of qemu virtual machine. When an ovs port is created, we add 'ifname' to external-ids. When setting qos on an ovs port, query its qos and queue. If found, change qos on queried queue and qos, otherwise create new queue and qos. When cleaning qos, query and clean queues and qos in ovs table record by 'ifname' and 'vmid'. --- src/libvirt_private.syms | 2 + src/util/virnetdevopenvswitch.c | 271 ++++++++++++++++++++++++++++++++ src/util/virnetdevopenvswitch.h | 11 ++ 3 files changed, 284 insertions(+) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 2bcbce7288..e60bfa62c5 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -2806,8 +2806,10 @@ virNetDevMidonetUnbindPort; virNetDevOpenvswitchAddPort; virNetDevOpenvswitchGetMigrateData; virNetDevOpenvswitchGetVhostuserIfname; +virNetDevOpenvswitchInterfaceClearQos; virNetDevOpenvswitchInterfaceGetMaster; virNetDevOpenvswitchInterfaceParseStats; +virNetDevOpenvswitchInterfaceSetQos; virNetDevOpenvswitchInterfaceStats; virNetDevOpenvswitchMaybeUnescapeReply; virNetDevOpenvswitchRemovePort; diff --git a/src/util/virnetdevopenvswitch.c b/src/util/virnetdevopenvswitc= h.c index eac68d9556..83acc869cd 100644 --- a/src/util/virnetdevopenvswitch.c +++ b/src/util/virnetdevopenvswitch.c @@ -30,6 +30,7 @@ #include "virlog.h" #include "virjson.h" #include "virfile.h" +#include "virutil.h" =20 #define VIR_FROM_THIS VIR_FROM_NONE =20 @@ -140,6 +141,7 @@ int virNetDevOpenvswitchAddPort(const char *brname, con= st char *ifname, g_autofree char *ifaceid_ex_id =3D NULL; g_autofree char *profile_ex_id =3D NULL; g_autofree char *vmid_ex_id =3D NULL; + g_autofree char *ifname_ex_id =3D NULL; =20 virMacAddrFormat(macaddr, macaddrstr); virUUIDFormat(ovsport->interfaceID, ifuuidstr); @@ -149,6 +151,7 @@ int virNetDevOpenvswitchAddPort(const char *brname, con= st char *ifname, macaddrstr); ifaceid_ex_id =3D g_strdup_printf("external-ids:iface-id=3D\"%s\"", if= uuidstr); vmid_ex_id =3D g_strdup_printf("external-ids:vm-id=3D\"%s\"", vmuuidst= r); + ifname_ex_id =3D g_strdup_printf("external-ids:ifname=3D\"%s\"", ifnam= e); if (ovsport->profileID[0] !=3D '\0') { profile_ex_id =3D g_strdup_printf("external-ids:port-profile=3D\"%= s\"", ovsport->profileID); @@ -174,6 +177,7 @@ int virNetDevOpenvswitchAddPort(const char *brname, con= st char *ifname, "--", "set", "Interface", ifname, ifaceid_ex_= id, "--", "set", "Interface", ifname, vmid_ex_id, "--", "set", "Interface", ifname, profile_ex_= id, + "--", "set", "Interface", ifname, ifname_ex_i= d, "--", "set", "Interface", ifname, "external-ids:iface-status=3Dactive", NULL); @@ -614,3 +618,270 @@ int virNetDevOpenvswitchUpdateVlan(const char *ifname, =20 return 0; } + + +/** + * virNetDevOpenvswitchInterfaceSetQos: + * @ifname: on which interface + * @bandwidth: rates to set (may be NULL) + * @swapped: true if IN/OUT should be set contrariwise + * + * Update qos configuration of an OVS port. + * + * If @swapped is set, the IN part of @bandwidth is set on + * @ifname's TX, and vice versa. If it is not set, IN is set on + * RX and OUT on TX. This is because for some types of interfaces + * domain and the host live on the same side of the interface (so + * domain's RX/TX is host's RX/TX), and for some it's swapped + * (domain's RX/TX is hosts's TX/RX). + * + * Return 0 on success, -1 otherwise. + */ +int virNetDevOpenvswitchInterfaceSetQos(const char *ifname, + const virNetDevBandwidth *bandwidt= h, + const unsigned char *vmid, + bool swapped) +{ + char vmuuidstr[VIR_UUID_STRING_BUFLEN]; + virNetDevBandwidthRate *rx =3D NULL; /* From domain POV */ + virNetDevBandwidthRate *tx =3D NULL; /* From domain POV */ + g_autoptr(virCommand) cmd =3D NULL; + g_autofree char *vmid_ex_id =3D NULL; + g_autofree char *ifname_ex_id =3D NULL; + g_autofree char *average =3D NULL; + g_autofree char *peak =3D NULL; + g_autofree char *burst =3D NULL; + g_autofree char *qos_uuid =3D NULL; + g_autofree char *queue_uuid =3D NULL; + g_autofree char **lines =3D NULL; + + if (!bandwidth) { + /* nothing to be enabled */ + return 0; + } + + if (geteuid() !=3D 0) { + virReportError(VIR_ERR_OPERATION_UNSUPPORTED, "%s", + _("Network bandwidth tuning is not available" + " in session mode")); + return -1; + } + + if (!ifname) { + virReportError(VIR_ERR_OPERATION_UNSUPPORTED, "%s", + _("Unable to set bandwidth for interface because " + "device name is unknown")); + return -1; + } + + if (swapped) { + rx =3D bandwidth->out; + tx =3D bandwidth->in; + } else { + rx =3D bandwidth->in; + tx =3D bandwidth->out; + } + if(!bandwidth->out && !bandwidth->in) { + if(virNetDevOpenvswitchInterfaceClearQos(ifname, vmid) < 0){ + VIR_WARN("Clean qos for interface %s failed", ifname); + } + } + if (tx && tx->average) { + average =3D g_strdup_printf("%llu", tx->average * 8192); + if (tx->burst) + burst =3D g_strdup_printf("%llu", tx->burst * 8192); + if (tx->peak) + peak =3D g_strdup_printf("%llu", tx->peak * 8192); + + /* find queue */ + cmd =3D virNetDevOpenvswitchCreateCmd(); + virUUIDFormat(vmid, vmuuidstr); + vmid_ex_id =3D g_strdup_printf("external-ids:vm-id=3D\"%s\"", vmuu= idstr); + ifname_ex_id =3D g_strdup_printf("external-ids:ifname=3D\"%s\"", i= fname); + virCommandAddArgList(cmd, "--no-heading", "--columns=3D_uuid", "fi= nd", "queue", + vmid_ex_id, ifname_ex_id,NULL); + virCommandSetOutputBuffer(cmd, &queue_uuid); + if (virCommandRun(cmd, NULL) < 0) { + VIR_WARN("Unable to find queue on port %s", ifname); + } + + /* find qos */ + virCommandFree(cmd); + cmd =3D virNetDevOpenvswitchCreateCmd(); + virCommandAddArgList(cmd, "--no-heading", "--columns=3D_uuid", "fi= nd", "qos", + vmid_ex_id, ifname_ex_id,NULL); + virCommandSetOutputBuffer(cmd, &qos_uuid); + if (virCommandRun(cmd, NULL) < 0) { + VIR_WARN("Unable to find qos on port %s", ifname); + } + + /* create qos and set */ + virCommandFree(cmd); + cmd =3D virNetDevOpenvswitchCreateCmd(); + if (queue_uuid && *queue_uuid) { + lines =3D g_strsplit(queue_uuid, "\n", 0); + virCommandAddArgList(cmd, "set", "queue", lines[0], NULL); + }else{ + virCommandAddArgList(cmd, "set", "port", ifname, "qos=3D@qos1", + vmid_ex_id, ifname_ex_id, + "--", "--id=3D@qos1", "create", "qos", "t= ype=3Dlinux-htb", NULL); + virCommandAddArgFormat(cmd, "other_config:min-rate=3D%s", aver= age); + if(burst){ + virCommandAddArgFormat(cmd, "other_config:burst=3D%s", bur= st); + } + if(peak){ + virCommandAddArgFormat(cmd, "other_config:max-rate=3D%s", = peak); + } + virCommandAddArgList(cmd, "queues:0=3D@queue0", vmid_ex_id, if= name_ex_id, + "--", "--id=3D@queue0", "create", "queue"= , NULL); + } + virCommandAddArgFormat(cmd, "other_config:min-rate=3D%s", average); + if(burst){ + virCommandAddArgFormat(cmd, "other_config:burst=3D%s", burst); + } + if(peak){ + virCommandAddArgFormat(cmd, "other_config:max-rate=3D%s", peak= ); + } + virCommandAddArgList(cmd, vmid_ex_id, ifname_ex_id, NULL); + if (virCommandRun(cmd, NULL) < 0) { + if(*queue_uuid){ + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Unable to set queue configuration on por= t %s"), ifname); + return -1; + } + else { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Unable to create and set qos configurati= on on port %s"), ifname); + return -1; + } + } + + if(qos_uuid && *qos_uuid){ + virCommandFree(cmd); + cmd =3D virNetDevOpenvswitchCreateCmd(); + lines =3D g_strsplit(qos_uuid, "\n", 0); + virCommandAddArgList(cmd, "set", "qos", lines[0], NULL); + virCommandAddArgFormat(cmd, "other_config:min-rate=3D%s", aver= age); + if(burst){ + virCommandAddArgFormat(cmd, "other_config:burst=3D%s", bur= st); + } + if(peak){ + virCommandAddArgFormat(cmd, "other_config:max-rate=3D%s", = peak); + } + virCommandAddArgList(cmd, vmid_ex_id, ifname_ex_id, NULL); + if (virCommandRun(cmd, NULL) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Unable to set qos configuration on port = %s"), ifname); + return -1; + } + } + } + + if (rx) { + average =3D g_strdup_printf("%llu", rx->average * 8); + if (rx->burst) + burst =3D g_strdup_printf("%llu", rx->burst * 8); + virCommandFree(cmd); + cmd =3D virNetDevOpenvswitchCreateCmd(); + virCommandAddArgList(cmd, "set", "Interface", ifname, NULL); + virCommandAddArgFormat(cmd, "ingress_policing_rate=3D%s", average); + if (burst) + virCommandAddArgFormat(cmd, "ingress_policing_burst=3D%s", bur= st); + + if (virCommandRun(cmd, NULL) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Unable to set vlan configuration on port %s"= ), ifname); + return -1; + } + virCommandFree(cmd); + } + + return 0; +} + +int virNetDevOpenvswitchInterfaceClearQos(const char *ifname, + const unsigned char *vmid){ + char vmuuidstr[VIR_UUID_STRING_BUFLEN]; + g_autoptr(virCommand) cmd =3D NULL; + g_autofree char *vmid_ex_id =3D NULL; + g_autofree char *qos_uuid =3D NULL; + g_autofree char *queue_uuid =3D NULL; + g_autofree char *port_qos =3D NULL; + g_autofree char **lines =3D NULL; + size_t i; + + /* find qos */ + cmd =3D virNetDevOpenvswitchCreateCmd(); + virUUIDFormat(vmid, vmuuidstr); + vmid_ex_id =3D g_strdup_printf("external-ids:vm-id=3D\"%s\"", vmuuidst= r); + virCommandAddArgList(cmd, "--no-heading", "--columns=3D_uuid", "find",= "qos", vmid_ex_id, NULL); + virCommandSetOutputBuffer(cmd, &qos_uuid); + if (virCommandRun(cmd, NULL) < 0) { + VIR_WARN("Unable to find qos on port %s", ifname); + } + + /* find queue */ + virCommandFree(cmd); + cmd =3D virNetDevOpenvswitchCreateCmd(); + vmid_ex_id =3D g_strdup_printf("external-ids:vm-id=3D\"%s\"", vmuuidst= r); + virCommandAddArgList(cmd, "--no-heading", "--columns=3D_uuid", "find",= "queue", vmid_ex_id, NULL); + virCommandSetOutputBuffer(cmd, &queue_uuid); + if (virCommandRun(cmd, NULL) < 0) { + VIR_WARN("Unable to find queue on port %s", ifname); + } + + if (qos_uuid && *qos_uuid) { + lines =3D g_strsplit(qos_uuid, "\n", 0); + /* destroy qos */ + for (i =3D 0; lines[i] !=3D NULL; i++) { + const char *line =3D lines[i]; + if (!*line) { + continue; + } + virCommandFree(cmd); + cmd =3D virNetDevOpenvswitchCreateCmd(); + virCommandAddArgList(cmd, "--no-heading", "--columns=3D_uuid",= "--if-exists", + "list", "port", ifname, "qos", NULL); + virCommandSetOutputBuffer(cmd, &port_qos); + if (virCommandRun(cmd, NULL) < 0) { + VIR_WARN("Unable to remove port qos on port %s", ifname); + } + if (port_qos && *port_qos) { + virCommandFree(cmd); + cmd =3D virNetDevOpenvswitchCreateCmd(); + virCommandAddArgList(cmd, "remove", "port", ifname, "qos",= line, NULL); + if (virCommandRun(cmd, NULL) < 0) { + VIR_WARN("Unable to remove port qos on port %s", ifnam= e); + } + } + virCommandFree(cmd); + cmd =3D virNetDevOpenvswitchCreateCmd(); + virCommandAddArgList(cmd, "destroy", "qos", line, NULL); + if (virCommandRun(cmd, NULL) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Unable to destroy qos on port %s"), ifna= me); + return -1; + } + } + } + /* destroy queue */ + if (queue_uuid && *queue_uuid) { + lines =3D g_strsplit(queue_uuid, "\n", 0); + for (i =3D 0; lines[i] !=3D NULL; i++) { + const char *line =3D lines[i]; + if (!*line) { + continue; + } + virCommandFree(cmd); + cmd =3D virNetDevOpenvswitchCreateCmd(); + virCommandAddArgList(cmd, "destroy", "queue", line, NULL); + if (virCommandRun(cmd, NULL) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("Unable to destroy queue on port %s"), if= name); + return -1; + } + } + } + + return 0; +} \ No newline at end of file diff --git a/src/util/virnetdevopenvswitch.h b/src/util/virnetdevopenvswitc= h.h index 7525376855..2dcd1aec6b 100644 --- a/src/util/virnetdevopenvswitch.h +++ b/src/util/virnetdevopenvswitch.h @@ -21,6 +21,7 @@ #pragma once =20 #include "internal.h" +#include "virnetdevbandwidth.h" #include "virnetdevvportprofile.h" #include "virnetdevvlan.h" =20 @@ -69,3 +70,13 @@ int virNetDevOpenvswitchGetVhostuserIfname(const char *p= ath, int virNetDevOpenvswitchUpdateVlan(const char *ifname, const virNetDevVlan *virtVlan) ATTRIBUTE_NONNULL(1) G_GNUC_WARN_UNUSED_RESULT; + +int virNetDevOpenvswitchInterfaceSetQos(const char *ifname, + const virNetDevBandwidth *bandwidt= h, + const unsigned char *vmid, + bool swapped) + ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(3) G_GNUC_WARN_UNUSED_RESULT; + +int virNetDevOpenvswitchInterfaceClearQos(const char *ifname, + const unsigned char *vmid) + ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) G_GNUC_WARN_UNUSED_RESULT; --=20 2.30.2.windows.1