From nobody Wed Apr 15 07:02:50 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) client-ip=8.43.85.245; envelope-from=devel-bounces@lists.libvirt.org; helo=lists.libvirt.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=pass(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1775585826; cv=none; d=zohomail.com; s=zohoarc; b=RYThHW9oklNaTisklQIisaJ20RmzhTDaVCRNj02eptTQg6mJ7TtMxjPd2h7NH8QciNDOxVnKMNJqdcqBHo1N9unOb+bR6jygHdiibQ5gh8q9n4Hdx9Tou/SVi/Cx6Nw06LiIHmcQ/m7aOaqRCX0QNOHgK4JDAxyQYKarFf7IWBI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1775585826; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:List-Subscribe:List-Post:List-Owner:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Subject:Subject:To:To:Message-Id:Reply-To; bh=HQzKLjIXuYXQ5Snq+oP3p5UZBvAVn8UNW6JSXs2xjVU=; b=CfhNBJPy2QHk8PEg8+QzzBbyi3msDnnKgGHQKdPf2tnmKIA1GXx8NuHzuA0RDhPWPly6Pf3Aeo/CHCW2XfeYNm0rI4dqCLU5unacESMzasjQLctkiD8xl3qu6WDMBup+WOLoxEjhfolqq2n+FNqf6W56B7UMycH5kqdiEA8EdJU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.libvirt.org designates 8.43.85.245 as permitted sender) smtp.mailfrom=devel-bounces@lists.libvirt.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.libvirt.org (lists.libvirt.org [8.43.85.245]) by mx.zohomail.com with SMTPS id 1775585826824871.4384396763774; Tue, 7 Apr 2026 11:17:06 -0700 (PDT) Received: by lists.libvirt.org (Postfix, from userid 993) id E16093F9B4; Tue, 7 Apr 2026 14:17:05 -0400 (EDT) Received: from [172.19.199.12] (lists.libvirt.org [8.43.85.245]) by lists.libvirt.org (Postfix) with ESMTP id 0438D418D3; Tue, 7 Apr 2026 14:16:07 -0400 (EDT) Received: by lists.libvirt.org (Postfix, from userid 993) id 9121F3F8A8; Tue, 7 Apr 2026 14:16:02 -0400 (EDT) Received: from mail-ed1-f47.google.com (mail-ed1-f47.google.com [209.85.208.47]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (3072 bits) server-digest SHA256) (No client certificate requested) by lists.libvirt.org (Postfix) with ESMTPS id 196003F2EF for ; Tue, 7 Apr 2026 14:16:01 -0400 (EDT) Received: by mail-ed1-f47.google.com with SMTP id 4fb4d7f45d1cf-66e6f1d8237so2439655a12.3 for ; Tue, 07 Apr 2026 11:16:01 -0700 (PDT) Received: from tulp.my.domain (2001-1c02-1a15-3000-ee82-4536-a8f2-9e22.cable.dynamic.v6.ziggo.nl. [2001:1c02:1a15:3000:ee82:4536:a8f2:9e22]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-66e034c6750sm4428645a12.27.2026.04.07.11.15.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Apr 2026 11:15:58 -0700 (PDT) X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on lists.libvirt.org X-Spam-Level: X-Spam-Status: No, score=-4.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,HELO_MISC_IP,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED,RCVD_IN_VALIDITY_CERTIFIED_BLOCKED, RCVD_IN_VALIDITY_RPBL_BLOCKED,RCVD_IN_VALIDITY_SAFE_BLOCKED,SPF_PASS autolearn=unavailable autolearn_force=no version=4.0.1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1775585760; x=1776190560; darn=lists.libvirt.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=HQzKLjIXuYXQ5Snq+oP3p5UZBvAVn8UNW6JSXs2xjVU=; b=dem5aIHOCgDUyILJJbLWF0gvoqCGT+R6RLP8uuNRa5TPF7txKKTRByJrt1Qq8DMRey 9rNBqGHw5ry/t5roObcRKIhqmH2b3mOrXgWKXQNCZYZs5J2UGH7l4V/8QX1hqoPav8CL iCUVpquYLyZjTkrs1tJgd3FZQrCnxHyM6MJeFPB83izbmIlCPyrC1+by2vXEdpsevn3P q07mOI2LTcQ7RB+DK+gb9jtQXtYnt/SGBHeiNJuokpzimPQKW+iybiwtQGy2ILKIJ79B uDF9IPITXdVh6UF844m7lnU/piqF5bCXIwxMIuiePCuTgyep/UVE9HCKqFAjTEF0YOqn I3Rw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775585760; x=1776190560; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=HQzKLjIXuYXQ5Snq+oP3p5UZBvAVn8UNW6JSXs2xjVU=; b=rk4BFbcuA3AyoRz5CSP3m3pAjPmILN3Z5ZVmDJWYhHbC0B3hKrYt2WoWLTXyvZ9lgj DW+JEzVRRWj8rfoE9BFUlDEfjrGM5TSQAw+ax8z1Mf0GOj5Y/NYPp5PHJ38dLCp8x663 x/hq41G8NrBUTyWmxHuLTWBmT0MrtDhBTaHsuNfXvYxZh0h0d4J+YxMPTk2uSQH1msbH paQ0fKrH/LeVWHwU4U7uCu4OglC5SKSZf982v++/W0WVkqY6P01ZOJuREEc0erv31ejo HiJHT61GbDl7aPsnj0bIrSIpdOM40a8NCIFea0+MbjM5OG95OXnyJ0W7r8yGcoUMU9p1 wY9w== X-Gm-Message-State: AOJu0YyjeaK4d9oPr2rFEcIb1Iabwscgps56rXDY6DccRBcyO7d6wAf3 AzbnDPhvCzOhtqBovbUnhO21hE3ar5cmO6X/OVClBH7EovJXlwX6hmzDinJI2g72 X-Gm-Gg: AeBDiev7yWAgfes/pgkPYl5oEeN6P04mPwu38Pz6k6TIq3lC23gzcoZ4Il4lidYynyE N3WeHEh1FSGVJr2qo3poXV3hWXhwjR2vI/oZzPZplUnKU37KNLdAOvyY2C+tHEXpOBaBdVi21N3 eTZvmxJVLSWm2iVm6zrhtVfw99JzwwPxXgTlaRGHazf3IR5oKMM26/f6e4pH/2bi/s9fvE7R8jd Ec5ueNgITwY4LMThHuHz+n3KcRQGzwBjasdNGiSSVX3t6YK180MYHaaKF9xWKxjiOIP/bOk/TVT v3hIqIDMET9B0MkPnLmKrbcjfZx4h7Ucy/dnILSiyPKqJm0MkX7Co030CvItS/AnAEvMzNssa59 WE9uWKV7cx1B+BVU9uqZw5E9YT5v3VPabsodkLbcgaHicnJ364MDd//pdo+paqNIJwt+vp/tx1L PBkQnvQR3vmm4wCCR8ooAIZ3NXANEwL3DNymZ5VYpNbFsXH53g87JqRkhSJ7qCdYhW1rSd9KM8C xZZsGCfZNrvx/DbUsfegTCkrnEzJCl67xme4Q== X-Received: by 2002:a17:907:1903:b0:b98:2c44:6631 with SMTP id a640c23a62f3a-b9c6742c9f5mr919919366b.14.1775585759190; Tue, 07 Apr 2026 11:15:59 -0700 (PDT) From: Roman Bogorodskiy To: devel@lists.libvirt.org Subject: [PATCH] bhyve: add blkiotune support Date: Tue, 7 Apr 2026 20:13:33 +0200 Message-ID: <20260407181333.69261-1-bogorodskiy@gmail.com> X-Mailer: git-send-email 2.52.0 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Message-ID-Hash: BHQXYNHHLJNFJF5EBCJG6GG3QEFJOKJV X-Message-ID-Hash: BHQXYNHHLJNFJF5EBCJG6GG3QEFJOKJV X-MailFrom: bogorodskiy@gmail.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; header-match-devel.lists.libvirt.org-0; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Roman Bogorodskiy X-Mailman-Version: 3.3.10 Precedence: list List-Id: Development discussions about the libvirt library & tools Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: X-ZohoMail-DKIM: pass (identity @gmail.com) X-ZM-MESSAGEID: 1775585827634158500 Content-Type: text/plain; charset="utf-8" FreeBSD supports resource limiting with the rctl(4) framework. It supports various resource types, including I/O resources. It allows to limit resources for users, processes, login classes, and jails. To apply blkiotune limits set limits for the bhyve process. I/O related resources supported by rctl(4) are: readbps filesystem reads, in bytes per second writebps filesystem writes, in bytes per second readiops filesystem reads, in operations per second writeiops filesystem writes, in operations per second Thus, the actual commands look like: rctl -a process:$bhyvepid:writebps:throttle=3D10000000 rctl -a process:$bhyvepid:readbps:throttle=3D10000000 rctl -a process:$bhyvepid:writeiops:throttle=3D20000 rctl -a process:$bhyvepid:readiops:throttle=3D20000 This is different from the current blkiotune modeling in libvirt as it requires specific device to apply limits to. To adapt this model to per-domain I/O limits, update domain schema to specify "*" as a device name. The rctl(8) may be not available or not enabled, so add a capability check for that. Per process rules get removed when the process disappears, so no special clean up is necessary. Signed-off-by: Roman Bogorodskiy --- src/bhyve/bhyve_capabilities.c | 25 +++++++- src/bhyve/bhyve_capabilities.h | 1 + src/bhyve/bhyve_domain.c | 6 ++ src/bhyve/bhyve_process.c | 59 +++++++++++++++++++ src/conf/schemas/domaincommon.rng | 5 +- ...yvexml2argv-blkiotune-multiple-devices.xml | 39 ++++++++++++ .../x86_64/bhyvexml2argv-blkiotune.args | 10 ++++ .../x86_64/bhyvexml2argv-blkiotune.ldargs | 4 ++ .../x86_64/bhyvexml2argv-blkiotune.xml | 32 ++++++++++ tests/bhyvexml2argvtest.c | 2 + .../x86_64/bhyvexml2xmlout-blkiotune.xml | 42 +++++++++++++ tests/bhyvexml2xmltest.c | 2 + 12 files changed, 225 insertions(+), 2 deletions(-) create mode 100644 tests/bhyvexml2argvdata/x86_64/bhyvexml2argv-blkiotune-= multiple-devices.xml create mode 100644 tests/bhyvexml2argvdata/x86_64/bhyvexml2argv-blkiotune.= args create mode 100644 tests/bhyvexml2argvdata/x86_64/bhyvexml2argv-blkiotune.= ldargs create mode 100644 tests/bhyvexml2argvdata/x86_64/bhyvexml2argv-blkiotune.= xml create mode 100644 tests/bhyvexml2xmloutdata/x86_64/bhyvexml2xmlout-blkiot= une.xml diff --git a/src/bhyve/bhyve_capabilities.c b/src/bhyve/bhyve_capabilities.c index c3fb88fe9f..1088d75407 100644 --- a/src/bhyve/bhyve_capabilities.c +++ b/src/bhyve/bhyve_capabilities.c @@ -24,6 +24,7 @@ #include #include #include +#include #include =20 #include "viralloc.h" @@ -40,7 +41,6 @@ =20 VIR_LOG_INIT("bhyve.bhyve_capabilities"); =20 - virCaps * virBhyveCapsBuild(void) { @@ -334,6 +334,27 @@ bhyveProbeCapsVNCPassword(unsigned int *caps, char *bi= nary) } =20 =20 +static int +bhyveProbeCapsRctl(unsigned int *caps) +{ + bool racct_enable; + size_t racct_enable_len; + g_autofree char *rctl =3D NULL; + + if (!(rctl =3D virFindFileInPath("rctl"))) + return 0; + + racct_enable_len =3D sizeof(racct_enable); + if (sysctlbyname("kern.racct.enable", &racct_enable, + &racct_enable_len, NULL, 0) < 0) + return 0; + + if (racct_enable) + *caps |=3D BHYVE_CAP_RCTL; + + return 0; +} + int virBhyveProbeCaps(unsigned int *caps) { @@ -356,6 +377,8 @@ virBhyveProbeCaps(unsigned int *caps) if ((ret =3D bhyveProbeCapsVNCPassword(caps, binary))) goto out; =20 + if ((ret =3D bhyveProbeCapsRctl(caps))) + goto out; =20 out: VIR_FREE(binary); diff --git a/src/bhyve/bhyve_capabilities.h b/src/bhyve/bhyve_capabilities.h index 31fd9ab86a..0302b68e22 100644 --- a/src/bhyve/bhyve_capabilities.h +++ b/src/bhyve/bhyve_capabilities.h @@ -57,6 +57,7 @@ typedef enum { BHYVE_CAP_NVME =3D 1 << 11, BHYVE_CAP_ACPI =3D 1 << 12, BHYVE_CAP_NUMA =3D 1 << 13, + BHYVE_CAP_RCTL =3D 1 << 14, } virBhyveCapsFlags; =20 int virBhyveProbeGrubCaps(virBhyveGrubCapsFlags *caps); diff --git a/src/bhyve/bhyve_domain.c b/src/bhyve/bhyve_domain.c index 4594d7673f..48e1451d8b 100644 --- a/src/bhyve/bhyve_domain.c +++ b/src/bhyve/bhyve_domain.c @@ -464,6 +464,12 @@ bhyveDomainDefValidate(const virDomainDef *def, } } =20 + if (def->blkio.ndevices > 1) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("Per device I/O tuning is not supported")); + return -1; + } + if (!def->os.loader) return 0; =20 diff --git a/src/bhyve/bhyve_process.c b/src/bhyve/bhyve_process.c index 1d436da609..b5ebe0aa61 100644 --- a/src/bhyve/bhyve_process.c +++ b/src/bhyve/bhyve_process.c @@ -34,6 +34,7 @@ =20 #include "bhyve_device.h" #include "bhyve_driver.h" +#include "bhyve_capabilities.h" #include "bhyve_command.h" #include "bhyve_firmware.h" #include "bhyve_monitor.h" @@ -132,6 +133,61 @@ bhyveProcessStopHook(struct _bhyveConn *driver, VIR_HOOK_SUBOP_END, NULL, xml, NULL); } =20 +static int +bhyveSetResourceLimits(struct _bhyveConn *driver, virDomainObj *vm) +{ + virBlkioDevice *device; + char **rules =3D NULL; + size_t i, nrules =3D 0; + int ret =3D -1; + + if (!vm->def->blkio.ndevices) + return 0; + + if ((bhyveDriverGetBhyveCaps(driver) & BHYVE_CAP_RCTL) =3D=3D 0) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("Cannot set resource limits: RACCT/RCTL is either= not supported or not enabled")); + return ret; + } + + device =3D &vm->def->blkio.devices[0]; + +#define BHYVE_APPEND_RCTL_RULE(field, type, format) \ + do { \ + if ((field)) { \ + virBuffer buf =3D VIR_BUFFER_INITIALIZER; \ + char *rule; \ + virBufferAsprintf(&buf, "process:%d:" type ":throttle=3D" form= at, \ + vm->pid, (field)); \ + rule =3D virBufferContentAndReset(&buf); \ + VIR_APPEND_ELEMENT(rules, nrules, rule); \ + } \ + } while (0) + + BHYVE_APPEND_RCTL_RULE(device->riops, "readiops", "%u"); + BHYVE_APPEND_RCTL_RULE(device->wiops, "writeiops", "%u"); + BHYVE_APPEND_RCTL_RULE(device->rbps, "readbps", "%llu"); + BHYVE_APPEND_RCTL_RULE(device->wbps, "writebps", "%llu"); + +#undef BHYVE_APPEND_RCTL_RULE + + for (i =3D 0; i < nrules; i++) { + g_autoptr(virCommand) cmd =3D virCommandNew("rctl"); + virCommandAddArgList(cmd, "-a", rules[i], NULL); + + if (virCommandRun(cmd, NULL) < 0) + goto cleanup; + } + + ret =3D 0; + + cleanup: + for (i =3D 0; i < nrules; i++) + VIR_FREE(rules[i]); + VIR_FREE(rules); + return ret; +} + static int virBhyveProcessStartImpl(struct _bhyveConn *driver, virDomainObj *vm, @@ -258,6 +314,9 @@ virBhyveProcessStartImpl(struct _bhyveConn *driver, BHYVE_STATE_DIR) < 0) goto cleanup; =20 + if (bhyveSetResourceLimits(driver, vm) < 0) + goto cleanup; + if (bhyveProcessStartHook(driver, vm, VIR_HOOK_BHYVE_OP_STARTED) < 0) goto cleanup; =20 diff --git a/src/conf/schemas/domaincommon.rng b/src/conf/schemas/domaincom= mon.rng index db1dcd3bb7..c7f442a4c1 100644 --- a/src/conf/schemas/domaincommon.rng +++ b/src/conf/schemas/domaincommon.rng @@ -1049,7 +1049,10 @@ - + + + * + diff --git a/tests/bhyvexml2argvdata/x86_64/bhyvexml2argv-blkiotune-multipl= e-devices.xml b/tests/bhyvexml2argvdata/x86_64/bhyvexml2argv-blkiotune-mult= iple-devices.xml new file mode 100644 index 0000000000..a6b689ad9a --- /dev/null +++ b/tests/bhyvexml2argvdata/x86_64/bhyvexml2argv-blkiotune-multiple-devic= es.xml @@ -0,0 +1,39 @@ + + bhyve + df3be7e7-a104-11e3-aeb0-50e5492bd3dc + 219136 + 1 + + hvm + + + + * + 10000 + 10000 + 20000 + 20000 + + + /dev/hda + 10000 + 10000 + 20000 + 20000 + + + + + + + +
+ + + + + +
+ + + diff --git a/tests/bhyvexml2argvdata/x86_64/bhyvexml2argv-blkiotune.args b/= tests/bhyvexml2argvdata/x86_64/bhyvexml2argv-blkiotune.args new file mode 100644 index 0000000000..507e0be668 --- /dev/null +++ b/tests/bhyvexml2argvdata/x86_64/bhyvexml2argv-blkiotune.args @@ -0,0 +1,10 @@ +bhyve \ +-c 1 \ +-m 214 \ +-u \ +-H \ +-P \ +-s 0:0,hostbridge \ +-s 2:0,ahci,hd:/tmp/freebsd.img \ +-s 3:0,virtio-net,faketapdev,mac=3D52:54:00:b9:94:02 \ +bhyve diff --git a/tests/bhyvexml2argvdata/x86_64/bhyvexml2argv-blkiotune.ldargs = b/tests/bhyvexml2argvdata/x86_64/bhyvexml2argv-blkiotune.ldargs new file mode 100644 index 0000000000..5905f4b3e6 --- /dev/null +++ b/tests/bhyvexml2argvdata/x86_64/bhyvexml2argv-blkiotune.ldargs @@ -0,0 +1,4 @@ +bhyveload \ +-m 214 \ +-d /tmp/freebsd.img \ +bhyve diff --git a/tests/bhyvexml2argvdata/x86_64/bhyvexml2argv-blkiotune.xml b/t= ests/bhyvexml2argvdata/x86_64/bhyvexml2argv-blkiotune.xml new file mode 100644 index 0000000000..956d96cf18 --- /dev/null +++ b/tests/bhyvexml2argvdata/x86_64/bhyvexml2argv-blkiotune.xml @@ -0,0 +1,32 @@ + + bhyve + df3be7e7-a104-11e3-aeb0-50e5492bd3dc + 219136 + 1 + + hvm + + + + * + 10000 + 10000 + 20000 + 20000 + + + + + + + +
+ + + + + +
+ + + diff --git a/tests/bhyvexml2argvtest.c b/tests/bhyvexml2argvtest.c index b7749fec6f..c098149ad5 100644 --- a/tests/bhyvexml2argvtest.c +++ b/tests/bhyvexml2argvtest.c @@ -289,6 +289,8 @@ mymain(void) DO_TEST_FAILURE("slirp-ip"); DO_TEST("virtio-scsi"); DO_TEST("vcpupin"); + DO_TEST("blkiotune"); + DO_TEST_FAILURE("blkiotune-multiple-devices"); =20 /* Address allocation tests */ DO_TEST("addr-single-sata-disk"); diff --git a/tests/bhyvexml2xmloutdata/x86_64/bhyvexml2xmlout-blkiotune.xml= b/tests/bhyvexml2xmloutdata/x86_64/bhyvexml2xmlout-blkiotune.xml new file mode 100644 index 0000000000..4170303a6e --- /dev/null +++ b/tests/bhyvexml2xmloutdata/x86_64/bhyvexml2xmlout-blkiotune.xml @@ -0,0 +1,42 @@ + + bhyve + df3be7e7-a104-11e3-aeb0-50e5492bd3dc + 219136 + 219136 + + + * + 20000 + 20000 + 10000 + 10000 + + + 1 + + hvm + + + + destroy + restart + destroy + + + + + +
+ + + +
+ + + + + +
+ + + diff --git a/tests/bhyvexml2xmltest.c b/tests/bhyvexml2xmltest.c index 950aaea672..ae12b37064 100644 --- a/tests/bhyvexml2xmltest.c +++ b/tests/bhyvexml2xmltest.c @@ -133,6 +133,8 @@ mymain(void) DO_TEST_DIFFERENT("slirp"); DO_TEST_DIFFERENT("virtio-scsi"); DO_TEST_DIFFERENT("numa"); + DO_TEST_DIFFERENT("blkiotune"); + DO_TEST_FAILURE("blkiotune-multiple-devices"); =20 /* Address allocation tests */ DO_TEST_DIFFERENT("addr-single-sata-disk"); --=20 2.52.0