From nobody Fri Dec 19 06:33:49 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1658918234; cv=none; d=zohomail.com; s=zohoarc; b=dhloaNAbHgKmx1W0vkxCXGMWRVALbB05tgY3VS5Mv1SIH/5LV4yAd2o+w95yqw8D1cnvLAbZZlmWSMQ0hE/dhCztOmRVEB3SCSCRLufn3uYvporRbkIT9hQDzrRL2JlOik18Iin7GbDNOwv0xkYBRpR65Szw35E/4oLDy2rAPj4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1658918234; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=HZfO0lTBM9JKLBroneUt4J0XdFL5PwvTXbgTf+ewBzw=; b=IAYbHqzFSwWFqz/JBIyi2uQHp25vFeubp7sB4L67cfU1v4NYykfS2z0qk77fhs1EIOPDOdfpFZ9YzdHYQrcqdVOFsI4uSspz7akcnVOrH1uSYHkl8PFIUYZ3+J+SK5jUMbKa4EIa1glE02HUotb8zg6dYqft7H2H2OvSRmz82f8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1658918234928541.2257377392831; Wed, 27 Jul 2022 03:37:14 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-385-e5Oc_oQlPsq7vtz1xXrprA-1; Wed, 27 Jul 2022 06:35:30 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4D704824068; Wed, 27 Jul 2022 10:35:21 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 339491121314; Wed, 27 Jul 2022 10:35:21 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 01EA21945DA8; Wed, 27 Jul 2022 10:35:21 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 0B73C1945DB0 for ; Wed, 27 Jul 2022 10:35:19 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id D1706C33AE6; Wed, 27 Jul 2022 10:35:18 +0000 (UTC) Received: from maggie.redhat.com (unknown [10.43.2.88]) by smtp.corp.redhat.com (Postfix) with ESMTP id 20317C28130; Wed, 27 Jul 2022 10:35:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1658918233; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=HZfO0lTBM9JKLBroneUt4J0XdFL5PwvTXbgTf+ewBzw=; b=PxB6EGpIfI2ZTwp907t37gK3kTRoMce2+T6EvPY1KMPnNDXuekNg0Jih0VeO7cHAnbb+S2 u/Kg10k6A5jXYAzlaPq81mK8yfF+c6g/JTncdI6vYn3UefNvzw9jaMyqCmzVQ60eJeLlH+ uFgzEKxLeOnF/iMRTIJ/1yS0LQCmulM= X-MC-Unique: e5Oc_oQlPsq7vtz1xXrprA-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Michal Privoznik To: libvir-list@redhat.com Subject: [PATCH v14 15/15] qemu: Add command-line to generate SGX EPC memory backend Date: Wed, 27 Jul 2022 12:35:01 +0200 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.85 on 10.11.54.8 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lin.a.yang@intel.com, jian-feng.ding@intel.com, haibin.huang@intel.com Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1658918236358100001 Content-Type: text/plain; charset="utf-8"; x-default="true" From: Lin Yang According to the result parsing from xml, add the argument of SGX EPC memory backend into QEMU command line. With NUMA config: #qemu-system-x86_64 \ ...... \ -object '{"qom-type":"memory-backend-epc","id":"memepc0","prealloc"= :true,"size":67108864,"host-nodes":[0,1],"policy":"bind"}' \ -object '{"qom-type":"memory-backend-epc","id":"memepc1","prealloc"= :true,"size":16777216,"host-nodes":[2,3],"policy":"bind"}' \ -machine sgx-epc.0.memdev=3Dmemepc0,sgx-epc.0.node=3D0,sgx-epc.1.me= mdev=3Dmemepc1,sgx-epc.1.node=3D1 Without NUMA config: #qemu-system-x86_64 \ ...... \ -object '{"qom-type":"memory-backend-epc","id":"memepc0","prealloc"= :true,"size":67108864}' \ -object '{"qom-type":"memory-backend-epc","id":"memepc1","prealloc"= :true,"size":16777216}' \ -machine sgx-epc.0.memdev=3Dmemepc0,sgx-epc.1.memdev=3Dmemepc1 Signed-off-by: Lin Yang Signed-off-by: Haibin Huang Signed-off-by: Michal Privoznik --- src/qemu/qemu_alias.c | 3 +- src/qemu/qemu_command.c | 82 +++++++++++++++++-- src/qemu/qemu_monitor_json.c | 41 ++++++++-- .../sgx-epc-numa.x86_64-latest.args | 40 +++++++++ .../sgx-epc.x86_64-6.2.0.args | 37 +++++++++ tests/qemuxml2argvtest.c | 3 + 6 files changed, 190 insertions(+), 16 deletions(-) create mode 100644 tests/qemuxml2argvdata/sgx-epc-numa.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/sgx-epc.x86_64-6.2.0.args diff --git a/src/qemu/qemu_alias.c b/src/qemu/qemu_alias.c index e7f76068cb..273b97697f 100644 --- a/src/qemu/qemu_alias.c +++ b/src/qemu/qemu_alias.c @@ -466,7 +466,8 @@ qemuDeviceMemoryGetAliasID(virDomainDef *def, * valid */ if (!oldAlias && mem->model !=3D VIR_DOMAIN_MEMORY_MODEL_VIRTIO_PMEM && - mem->model !=3D VIR_DOMAIN_MEMORY_MODEL_VIRTIO_MEM) + mem->model !=3D VIR_DOMAIN_MEMORY_MODEL_VIRTIO_MEM && + mem->model !=3D VIR_DOMAIN_MEMORY_MODEL_SGX_EPC) return mem->info.addr.dimm.slot; =20 for (i =3D 0; i < def->nmems; i++) { diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index f3f6870c58..304a2ae6d1 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -3718,7 +3718,11 @@ qemuBuildMemoryBackendProps(virJSONValue **backendPr= ops, =20 props =3D virJSONValueNewObject(); =20 - if (!mem->nvdimmPath && + if (mem->model =3D=3D VIR_DOMAIN_MEMORY_MODEL_SGX_EPC) { + backendType =3D "memory-backend-epc"; + if (!priv->memPrealloc) + prealloc =3D true; + } else if (!mem->nvdimmPath && def->mem.source =3D=3D VIR_DOMAIN_MEMORY_SOURCE_MEMFD) { backendType =3D "memory-backend-memfd"; =20 @@ -3733,7 +3737,6 @@ qemuBuildMemoryBackendProps(virJSONValue **backendPro= ps, =20 if (systemMemory) disableCanonicalPath =3D true; - } else if (useHugepage || mem->nvdimmPath || memAccess || def->mem.source =3D=3D VIR_DOMAIN_MEMORY_SOURCE_FILE) { =20 @@ -7080,6 +7083,9 @@ qemuAppendDomainMemoryMachineParams(virBuffer *buf, const virDomainDef *def, virQEMUCaps *qemuCaps) { + virSGXCapability *sgxCaps =3D virQEMUCapsGetSGXCapabilities(qemuCaps); + bool nvdimmAdded =3D false; + int epcNum =3D 0; size_t i; =20 if (def->mem.dump_core) { @@ -7094,8 +7100,52 @@ qemuAppendDomainMemoryMachineParams(virBuffer *buf, virBufferAddLit(buf, ",mem-merge=3Doff"); =20 for (i =3D 0; i < def->nmems; i++) { - if (def->mems[i]->model =3D=3D VIR_DOMAIN_MEMORY_MODEL_NVDIMM) { - virBufferAddLit(buf, ",nvdimm=3Don"); + int targetNode =3D def->mems[i]->targetNode; + + switch (def->mems[i]->model) { + case VIR_DOMAIN_MEMORY_MODEL_NVDIMM: + if (!nvdimmAdded) { + virBufferAddLit(buf, ",nvdimm=3Don"); + nvdimmAdded =3D true; + } + break; + + case VIR_DOMAIN_MEMORY_MODEL_SGX_EPC: + /* add sgx epc memory to -machine parameter */ + + if (targetNode < 0) { + /* set NUMA target node to 0 by default if user doesn't + * specify it. */ + targetNode =3D 0; + } + + if (sgxCaps->nsections =3D=3D 0) { + /* Assume QEMU cannot specify guest NUMA node for each SGX= EPC, + * because it doesn't provide EPC NUMA info + */ + if (targetNode > 0) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("can't add SGX EPC for guest node '%d= ' because this QEMU version didn't provide SGX EPC NUMA info"), + targetNode); + return -1; + } + + virBufferAsprintf(buf, ",sgx-epc.%d.memdev=3Dmem%s", + epcNum, def->mems[i]->info.alias); + } else { + /* The .node attribute is required since QEMU provide EPC = NUMA info */ + virBufferAsprintf(buf, ",sgx-epc.%d.memdev=3Dmem%s,sgx-epc= .%d.node=3D%d", + epcNum, def->mems[i]->info.alias, epcNum= , targetNode); + } + + epcNum++; + break; + + case VIR_DOMAIN_MEMORY_MODEL_DIMM: + case VIR_DOMAIN_MEMORY_MODEL_VIRTIO_PMEM: + case VIR_DOMAIN_MEMORY_MODEL_VIRTIO_MEM: + case VIR_DOMAIN_MEMORY_MODEL_NONE: + case VIR_DOMAIN_MEMORY_MODEL_LAST: break; } } @@ -7813,11 +7863,27 @@ qemuBuildMemoryDeviceCommandLine(virCommand *cmd, if (qemuBuildMemoryDimmBackendStr(cmd, def->mems[i], def, cfg, pri= v) < 0) return -1; =20 - if (!(props =3D qemuBuildMemoryDeviceProps(cfg, priv, def, def->me= ms[i]))) - return -1; + switch (def->mems[i]->model) { + case VIR_DOMAIN_MEMORY_MODEL_NVDIMM: + case VIR_DOMAIN_MEMORY_MODEL_DIMM: + case VIR_DOMAIN_MEMORY_MODEL_VIRTIO_PMEM: + case VIR_DOMAIN_MEMORY_MODEL_VIRTIO_MEM: + if (!(props =3D qemuBuildMemoryDeviceProps(cfg, priv, def, def= ->mems[i]))) + return -1; =20 - if (qemuBuildDeviceCommandlineFromJSON(cmd, props, def, priv->qemu= Caps) < 0) - return -1; + if (qemuBuildDeviceCommandlineFromJSON(cmd, props, def, priv->= qemuCaps) < 0) + return -1; + + break; + + /* sgx epc memory will be added to -machine parameter, so skip her= e */ + case VIR_DOMAIN_MEMORY_MODEL_SGX_EPC: + break; + + case VIR_DOMAIN_MEMORY_MODEL_NONE: + case VIR_DOMAIN_MEMORY_MODEL_LAST: + break; + } } =20 return 0; diff --git a/src/qemu/qemu_monitor_json.c b/src/qemu/qemu_monitor_json.c index b045efa203..c41a6355ba 100644 --- a/src/qemu/qemu_monitor_json.c +++ b/src/qemu/qemu_monitor_json.c @@ -7575,13 +7575,25 @@ qemuMonitorJSONGetMemoryDeviceInfo(qemuMonitor *mon, return -1; } =20 - /* While 'id' attribute is marked as optional in QEMU's QAPI - * specification, Libvirt always sets it. Thus we can fail if not - * present. */ - if (!(devalias =3D virJSONValueObjectGetString(dimminfo, "id"))) { - virReportError(VIR_ERR_INTERNAL_ERROR, "%s", - _("dimm memory info data is missing 'id'")); - return -1; + if (STREQ(type, "dimm") || STREQ(type, "nvdimm") || STREQ(type, "v= irtio-mem")) { + /* While 'id' attribute is marked as optional in QEMU's QAPI + * specification, Libvirt always sets it. Thus we can fail if n= ot + * present. */ + if (!(devalias =3D virJSONValueObjectGetString(dimminfo, "id")= )) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("dimm memory info data is missing 'id'")); + return -1; + } + } else if (STREQ(type, "sgx-epc")) { + if (!(devalias =3D virJSONValueObjectGetString(dimminfo, "memd= ev"))) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("sgx-epc memory info data is missing 'memdev= '")); + return -1; + } + } else { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("%s memory device info is not handled yet"), typ= e); + return -1; } =20 meminfo =3D g_new0(qemuMonitorMemoryDeviceInfo, 1); @@ -7625,6 +7637,21 @@ qemuMonitorJSONGetMemoryDeviceInfo(qemuMonitor *mon, _("malformed/missing size in virtio memory = info")); return -1; } + } else if (STREQ(type, "sgx-epc")) { + /* sgx-epc memory devices */ + if (virJSONValueObjectGetNumberUlong(dimminfo, "memaddr", + &meminfo->address) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("malformed/missing memaddr in sgx-epc mem= ory info")); + return -1; + } + + if (virJSONValueObjectGetNumberUlong(dimminfo, "size", + &meminfo->size) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("malformed/missing size in sgx-epc memory= info")); + return -1; + } } else { /* type not handled yet */ continue; diff --git a/tests/qemuxml2argvdata/sgx-epc-numa.x86_64-latest.args b/tests= /qemuxml2argvdata/sgx-epc-numa.x86_64-latest.args new file mode 100644 index 0000000000..59c1041679 --- /dev/null +++ b/tests/qemuxml2argvdata/sgx-epc-numa.x86_64-latest.args @@ -0,0 +1,40 @@ +LC_ALL=3DC \ +PATH=3D/bin \ +HOME=3D/tmp/lib/domain--1-QEMUGuest1 \ +USER=3Dtest \ +LOGNAME=3Dtest \ +XDG_DATA_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.local/share \ +XDG_CACHE_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.cache \ +XDG_CONFIG_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.config \ +/usr/bin/qemu-system-x86_64 \ +-name guest=3DQEMUGuest1,debug-threads=3Don \ +-S \ +-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/tm= p/lib/domain--1-QEMUGuest1/master-key.aes"}' \ +-machine q35,usb=3Doff,dump-guest-core=3Doff,sgx-epc.0.memdev=3Dmemepc0,sg= x-epc.0.node=3D0,sgx-epc.1.memdev=3Dmemepc1,sgx-epc.1.node=3D1 \ +-accel tcg \ +-cpu qemu64 \ +-m 214 \ +-overcommit mem-lock=3Doff \ +-smp 2,sockets=3D2,cores=3D1,threads=3D1 \ +-object '{"qom-type":"memory-backend-ram","id":"ram-node0","size":11219763= 2}' \ +-numa node,nodeid=3D0,cpus=3D0,memdev=3Dram-node0 \ +-object '{"qom-type":"memory-backend-ram","id":"ram-node1","size":11219763= 2}' \ +-numa node,nodeid=3D1,cpus=3D1,memdev=3Dram-node1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=3Dcharmonitor,fd=3D1729,server=3Don,wait=3Doff \ +-mon chardev=3Dcharmonitor,id=3Dmonitor,mode=3Dcontrol \ +-rtc base=3Dutc \ +-no-shutdown \ +-no-acpi \ +-boot strict=3Don \ +-device '{"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus= ":"pcie.0","multifunction":true,"addr":"0x1"}' \ +-device '{"driver":"pcie-root-port","port":9,"chassis":2,"id":"pci.2","bus= ":"pcie.0","addr":"0x1.0x1"}' \ +-object '{"qom-type":"memory-backend-epc","id":"memepc0","prealloc":true,"= size":67108864,"host-nodes":[0,1],"policy":"bind"}' \ +-object '{"qom-type":"memory-backend-epc","id":"memepc1","prealloc":true,"= size":16777216,"host-nodes":[2,3],"policy":"bind"}' \ +-audiodev '{"id":"audio1","driver":"none"}' \ +-device '{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.1","add= r":"0x0"}' \ +-sandbox on,obsolete=3Ddeny,elevateprivileges=3Ddeny,spawn=3Ddeny,resource= control=3Ddeny \ +-msg timestamp=3Don diff --git a/tests/qemuxml2argvdata/sgx-epc.x86_64-6.2.0.args b/tests/qemux= ml2argvdata/sgx-epc.x86_64-6.2.0.args new file mode 100644 index 0000000000..57a0ee0724 --- /dev/null +++ b/tests/qemuxml2argvdata/sgx-epc.x86_64-6.2.0.args @@ -0,0 +1,37 @@ +LC_ALL=3DC \ +PATH=3D/bin \ +HOME=3D/tmp/lib/domain--1-QEMUGuest1 \ +USER=3Dtest \ +LOGNAME=3Dtest \ +XDG_DATA_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.local/share \ +XDG_CACHE_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.cache \ +XDG_CONFIG_HOME=3D/tmp/lib/domain--1-QEMUGuest1/.config \ +/usr/bin/qemu-system-x86_64 \ +-name guest=3DQEMUGuest1,debug-threads=3Don \ +-S \ +-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/tm= p/lib/domain--1-QEMUGuest1/master-key.aes"}' \ +-machine pc-q35-6.2,usb=3Doff,dump-guest-core=3Doff,sgx-epc.0.memdev=3Dmem= epc0,sgx-epc.1.memdev=3Dmemepc1,memory-backend=3Dpc.ram \ +-accel tcg \ +-cpu qemu64 \ +-m 134 \ +-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":140509184}'= \ +-overcommit mem-lock=3Doff \ +-smp 1,sockets=3D1,cores=3D1,threads=3D1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=3Dcharmonitor,fd=3D1729,server=3Don,wait=3Doff \ +-mon chardev=3Dcharmonitor,id=3Dmonitor,mode=3Dcontrol \ +-rtc base=3Dutc \ +-no-shutdown \ +-no-acpi \ +-boot strict=3Don \ +-device pcie-root-port,port=3D8,chassis=3D1,id=3Dpci.1,bus=3Dpcie.0,multif= unction=3Don,addr=3D0x1 \ +-device pcie-root-port,port=3D9,chassis=3D2,id=3Dpci.2,bus=3Dpcie.0,addr= =3D0x1.0x1 \ +-object '{"qom-type":"memory-backend-epc","id":"memepc0","prealloc":true,"= size":67108864}' \ +-object '{"qom-type":"memory-backend-epc","id":"memepc1","prealloc":true,"= size":16777216}' \ +-audiodev '{"id":"audio1","driver":"none"}' \ +-device virtio-balloon-pci,id=3Dballoon0,bus=3Dpci.1,addr=3D0x0 \ +-sandbox on,obsolete=3Ddeny,elevateprivileges=3Ddeny,spawn=3Ddeny,resource= control=3Ddeny \ +-msg timestamp=3Don diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index 05537d9e96..7a4c09b172 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -3470,6 +3470,9 @@ mymain(void) /* HVF guests should not work on Linux with KVM */ DO_TEST_CAPS_LATEST_PARSE_ERROR("hvf-x86_64-q35-headless"); =20 + DO_TEST_CAPS_VER("sgx-epc", "6.2.0"); + DO_TEST_CAPS_LATEST("sgx-epc-numa"); + if (getenv("LIBVIRT_SKIP_CLEANUP") =3D=3D NULL) virFileDeleteTree(fakerootdir); =20 --=20 2.35.1