[libvirt][PATCH RESEND v12 0/6] Support query and use SGX

Haibin Huang posted 6 patches 1 year, 11 months ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/libvirt tags/patchew/20220518075933.19943-1-haibin.huang@intel.com
There is a newer version of this series
docs/formatdomain.rst                         |   9 +-
docs/formatdomaincaps.rst                     |  26 ++++
src/conf/domain_capabilities.c                |  33 ++++
src/conf/domain_capabilities.h                |  13 ++
src/conf/domain_conf.c                        |   6 +
src/conf/domain_conf.h                        |   1 +
src/conf/domain_validate.c                    |  16 ++
src/conf/schemas/domaincaps.rng               |  22 ++-
src/conf/schemas/domaincommon.rng             |   1 +
src/libvirt_private.syms                      |   1 +
src/qemu/qemu_alias.c                         |   6 +-
src/qemu/qemu_capabilities.c                  | 145 ++++++++++++++++++
src/qemu/qemu_capabilities.h                  |   4 +
src/qemu/qemu_capspriv.h                      |   4 +
src/qemu/qemu_command.c                       |  54 ++++++-
src/qemu/qemu_domain.c                        |  38 +++--
src/qemu/qemu_domain_address.c                |   6 +
src/qemu/qemu_driver.c                        |   1 +
src/qemu/qemu_monitor.c                       |  10 ++
src/qemu/qemu_monitor.h                       |   3 +
src/qemu/qemu_monitor_json.c                  | 104 ++++++++++++-
src/qemu/qemu_monitor_json.h                  |   9 ++
src/qemu/qemu_process.c                       |   2 +
src/qemu/qemu_validate.c                      |   8 +
src/security/security_apparmor.c              |   1 +
src/security/security_dac.c                   |   2 +
src/security/security_selinux.c               |   2 +
tests/domaincapsdata/bhyve_basic.x86_64.xml   |   1 +
tests/domaincapsdata/bhyve_fbuf.x86_64.xml    |   1 +
tests/domaincapsdata/bhyve_uefi.x86_64.xml    |   1 +
tests/domaincapsdata/empty.xml                |   1 +
tests/domaincapsdata/libxl-xenfv.xml          |   1 +
tests/domaincapsdata/libxl-xenpv.xml          |   1 +
.../domaincapsdata/qemu_2.11.0-q35.x86_64.xml |   1 +
.../domaincapsdata/qemu_2.11.0-tcg.x86_64.xml |   1 +
tests/domaincapsdata/qemu_2.11.0.s390x.xml    |   1 +
tests/domaincapsdata/qemu_2.11.0.x86_64.xml   |   1 +
.../domaincapsdata/qemu_2.12.0-q35.x86_64.xml |   1 +
.../domaincapsdata/qemu_2.12.0-tcg.x86_64.xml |   1 +
.../qemu_2.12.0-virt.aarch64.xml              |   1 +
tests/domaincapsdata/qemu_2.12.0.aarch64.xml  |   1 +
tests/domaincapsdata/qemu_2.12.0.ppc64.xml    |   1 +
tests/domaincapsdata/qemu_2.12.0.s390x.xml    |   1 +
tests/domaincapsdata/qemu_2.12.0.x86_64.xml   |   1 +
.../domaincapsdata/qemu_3.0.0-q35.x86_64.xml  |   1 +
.../domaincapsdata/qemu_3.0.0-tcg.x86_64.xml  |   1 +
tests/domaincapsdata/qemu_3.0.0.ppc64.xml     |   1 +
tests/domaincapsdata/qemu_3.0.0.s390x.xml     |   1 +
tests/domaincapsdata/qemu_3.0.0.x86_64.xml    |   1 +
.../domaincapsdata/qemu_3.1.0-q35.x86_64.xml  |   1 +
.../domaincapsdata/qemu_3.1.0-tcg.x86_64.xml  |   1 +
tests/domaincapsdata/qemu_3.1.0.ppc64.xml     |   1 +
tests/domaincapsdata/qemu_3.1.0.x86_64.xml    |   1 +
.../domaincapsdata/qemu_4.0.0-q35.x86_64.xml  |   1 +
.../domaincapsdata/qemu_4.0.0-tcg.x86_64.xml  |   1 +
.../qemu_4.0.0-virt.aarch64.xml               |   1 +
tests/domaincapsdata/qemu_4.0.0.aarch64.xml   |   1 +
tests/domaincapsdata/qemu_4.0.0.ppc64.xml     |   1 +
tests/domaincapsdata/qemu_4.0.0.s390x.xml     |   1 +
tests/domaincapsdata/qemu_4.0.0.x86_64.xml    |   1 +
.../domaincapsdata/qemu_4.1.0-q35.x86_64.xml  |   1 +
.../domaincapsdata/qemu_4.1.0-tcg.x86_64.xml  |   1 +
tests/domaincapsdata/qemu_4.1.0.x86_64.xml    |   1 +
.../domaincapsdata/qemu_4.2.0-q35.x86_64.xml  |   1 +
.../domaincapsdata/qemu_4.2.0-tcg.x86_64.xml  |   1 +
.../qemu_4.2.0-virt.aarch64.xml               |   1 +
tests/domaincapsdata/qemu_4.2.0.aarch64.xml   |   1 +
tests/domaincapsdata/qemu_4.2.0.ppc64.xml     |   1 +
tests/domaincapsdata/qemu_4.2.0.s390x.xml     |   1 +
tests/domaincapsdata/qemu_4.2.0.x86_64.xml    |   1 +
.../domaincapsdata/qemu_5.0.0-q35.x86_64.xml  |   1 +
.../domaincapsdata/qemu_5.0.0-tcg.x86_64.xml  |   1 +
.../qemu_5.0.0-virt.aarch64.xml               |   1 +
tests/domaincapsdata/qemu_5.0.0.aarch64.xml   |   1 +
tests/domaincapsdata/qemu_5.0.0.ppc64.xml     |   1 +
tests/domaincapsdata/qemu_5.0.0.x86_64.xml    |   1 +
.../domaincapsdata/qemu_5.1.0-q35.x86_64.xml  |   1 +
.../domaincapsdata/qemu_5.1.0-tcg.x86_64.xml  |   1 +
tests/domaincapsdata/qemu_5.1.0.sparc.xml     |   1 +
tests/domaincapsdata/qemu_5.1.0.x86_64.xml    |   1 +
.../domaincapsdata/qemu_5.2.0-q35.x86_64.xml  |   1 +
.../domaincapsdata/qemu_5.2.0-tcg.x86_64.xml  |   1 +
.../qemu_5.2.0-virt.aarch64.xml               |   1 +
tests/domaincapsdata/qemu_5.2.0.aarch64.xml   |   1 +
tests/domaincapsdata/qemu_5.2.0.ppc64.xml     |   1 +
tests/domaincapsdata/qemu_5.2.0.s390x.xml     |   1 +
tests/domaincapsdata/qemu_5.2.0.x86_64.xml    |   1 +
.../domaincapsdata/qemu_6.0.0-q35.x86_64.xml  |   1 +
.../domaincapsdata/qemu_6.0.0-tcg.x86_64.xml  |   1 +
.../qemu_6.0.0-virt.aarch64.xml               |   1 +
tests/domaincapsdata/qemu_6.0.0.aarch64.xml   |   1 +
tests/domaincapsdata/qemu_6.0.0.s390x.xml     |   1 +
tests/domaincapsdata/qemu_6.0.0.x86_64.xml    |   1 +
.../domaincapsdata/qemu_6.1.0-q35.x86_64.xml  |   1 +
.../domaincapsdata/qemu_6.1.0-tcg.x86_64.xml  |   1 +
tests/domaincapsdata/qemu_6.1.0.x86_64.xml    |   1 +
.../domaincapsdata/qemu_6.2.0-q35.x86_64.xml  |   4 +
.../domaincapsdata/qemu_6.2.0-tcg.x86_64.xml  |   4 +
.../qemu_6.2.0-virt.aarch64.xml               |   1 +
tests/domaincapsdata/qemu_6.2.0.aarch64.xml   |   1 +
tests/domaincapsdata/qemu_6.2.0.ppc64.xml     |   1 +
tests/domaincapsdata/qemu_6.2.0.x86_64.xml    |   4 +
.../domaincapsdata/qemu_7.0.0-q35.x86_64.xml  |   4 +
.../domaincapsdata/qemu_7.0.0-tcg.x86_64.xml  |   4 +
.../qemu_7.0.0-virt.aarch64.xml               |   1 +
tests/domaincapsdata/qemu_7.0.0.aarch64.xml   |   1 +
tests/domaincapsdata/qemu_7.0.0.ppc64.xml     |   1 +
tests/domaincapsdata/qemu_7.0.0.x86_64.xml    |   4 +
.../domaincapsdata/qemu_7.1.0-q35.x86_64.xml  |   1 +
.../domaincapsdata/qemu_7.1.0-tcg.x86_64.xml  |   1 +
tests/domaincapsdata/qemu_7.1.0.x86_64.xml    |   1 +
.../caps_6.2.0.x86_64.replies                 |  22 ++-
.../caps_6.2.0.x86_64.xml                     |   5 +
.../caps_7.0.0.x86_64.replies                 |  22 ++-
.../caps_7.0.0.x86_64.xml                     |   5 +
.../caps_7.1.0.x86_64.replies                 |  21 ++-
.../sgx-epc.x86_64-6.2.0.args                 |  37 +++++
tests/qemuxml2argvdata/sgx-epc.xml            |  36 +++++
tests/qemuxml2argvtest.c                      |   2 +
.../sgx-epc.x86_64-6.2.0.xml                  |  52 +++++++
tests/qemuxml2xmltest.c                       |   2 +
121 files changed, 793 insertions(+), 40 deletions(-)
create mode 100644 tests/qemuxml2argvdata/sgx-epc.x86_64-6.2.0.args
create mode 100644 tests/qemuxml2argvdata/sgx-epc.xml
create mode 100644 tests/qemuxml2xmloutdata/sgx-epc.x86_64-6.2.0.xml
[libvirt][PATCH RESEND v12 0/6] Support query and use SGX
Posted by Haibin Huang 1 year, 11 months ago
This patch series provides support for enabling Intel's Software Guard Extensions (SGX) feature in guest VM.

Giving the SGX support in QEMU had been merged. Intel SGX is a set of instructions that increases the security of application code and data, giving them more protection from disclosure or modification.
Developers can partition sensitive information into enclaves, which are areas of execution in memory with more security protection.

The typical flow looks below at very high level:

1. Calls virConnectGetDomainCapabilities API to domain capabilities that includes the following SGX information.

<feature>
   ...
   <sgx supported='yes'>
     <epc_size unit='KiB'>N</epc_size>
   </sgx>
   ...
 </feature>

2. User requests to start a guest calling virCreateXML() with SGX requirement. It does not support NUMA yet, since latest QEMU 6.2 release does not support NUMA.
It should contain

<devices>
    ...
    <memory model='sgx-epc'>
       <target>
           <size unit='KiB'>N</size>
       </target>
    </memory>
    ...
</devices>

Please note that SGX NUMA support will be implemented in future patches.

Haibin Huang (4):
  Define SGX capabilities structs
  Get SGX capabilities form QMP
  Convert QMP capabilities to domain capabilities
  conf: expose SGX feature in domain capabilities

Lin Yang (2):
  conf: Introduce SGX EPC element into device memory xml
  qemu: Add command-line to generate SGX EPC memory backend

 docs/formatdomain.rst                         |   9 +-
 docs/formatdomaincaps.rst                     |  26 ++++
 src/conf/domain_capabilities.c                |  33 ++++
 src/conf/domain_capabilities.h                |  13 ++
 src/conf/domain_conf.c                        |   6 +
 src/conf/domain_conf.h                        |   1 +
 src/conf/domain_validate.c                    |  16 ++
 src/conf/schemas/domaincaps.rng               |  22 ++-
 src/conf/schemas/domaincommon.rng             |   1 +
 src/libvirt_private.syms                      |   1 +
 src/qemu/qemu_alias.c                         |   6 +-
 src/qemu/qemu_capabilities.c                  | 145 ++++++++++++++++++
 src/qemu/qemu_capabilities.h                  |   4 +
 src/qemu/qemu_capspriv.h                      |   4 +
 src/qemu/qemu_command.c                       |  54 ++++++-
 src/qemu/qemu_domain.c                        |  38 +++--
 src/qemu/qemu_domain_address.c                |   6 +
 src/qemu/qemu_driver.c                        |   1 +
 src/qemu/qemu_monitor.c                       |  10 ++
 src/qemu/qemu_monitor.h                       |   3 +
 src/qemu/qemu_monitor_json.c                  | 104 ++++++++++++-
 src/qemu/qemu_monitor_json.h                  |   9 ++
 src/qemu/qemu_process.c                       |   2 +
 src/qemu/qemu_validate.c                      |   8 +
 src/security/security_apparmor.c              |   1 +
 src/security/security_dac.c                   |   2 +
 src/security/security_selinux.c               |   2 +
 tests/domaincapsdata/bhyve_basic.x86_64.xml   |   1 +
 tests/domaincapsdata/bhyve_fbuf.x86_64.xml    |   1 +
 tests/domaincapsdata/bhyve_uefi.x86_64.xml    |   1 +
 tests/domaincapsdata/empty.xml                |   1 +
 tests/domaincapsdata/libxl-xenfv.xml          |   1 +
 tests/domaincapsdata/libxl-xenpv.xml          |   1 +
 .../domaincapsdata/qemu_2.11.0-q35.x86_64.xml |   1 +
 .../domaincapsdata/qemu_2.11.0-tcg.x86_64.xml |   1 +
 tests/domaincapsdata/qemu_2.11.0.s390x.xml    |   1 +
 tests/domaincapsdata/qemu_2.11.0.x86_64.xml   |   1 +
 .../domaincapsdata/qemu_2.12.0-q35.x86_64.xml |   1 +
 .../domaincapsdata/qemu_2.12.0-tcg.x86_64.xml |   1 +
 .../qemu_2.12.0-virt.aarch64.xml              |   1 +
 tests/domaincapsdata/qemu_2.12.0.aarch64.xml  |   1 +
 tests/domaincapsdata/qemu_2.12.0.ppc64.xml    |   1 +
 tests/domaincapsdata/qemu_2.12.0.s390x.xml    |   1 +
 tests/domaincapsdata/qemu_2.12.0.x86_64.xml   |   1 +
 .../domaincapsdata/qemu_3.0.0-q35.x86_64.xml  |   1 +
 .../domaincapsdata/qemu_3.0.0-tcg.x86_64.xml  |   1 +
 tests/domaincapsdata/qemu_3.0.0.ppc64.xml     |   1 +
 tests/domaincapsdata/qemu_3.0.0.s390x.xml     |   1 +
 tests/domaincapsdata/qemu_3.0.0.x86_64.xml    |   1 +
 .../domaincapsdata/qemu_3.1.0-q35.x86_64.xml  |   1 +
 .../domaincapsdata/qemu_3.1.0-tcg.x86_64.xml  |   1 +
 tests/domaincapsdata/qemu_3.1.0.ppc64.xml     |   1 +
 tests/domaincapsdata/qemu_3.1.0.x86_64.xml    |   1 +
 .../domaincapsdata/qemu_4.0.0-q35.x86_64.xml  |   1 +
 .../domaincapsdata/qemu_4.0.0-tcg.x86_64.xml  |   1 +
 .../qemu_4.0.0-virt.aarch64.xml               |   1 +
 tests/domaincapsdata/qemu_4.0.0.aarch64.xml   |   1 +
 tests/domaincapsdata/qemu_4.0.0.ppc64.xml     |   1 +
 tests/domaincapsdata/qemu_4.0.0.s390x.xml     |   1 +
 tests/domaincapsdata/qemu_4.0.0.x86_64.xml    |   1 +
 .../domaincapsdata/qemu_4.1.0-q35.x86_64.xml  |   1 +
 .../domaincapsdata/qemu_4.1.0-tcg.x86_64.xml  |   1 +
 tests/domaincapsdata/qemu_4.1.0.x86_64.xml    |   1 +
 .../domaincapsdata/qemu_4.2.0-q35.x86_64.xml  |   1 +
 .../domaincapsdata/qemu_4.2.0-tcg.x86_64.xml  |   1 +
 .../qemu_4.2.0-virt.aarch64.xml               |   1 +
 tests/domaincapsdata/qemu_4.2.0.aarch64.xml   |   1 +
 tests/domaincapsdata/qemu_4.2.0.ppc64.xml     |   1 +
 tests/domaincapsdata/qemu_4.2.0.s390x.xml     |   1 +
 tests/domaincapsdata/qemu_4.2.0.x86_64.xml    |   1 +
 .../domaincapsdata/qemu_5.0.0-q35.x86_64.xml  |   1 +
 .../domaincapsdata/qemu_5.0.0-tcg.x86_64.xml  |   1 +
 .../qemu_5.0.0-virt.aarch64.xml               |   1 +
 tests/domaincapsdata/qemu_5.0.0.aarch64.xml   |   1 +
 tests/domaincapsdata/qemu_5.0.0.ppc64.xml     |   1 +
 tests/domaincapsdata/qemu_5.0.0.x86_64.xml    |   1 +
 .../domaincapsdata/qemu_5.1.0-q35.x86_64.xml  |   1 +
 .../domaincapsdata/qemu_5.1.0-tcg.x86_64.xml  |   1 +
 tests/domaincapsdata/qemu_5.1.0.sparc.xml     |   1 +
 tests/domaincapsdata/qemu_5.1.0.x86_64.xml    |   1 +
 .../domaincapsdata/qemu_5.2.0-q35.x86_64.xml  |   1 +
 .../domaincapsdata/qemu_5.2.0-tcg.x86_64.xml  |   1 +
 .../qemu_5.2.0-virt.aarch64.xml               |   1 +
 tests/domaincapsdata/qemu_5.2.0.aarch64.xml   |   1 +
 tests/domaincapsdata/qemu_5.2.0.ppc64.xml     |   1 +
 tests/domaincapsdata/qemu_5.2.0.s390x.xml     |   1 +
 tests/domaincapsdata/qemu_5.2.0.x86_64.xml    |   1 +
 .../domaincapsdata/qemu_6.0.0-q35.x86_64.xml  |   1 +
 .../domaincapsdata/qemu_6.0.0-tcg.x86_64.xml  |   1 +
 .../qemu_6.0.0-virt.aarch64.xml               |   1 +
 tests/domaincapsdata/qemu_6.0.0.aarch64.xml   |   1 +
 tests/domaincapsdata/qemu_6.0.0.s390x.xml     |   1 +
 tests/domaincapsdata/qemu_6.0.0.x86_64.xml    |   1 +
 .../domaincapsdata/qemu_6.1.0-q35.x86_64.xml  |   1 +
 .../domaincapsdata/qemu_6.1.0-tcg.x86_64.xml  |   1 +
 tests/domaincapsdata/qemu_6.1.0.x86_64.xml    |   1 +
 .../domaincapsdata/qemu_6.2.0-q35.x86_64.xml  |   4 +
 .../domaincapsdata/qemu_6.2.0-tcg.x86_64.xml  |   4 +
 .../qemu_6.2.0-virt.aarch64.xml               |   1 +
 tests/domaincapsdata/qemu_6.2.0.aarch64.xml   |   1 +
 tests/domaincapsdata/qemu_6.2.0.ppc64.xml     |   1 +
 tests/domaincapsdata/qemu_6.2.0.x86_64.xml    |   4 +
 .../domaincapsdata/qemu_7.0.0-q35.x86_64.xml  |   4 +
 .../domaincapsdata/qemu_7.0.0-tcg.x86_64.xml  |   4 +
 .../qemu_7.0.0-virt.aarch64.xml               |   1 +
 tests/domaincapsdata/qemu_7.0.0.aarch64.xml   |   1 +
 tests/domaincapsdata/qemu_7.0.0.ppc64.xml     |   1 +
 tests/domaincapsdata/qemu_7.0.0.x86_64.xml    |   4 +
 .../domaincapsdata/qemu_7.1.0-q35.x86_64.xml  |   1 +
 .../domaincapsdata/qemu_7.1.0-tcg.x86_64.xml  |   1 +
 tests/domaincapsdata/qemu_7.1.0.x86_64.xml    |   1 +
 .../caps_6.2.0.x86_64.replies                 |  22 ++-
 .../caps_6.2.0.x86_64.xml                     |   5 +
 .../caps_7.0.0.x86_64.replies                 |  22 ++-
 .../caps_7.0.0.x86_64.xml                     |   5 +
 .../caps_7.1.0.x86_64.replies                 |  21 ++-
 .../sgx-epc.x86_64-6.2.0.args                 |  37 +++++
 tests/qemuxml2argvdata/sgx-epc.xml            |  36 +++++
 tests/qemuxml2argvtest.c                      |   2 +
 .../sgx-epc.x86_64-6.2.0.xml                  |  52 +++++++
 tests/qemuxml2xmltest.c                       |   2 +
 121 files changed, 793 insertions(+), 40 deletions(-)
 create mode 100644 tests/qemuxml2argvdata/sgx-epc.x86_64-6.2.0.args
 create mode 100644 tests/qemuxml2argvdata/sgx-epc.xml
 create mode 100644 tests/qemuxml2xmloutdata/sgx-epc.x86_64-6.2.0.xml

-- 
2.17.1
Re: [libvirt][PATCH RESEND v12 0/6] Support query and use SGX
Posted by Michal Prívozník 1 year, 11 months ago
On 5/18/22 09:59, Haibin Huang wrote:
>

Overall, these patches work. I've raised couple of points and for your
convenience you can find reworked patches here:

https://gitlab.com/MichalPrivoznik/libvirt/-/commits/sgx_fixups/

Please see individual patches for comments.

Michal
Re: [libvirt][PATCH RESEND v12 0/6] Support query and use SGX
Posted by Michal Prívozník 1 year, 11 months ago
On 5/30/22 15:09, Michal Prívozník wrote:
> On 5/18/22 09:59, Haibin Huang wrote:
>>
> 
> Overall, these patches work. I've raised couple of points and for your
> convenience you can find reworked patches here:
> 
> https://gitlab.com/MichalPrivoznik/libvirt/-/commits/sgx_fixups/
> 
> Please see individual patches for comments.
> 
So, now that I've cleaned up the code I can start to test it, but
unfortunately, I don't have good news. It's not working. I've put the
following into my domain XML:

    <memory model='sgx-epc'>
      <target>
        <size unit='KiB'>16384</size>
      </target>
    </memory>

and this is the generated cmd line:

-machine
pc-i440fx-6.2,usb=off,dump-guest-core=off,sgx-epc.0.memdev=memepc0 \

-object
'{"qom-type":"memory-backend-memfd","id":"memepc0","hugetlb":true,"hugetlbsize":2097152,"share":true,"prealloc":true,"prealloc-threads":16,"size":16777216,"host-nodes":[0],"policy":"bind"}'
\

but if fails with:

2022-05-31T14:05:22.988793Z qemu-system-x86_64: Parameter
'sgx-epc.0.node' is missing

Now, there are two problems here:

1) obviously, wrong backend was picked. But this is easy to solve - just
move the  if (mem->model == VIR_DOMAIN_MEMORY_MODEL_SGX_EPC) case in
qemuBuildMemoryBackendProps() from the last patch before the memfd case.


2) apparently, .node attribute is required? Now, it's true that
initially my guest has 2 NUMA nodes defined, but even after I remove
those I still see the error. I believe I've raised this issue in one of
earlier reviews:

https://listman.redhat.com/archives/libvir-list/2022-February/228835.html

Please make sure that in v13 this is addressed (even at expense of not
working with QEMU-6.2.0 and requiring newer QEMU, if that's needed).
Wasting precious reviewer bandwidth does not help anybody.

Michal

Re: [libvirt][PATCH RESEND v12 0/6] Support query and use SGX
Posted by Yang, Lin A 1 year, 10 months ago
On 5/31/22, 7:29 AM, "Michal Prívozník" <mprivozn@redhat.com> wrote:
> So, now that I've cleaned up the code I can start to test it, but
> unfortunately, I don't have good news. It's not working. I've put the
> following into my domain XML:
>
>     <memory model='sgx-epc'>
>       <target>
>         <size unit='KiB'>16384</size>
>       </target>
>     </memory>
>
> and this is the generated cmd line:
>
> -machine
> pc-i440fx-6.2,usb=off,dump-guest-core=off,sgx-epc.0.memdev=memepc0 \
>
> -object
> '{"qom-type":"memory-backend-memfd","id":"memepc0","hugetlb":true,"hugetlbsize":2097152,"share":true,"prealloc":true,> "prealloc-threads":16,"size":16777216,"host-nodes":[0],"policy":"bind"}'
> \
>
> but if fails with:
>
> 2022-05-31T14:05:22.988793Z qemu-system-x86_64: Parameter
> 'sgx-epc.0.node' is missing
>
> Now, there are two problems here:
>
> 1) obviously, wrong backend was picked. But this is easy to solve - just
> move the  if (mem->model == VIR_DOMAIN_MEMORY_MODEL_SGX_EPC) case in
> qemuBuildMemoryBackendProps() from the last patch before the memfd case.
>
Thank you so much for resolving this issue.

> 2) apparently, .node attribute is required? Now, it's true that
> initially my guest has 2 NUMA nodes defined, but even after I remove
> those I still see the error. I believe I've raised this issue in one of
> earlier reviews:
>
> https://listman.redhat.com/archives/libvir-list/2022-February/228835.html
>
> Please make sure that in v13 this is addressed (even at expense of not
> working with QEMU-6.2.0 and requiring newer QEMU, if that's needed).
> Wasting precious reviewer bandwidth does not help anybody.

The .node attribute is required for qemu 7.0.0, but qemu 6.2.0 doesn’t need
It. Missing .node will fail with qemu 7.0.0, even only define one NUMA node.

Giving it’s probably impossible to detect whether .node is needed or not, our
proposal here is this patch only works with qemu 6.2.0, then support qemu 7.0.0
in another patch. It might give user a chance to choose different libvirt version,
or pick up some commits to work with different qemu version.

If it’s unnecessary, we can definitely only support 7.0.0 as you suggested.

Thanks,
Lin.



Re: [libvirt][PATCH RESEND v12 0/6] Support query and use SGX
Posted by Michal Prívozník 1 year, 10 months ago
On 6/2/22 02:52, Yang, Lin A wrote:
> On 5/31/22, 7:29 AM, "Michal Prívozník" <mprivozn@redhat.com> wrote:
> 

> 
>  
> 
>> 2) apparently, .node attribute is required? Now, it's true that
> 
>> initially my guest has 2 NUMA nodes defined, but even after I remove
> 
>> those I still see the error. I believe I've raised this issue in one of
> 
>> earlier reviews:
> 
>>
> 
>> https://listman.redhat.com/archives/libvir-list/2022-February/228835.html
> 
>>
> 
>> Please make sure that in v13 this is addressed (even at expense of not
> 
>> working with QEMU-6.2.0 and requiring newer QEMU, if that's needed).
> 
>> Wasting precious reviewer bandwidth does not help anybody.
> 
>  
> 
> The .node attribute is required for qemu 7.0.0, but qemu 6.2.0 doesn’t need
> 
> It. Missing .node will fail with qemu 7.0.0, even only define one NUMA node.
> 
>  
> 
> Giving it’s probably impossible to detect whether .node is needed or
> not, our
> 
> proposal here is this patch only works with qemu 6.2.0, then support
> qemu 7.0.0
> 
> in another patch. It might give user a chance to choose different
> libvirt version,
> 
> or pick up some commits to work with different qemu version.
> 
>  
> 
> If it’s unnecessary, we can definitely only support 7.0.0 as you suggested.

Worst case scenario we can do a version check. It's very suboptimal
because if somebody backports your patches in QEMU, libvirt will stop
working despite having the version check.

Therefore, I'm more inclined to just use the newest API and well, 6.2
won't work. In the long run - it's just one release that has the feature
but libvirt can't use it versus plenty of releases (that come after 7.0)
which have the feature and libvirt can use it. If we go this way then
we'll still need version check, but the other way round:

    if (qemuCaps->version < 7000000)
        virQEMUCapsClear(qemuCaps, QEMU_CAPS_SGX_EPC);

Or just exit early and don't even bother detecting SGX when version is
not 7.0.0:

    if (qemuCaps->version < 7000000)
        return 0;

This has the downside that when somebody backports QEMU patches to 6.2
to match the QAPI of 7.0 libvirt would still refuse to use the feature.
But one can argue that at that point the maintainer should also patch
libvirt (very trivial patch to remove these two lines of condition).

This is the reason we like QEMU to make features introspectable - we
could avoid all of this if we were able to detect .node attribute :-(

Now that I look at the output of query-qmp-schema command I see that 6.2
returns:

    {
      "name": "237",
      "members": [
        {
          "name": "sgx",
          "type": "bool"
        },
        {
          "name": "sgx1",
          "type": "bool"
        },
        {
          "name": "sgx2",
          "type": "bool"
        },
        {
          "name": "flc",
          "type": "bool"
        },
        {
          "name": "section-size",
          "type": "int"
        }
      ],
      "meta-type": "object"
    },

while 7.0 returns:

{
      "name": "237",
      "members": [
        {
          "name": "sgx",
          "type": "bool"
        },
        {
          "name": "sgx1",
          "type": "bool"
        },
        {
          "name": "sgx2",
          "type": "bool"
        },
        {
          "name": "flc",
          "type": "bool"
        },
        {
          "name": "section-size",
          "type": "int",
          "features": [
            "deprecated"
          ]
        },
        {
          "name": "sections",
          "type": "[454]"
        }
      ],
      "meta-type": "object"
    }


So maybe in the end libvirt CAN know the difference without having to do
any version check. We have a "dialect" of XPATH that we use to traverse
the QMP schema: look at the comment above virQEMUQAPISchemaPathGet().

Michal

Re: [libvirt][PATCH RESEND v12 0/6] Support query and use SGX
Posted by Yang, Lin A 1 year, 10 months ago
On 6/1/22, 11:37 PM, "Michal Prívozník" <mprivozn@redhat.com> wrote:

> Worst case scenario we can do a version check. It's very suboptimal
> because if somebody backports your patches in QEMU, libvirt will stop
> working despite having the version check.
>
> Therefore, I'm more inclined to just use the newest API and well, 6.2
> won't work. In the long run - it's just one release that has the feature
> but libvirt can't use it versus plenty of releases (that come after 7.0)
> which have the feature and libvirt can use it. If we go this way then
> we'll still need version check, but the other way round:
>
>     if (qemuCaps->version < 7000000)
>         virQEMUCapsClear(qemuCaps, QEMU_CAPS_SGX_EPC);
>
> Or just exit early and don't even bother detecting SGX when version is
> not 7.0.0:
>
>     if (qemuCaps->version < 7000000)
>         return 0;
>
> This has the downside that when somebody backports QEMU patches to 6.2
> to match the QAPI of 7.0 libvirt would still refuse to use the feature.
> But one can argue that at that point the maintainer should also patch
> libvirt (very trivial patch to remove these two lines of condition).
>
> This is the reason we like QEMU to make features introspectable - we
> could avoid all of this if we were able to detect .node attribute :-(
>
> Now that I look at the output of query-qmp-schema command I see that 6.2
> returns:
>
>     {
>       "name": "237",
>       "members": [
>         {
>           "name": "sgx",
>           "type": "bool"
>         },
>         {
>           "name": "sgx1",
>           "type": "bool"
>         },
>         {
>           "name": "sgx2",
>           "type": "bool"
>         },
>         {
>           "name": "flc",
>           "type": "bool"
>         },
>         {
>           "name": "section-size",
>           "type": "int"
>         }
>       ],
>       "meta-type": "object"
>     },
>
> while 7.0 returns:
>
> {
>       "name": "237",
>       "members": [
>         {
>           "name": "sgx",
>           "type": "bool"
>         },
>         {
>           "name": "sgx1",
>           "type": "bool"
>         },
>         {
>           "name": "sgx2",
>           "type": "bool"
>         },
>         {
>           "name": "flc",
>           "type": "bool"
>         },
>         {
>           "name": "section-size",
>           "type": "int",
>           "features": [
>             "deprecated"
>           ]
>         },
>         {
>           "name": "sections",
>           "type": "[454]"
>         }
>       ],
>       "meta-type": "object"
>     }
>
>
> So maybe in the end libvirt CAN know the difference without having to do
> any version check. We have a "dialect" of XPATH that we use to traverse
> the QMP schema: look at the comment above virQEMUQAPISchemaPathGet().

This is awesome! Thank you so much for this very informative explanation.
QEMU 7.0.0 provides more NUMA info in SGX part, so if we see "sections" in
QAPI schema, we can assume .node attribute is required.

Let me try this solution at first. We can support both QEMU 6.2.0 and 7.0.0 in
V13 patches if it is doable.

Thanks,
Lin.

Re: [libvirt][PATCH RESEND v12 0/6] Support query and use SGX
Posted by Yang, Lin A 1 year, 10 months ago
On 6/2/22, 11:28 AM, "Yang, Lin A" <lin.a.yang@intel.com> wrote:
> On 6/1/22, 11:37 PM, "Michal Prívozník" <mprivozn@redhat.com> wrote:
> > Worst case scenario we can do a version check. It's very suboptimal
> > because if somebody backports your patches in QEMU, libvirt will stop
> > working despite having the version check.
> >
> > Therefore, I'm more inclined to just use the newest API and well, 6.2
> > won't work. In the long run - it's just one release that has the feature
> > but libvirt can't use it versus plenty of releases (that come after 7.0)
> > which have the feature and libvirt can use it. If we go this way then
> > we'll still need version check, but the other way round:
> >
> >     if (qemuCaps->version < 7000000)
> >         virQEMUCapsClear(qemuCaps, QEMU_CAPS_SGX_EPC);
> >
> > Or just exit early and don't even bother detecting SGX when version is
> > not 7.0.0:
> >
> >     if (qemuCaps->version < 7000000)
> >         return 0;
> >
> > This has the downside that when somebody backports QEMU patches to 6.2
> > to match the QAPI of 7.0 libvirt would still refuse to use the feature.
> > But one can argue that at that point the maintainer should also patch
> > libvirt (very trivial patch to remove these two lines of condition).
> >
> > This is the reason we like QEMU to make features introspectable - we
> > could avoid all of this if we were able to detect .node attribute :-(
> >
> > Now that I look at the output of query-qmp-schema command I see that 6.2
> > returns:
> >
> >     {
> >       "name": "237",
> >       "members": [
> >         {
> >           "name": "sgx",
> >           "type": "bool"
> >         },
> >         {
> >           "name": "sgx1",
> >           "type": "bool"
> >         },
> >         {
> >           "name": "sgx2",
> >           "type": "bool"
> >         },
> >         {
> >           "name": "flc",
> >           "type": "bool"
> >         },
> >         {
> >           "name": "section-size",
> >           "type": "int"
> >         }
> >       ],
> >       "meta-type": "object"
> >     },
> >
> > while 7.0 returns:
> >
> > {
> >       "name": "237",
> >       "members": [
> >         {
> >           "name": "sgx",
> >           "type": "bool"
> >         },
> >         {
> >           "name": "sgx1",
> >           "type": "bool"
> >         },
> >         {
> >           "name": "sgx2",
> >           "type": "bool"
> >         },
> >         {
> >           "name": "flc",
> >           "type": "bool"
> >         },
> >         {
> >           "name": "section-size",
> >           "type": "int",
> >           "features": [
> >             "deprecated"
> >           ]
> >         },
> >         {
> >           "name": "sections",
> >           "type": "[454]"
> >         }
> >       ],
> >       "meta-type": "object"
> >     }
> >
> >
> > So maybe in the end libvirt CAN know the difference without having to do
> > any version check. We have a "dialect" of XPATH that we use to traverse
> > the QMP schema: look at the comment above virQEMUQAPISchemaPathGet().
>
> This is awesome! Thank you so much for this very informative explanation.
> QEMU 7.0.0 provides more NUMA info in SGX part, so if we see "sections" in
> QAPI schema, we can assume .node attribute is required.
>
> Let me try this solution at first. We can support both QEMU 6.2.0 and 7.0.0 in
> V13 patches if it is doable.

Sorry for multiple emails here.

Since these patches here have been review several times, and support 7.0.0 will bring
some new commits. Each update to new commit will require git rebase and resolve
conflict for old commits. Is it possible that we add the feature to compare QAPI schema
and detect .node, but only work with 6.2.0 in this patch and return error message for
7.0.0. After finishing this thread, we can start a new thread to support NUMA for qemu
7.0.0. Any preference?

Thanks,
Lin.

Re: [libvirt][PATCH RESEND v12 0/6] Support query and use SGX
Posted by Peter Krempa 1 year, 10 months ago
On Thu, Jun 02, 2022 at 22:49:15 +0000, Yang, Lin A wrote:
> On 6/2/22, 11:28 AM, "Yang, Lin A" <lin.a.yang@intel.com> wrote:
> > On 6/1/22, 11:37 PM, "Michal Prívozník" <mprivozn@redhat.com> wrote:

[...]

> > > So maybe in the end libvirt CAN know the difference without having to do
> > > any version check. We have a "dialect" of XPATH that we use to traverse
> > > the QMP schema: look at the comment above virQEMUQAPISchemaPathGet().
> >
> > This is awesome! Thank you so much for this very informative explanation.
> > QEMU 7.0.0 provides more NUMA info in SGX part, so if we see "sections" in
> > QAPI schema, we can assume .node attribute is required.
> >
> > Let me try this solution at first. We can support both QEMU 6.2.0 and 7.0.0 in
> > V13 patches if it is doable.
> 
> Sorry for multiple emails here.
> 
> Since these patches here have been review several times, and support 7.0.0 will bring
> some new commits. Each update to new commit will require git rebase and resolve
> conflict for old commits. Is it possible that we add the feature to compare QAPI schema
> and detect .node, but only work with 6.2.0 in this patch and return error message for
> 7.0.0. After finishing this thread, we can start a new thread to support NUMA for qemu
> 7.0.0. Any preference?

Usually we prefer if everything is done in one series, but obviously
that is not always possible.

In case you want to commit partially-incomplete patches you need to
ensure that they behave sanely for anyone attempting to use it.

This means mostly that if e.g. your code would work with qemu-6.2 and
would then break with qemu-7.0 it _must_ be kept disabled.

I didn't go through the conversation here, but from the above it seems
that you are discussing that there are two distinct ways how to detect
the presence of the feature in qemu between qemu-6.2 and qemu-7.0.

If that is true then I'd simply say it's strongly preferrable to just
drop the code for qemu-6.2 and just do the necessary steps to make it
work with qemu-7.0. Adding another bit of code just to make it work with
one extra version doesn't seem to be worth it unless you have a very
good justification, because it's more code that we'll have to maintain
in the end.
Re: [libvirt][PATCH RESEND v12 0/6] Support query and use SGX
Posted by Michal Prívozník 1 year, 10 months ago
On 6/3/22 09:36, Peter Krempa wrote:
> On Thu, Jun 02, 2022 at 22:49:15 +0000, Yang, Lin A wrote:
>> On 6/2/22, 11:28 AM, "Yang, Lin A" <lin.a.yang@intel.com> wrote:
>>> On 6/1/22, 11:37 PM, "Michal Prívozník" <mprivozn@redhat.com> wrote:
> 
> [...]
> 
>>>> So maybe in the end libvirt CAN know the difference without having to do
>>>> any version check. We have a "dialect" of XPATH that we use to traverse
>>>> the QMP schema: look at the comment above virQEMUQAPISchemaPathGet().
>>>
>>> This is awesome! Thank you so much for this very informative explanation.
>>> QEMU 7.0.0 provides more NUMA info in SGX part, so if we see "sections" in
>>> QAPI schema, we can assume .node attribute is required.
>>>
>>> Let me try this solution at first. We can support both QEMU 6.2.0 and 7.0.0 in
>>> V13 patches if it is doable.
>>
>> Sorry for multiple emails here.
>>
>> Since these patches here have been review several times, and support 7.0.0 will bring
>> some new commits. Each update to new commit will require git rebase and resolve
>> conflict for old commits. Is it possible that we add the feature to compare QAPI schema
>> and detect .node, but only work with 6.2.0 in this patch and return error message for
>> 7.0.0. After finishing this thread, we can start a new thread to support NUMA for qemu
>> 7.0.0. Any preference?
> 
> Usually we prefer if everything is done in one series, but obviously
> that is not always possible.
> 
> In case you want to commit partially-incomplete patches you need to
> ensure that they behave sanely for anyone attempting to use it.
> 
> This means mostly that if e.g. your code would work with qemu-6.2 and
> would then break with qemu-7.0 it _must_ be kept disabled.
> 
> I didn't go through the conversation here, but from the above it seems
> that you are discussing that there are two distinct ways how to detect
> the presence of the feature in qemu between qemu-6.2 and qemu-7.0.

More or less, yes. The feature works differently in 6.2. And the same
cmd line doesn't work in 7.0.

> 
> If that is true then I'd simply say it's strongly preferrable to just
> drop the code for qemu-6.2 and just do the necessary steps to make it
> work with qemu-7.0. Adding another bit of code just to make it work with
> one extra version doesn't seem to be worth it unless you have a very
> good justification, because it's more code that we'll have to maintain
> in the end.
> 

Exactly. This is what I was also suggesting, to just not bother with 6.2
at all and aim at 7.0. But it looks like Yang is aiming on supporting
6.2 too. This is going to create complicated implementation for a little
benefit IMO. But I'm not the one writing patches :-)

Michal

Re: [libvirt][PATCH RESEND v12 0/6] Support query and use SGX
Posted by Peter Krempa 1 year, 10 months ago
On Fri, Jun 03, 2022 at 16:43:30 +0200, Michal Prívozník wrote:
> On 6/3/22 09:36, Peter Krempa wrote:
> > On Thu, Jun 02, 2022 at 22:49:15 +0000, Yang, Lin A wrote:
> >> On 6/2/22, 11:28 AM, "Yang, Lin A" <lin.a.yang@intel.com> wrote:
> >>> On 6/1/22, 11:37 PM, "Michal Prívozník" <mprivozn@redhat.com> wrote:

[...]

> Exactly. This is what I was also suggesting, to just not bother with 6.2
> at all and aim at 7.0. But it looks like Yang is aiming on supporting
> 6.2 too. This is going to create complicated implementation for a little
> benefit IMO. But I'm not the one writing patches :-)

One thing to also consider is that we'll be the ones stuck with
maintaining them once they are upstream. So I strongly suggest that only
the new approach is implemented unless there are good reasons do support
6.2 too which should be properly justified or with a commitment to more
upstream contributions to offset it.