[v3] LIBVIRT: X86: TDX support

[PATCH v3 00/21] LIBVIRT: X86: TDX support

Posted by Zhenzhong Duan 5 months, 2 weeks ago

Hi,

This series brings libvirt the x86 TDX support.

* What's TDX?
TDX stands for Trust Domain Extensions which isolates VMs from
the virtual-machine manager (VMM)/hypervisor and any other software on
the platform.

This patchset extends libvirt to support TDX, with which one can start a TDX
guest from high level rather than running qemu directly.

* Misc
As QEMU use a software emulated way to reset guest which isn't supported by TDX
guest for security reason. We simulate reboot for TDX guest by kill and create a
new one in FakeReboot framework.

Complete code can be found at [1].

* Test
Tested with upstream qemu v10.0.0-1724-gf9a3def17b
shutdown/reboot/reset with virsh
shutdown/reboot trigger in guest
shutdown with on_poweroff=destroy/restart
reboot with on_reboot=destroy/restart

* Patch organization
- patch 1-4:   Some preparing work
- patch 5-6:   Support query of TDX capabilities
- patch 7-13:  Add TDX type to launchsecurity framework
- patch 14-19: Add reboot/reset support to TDX guest
- patch 20:    Add conf test dump/cases for '+inteltdx' variant
- patch 21:    Add docs

TODO:
- add reconnect logic in virsh command

[1] https://github.com/intel/libvirt-tdx/commits/tdx_for_upstream_v3

Thanks
Zhenzhong

Changelog:
v3:
- fix a hiden failure in qemuBuildTDXQGSCommandLine() (Peter Krempa)
- avoid the use of the ternary operator (Peter Krempa)
- add capability test dump before capability introduced (Peter Krempa)
- change tests version number from 11.0.0 to 10.1.0 (Peter Krempa)

v2:
- add capability and xmlconf test (Peter Krempa)

v1:
- s/virQEMUCapsKVMSupportsSecureGuestINTEL/virQEMUCapsKVMSupportsSecureGuestTDX (Daniel)
- make policy element optional and expose to QEMU directly (Daniel)
- s/qemuProcessSecFakeReboot/qemuProcessFakeRebootViaRecreate (Daniel)
- simplify QGS element schema by supporting only UNIX socket (Daniel)
- add new events VIR_DOMAIN_EVENT_[STOPPED|STARTED] for control plane (Daniel)
- s/quoteGenerationService/quoteGenerationSocket as QEMU
- add virsh reset support

rfcv4:
- add a check to tools/virt-host-validate-qemu.c (Daniel)
- remove check of q35 (Daniel)
- model 'SocktetAddress' QAPI in xml schema (Daniel)
- s/Quote-Generation-Service/quoteGenerationService/ (Daniel)
- define bits in tdx->policy and add validating logic (Daniel)
- presume QEMU choose split kernel irqchip for TDX guest by default (Daniel)
- utilize existing FakeReboot framework to do reboot for TDX guest (Daniel)
- drop patch11 'conf: Add support to keep same domid for hard reboot' (Daniel)
- add test in tests/ to validate parsing and formatting logic (Daniel)
- add doc in docs/formatdomain.rst (Daniel)
- add R-B

rfcv3:
- Change to generate qemu cmdline with -bios
- drop firmware auto match as -bios is used
- add a hard reboot method to reboot TDX guest

rfcv3: https://www.mail-archive.com/devel@lists.libvirt.org/msg00385.html

rfcv2:
- give up using qmp cmd and check TDX directly on host for TDX capabilities.
- use launchsecurity framework to support TDX
- use <os>.<loader> for general loader
- add auto firmware match feature for TDX

A example TDVF fimware description file 70-edk2-x86_64-tdx.json:
{
    "description": "UEFI firmware for x86_64, supporting Intel TDX",
    "interface-types": [
        "uefi"
    ],
    "mapping": {
        "device": "generic",
        "filename": "/usr/share/OVMF/OVMF_CODE-tdx.fd"
    },
    "targets": [
        {
            "architecture": "x86_64",
            "machines": [
                "pc-q35-*"
            ]
        }
    ],
    "features": [
        "intel-tdx",
        "verbose-dynamic"
    ],
    "tags": [

    ]
}

rfcv2: https://www.mail-archive.com/libvir-list@redhat.com/msg219378.html


Zhenzhong Duan (21):
  tools: Secure guest check for Intel in virt-host-validate
  qemu: Check if INTEL Trust Domain Extention support is enabled
  qemucapabilitiesdata: Document '+inteltdx' variant
  qemucapabilitiestest: Add data for the qemu-10.1.0 dev cycle on x86_64
    for the '+inteltdx' variant
  qemu: Add TDX capability
  conf: Expose TDX feature in domain capabilities
  conf: Add tdx as launch security type
  conf: Validate TDX launchSecurity element
    mrConfigId/mrOwner/mrOwnerConfig
  qemu: Add command line and validation for TDX type
  conf: Expose TDX type in domain launch security capability
  qemu: Force special parameters enabled for TDX guest
  conf: Add Intel TDX Quote Generation Service(QGS) support
  qemu: Add command line for TDX Quote Generation Service(QGS)
  qemu: Add FakeReboot support for TDX guest
  qemu: Support reboot command in guest
  qemu: Avoid duplicate FakeReboot for secure guest
  qemu: Send event VIR_DOMAIN_EVENT_[STOPPED|STARTED] during recreation
  qemu: Bypass sending VIR_DOMAIN_EVENT_RESUMED event when TD VM reboot
  qemu: Support domain reset command for TDX guest
  qemuxmlconftest: Add latest version of 'launch-security-tdx*' test
    data
  docs: domain: Add documentation for Intel TDX guest

 docs/formatdomain.rst                         |    63 +
 docs/formatdomaincaps.rst                     |     1 +
 examples/c/misc/event-test.c                  |     6 +
 include/libvirt/libvirt-domain.h              |     2 +
 src/conf/domain_capabilities.c                |     1 +
 src/conf/domain_capabilities.h                |     1 +
 src/conf/domain_conf.c                        |    82 +
 src/conf/domain_conf.h                        |    21 +
 src/conf/domain_validate.c                    |    11 +
 src/conf/schemas/domaincaps.rng               |     9 +
 src/conf/schemas/domaincommon.rng             |    41 +
 src/conf/virconftypes.h                       |     2 +
 src/qemu/qemu_capabilities.c                  |    38 +-
 src/qemu/qemu_capabilities.h                  |     1 +
 src/qemu/qemu_cgroup.c                        |     1 +
 src/qemu/qemu_command.c                       |    43 +
 src/qemu/qemu_domain.h                        |     1 +
 src/qemu/qemu_driver.c                        |    11 +-
 src/qemu/qemu_firmware.c                      |     1 +
 src/qemu/qemu_monitor.c                       |    34 +-
 src/qemu/qemu_monitor.h                       |     2 +-
 src/qemu/qemu_monitor_json.c                  |     6 +-
 src/qemu/qemu_namespace.c                     |     1 +
 src/qemu/qemu_process.c                       |   104 +-
 src/qemu/qemu_process.h                       |     2 +
 src/qemu/qemu_validate.c                      |    45 +
 src/security/security_dac.c                   |     2 +
 .../qemu_10.1.0-q35.x86_64+inteltdx.xml       |   783 +
 .../qemu_10.1.0-tcg.x86_64+inteltdx.xml       |  1830 +
 .../qemu_10.1.0.x86_64+inteltdx.xml           |   783 +
 tests/domaincapsmock.c                        |     3 +-
 tests/qemucapabilitiesdata/README.rst         |     5 +
 .../caps_10.1.0_x86_64+inteltdx.replies       | 44552 ++++++++++++++++
 .../caps_10.1.0_x86_64+inteltdx.xml           |  3585 ++
 .../caps.x86_64+inteltdx.xml                  |    29 +
 ...h-security-tdx.x86_64-latest+inteltdx.args |    44 +
 ...ch-security-tdx.x86_64-latest+inteltdx.xml |    74 +
 tests/qemuxmlconfdata/launch-security-tdx.xml |    27 +
 tests/qemuxmlconftest.c                       |     3 +
 tools/virsh-domain-event.c                    |     6 +-
 tools/virt-host-validate-common.c             |    31 +-
 tools/virt-host-validate-common.h             |     1 +
 42 files changed, 52273 insertions(+), 15 deletions(-)
 create mode 100644 tests/domaincapsdata/qemu_10.1.0-q35.x86_64+inteltdx.xml
 create mode 100644 tests/domaincapsdata/qemu_10.1.0-tcg.x86_64+inteltdx.xml
 create mode 100644 tests/domaincapsdata/qemu_10.1.0.x86_64+inteltdx.xml
 create mode 100644 tests/qemucapabilitiesdata/caps_10.1.0_x86_64+inteltdx.replies
 create mode 100644 tests/qemucapabilitiesdata/caps_10.1.0_x86_64+inteltdx.xml
 create mode 100644 tests/qemucaps2xmloutdata/caps.x86_64+inteltdx.xml
 create mode 100644 tests/qemuxmlconfdata/launch-security-tdx.x86_64-latest+inteltdx.args
 create mode 100644 tests/qemuxmlconfdata/launch-security-tdx.x86_64-latest+inteltdx.xml
 create mode 100644 tests/qemuxmlconfdata/launch-security-tdx.xml

-- 
2.34.1

Re: [PATCH v3 00/21] LIBVIRT: X86: TDX support

Posted by Daniel P. Berrangé via Devel 5 months, 1 week ago

On Mon, Jun 30, 2025 at 02:17:11PM +0800, Zhenzhong Duan wrote:
> Hi,
> 
> This series brings libvirt the x86 TDX support.
> 
> * What's TDX?
> TDX stands for Trust Domain Extensions which isolates VMs from
> the virtual-machine manager (VMM)/hypervisor and any other software on
> the platform.
> 
> This patchset extends libvirt to support TDX, with which one can start a TDX
> guest from high level rather than running qemu directly.
> 
> * Misc
> As QEMU use a software emulated way to reset guest which isn't supported by TDX
> guest for security reason. We simulate reboot for TDX guest by kill and create a
> new one in FakeReboot framework.
> 
> Complete code can be found at [1].
> 
> * Test
> Tested with upstream qemu v10.0.0-1724-gf9a3def17b
> shutdown/reboot/reset with virsh
> shutdown/reboot trigger in guest
> shutdown with on_poweroff=destroy/restart
> reboot with on_reboot=destroy/restart

FYI when I have a guest crash libvirt logs

  error : qemuMonitorJSONGuestPanicExtractInfo:638 : internal error: unknown panic info type 'tdx'

this is a new type of panic data structure from upstream QEMU
that libvirt needs to handle in some way



With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

RE: [PATCH v3 00/21] LIBVIRT: X86: TDX support

Posted by Duan, Zhenzhong 5 months, 1 week ago


>-----Original Message-----
>From: Daniel P. Berrangé <berrange@redhat.com>
>Subject: Re: [PATCH v3 00/21] LIBVIRT: X86: TDX support
>
>On Mon, Jun 30, 2025 at 02:17:11PM +0800, Zhenzhong Duan wrote:
>> Hi,
>>
>> This series brings libvirt the x86 TDX support.
>>
>> * What's TDX?
>> TDX stands for Trust Domain Extensions which isolates VMs from
>> the virtual-machine manager (VMM)/hypervisor and any other software on
>> the platform.
>>
>> This patchset extends libvirt to support TDX, with which one can start a TDX
>> guest from high level rather than running qemu directly.
>>
>> * Misc
>> As QEMU use a software emulated way to reset guest which isn't supported
>by TDX
>> guest for security reason. We simulate reboot for TDX guest by kill and
>create a
>> new one in FakeReboot framework.
>>
>> Complete code can be found at [1].
>>
>> * Test
>> Tested with upstream qemu v10.0.0-1724-gf9a3def17b
>> shutdown/reboot/reset with virsh
>> shutdown/reboot trigger in guest
>> shutdown with on_poweroff=destroy/restart
>> reboot with on_reboot=destroy/restart
>
>FYI when I have a guest crash libvirt logs
>
>  error : qemuMonitorJSONGuestPanicExtractInfo:638 : internal error:
>unknown panic info type 'tdx'
>
>this is a new type of panic data structure from upstream QEMU
>that libvirt needs to handle in some way

Oh yes, thanks, will add processing on libvirt side.

BRs,
Zhenzhong

Re: [PATCH v3 00/21] LIBVIRT: X86: TDX support

Posted by Daniel P. Berrangé via Devel 5 months, 1 week ago

On Wed, Jul 09, 2025 at 08:55:48AM +0000, Duan, Zhenzhong wrote:
> 
> 
> >-----Original Message-----
> >From: Daniel P. Berrangé <berrange@redhat.com>
> >Subject: Re: [PATCH v3 00/21] LIBVIRT: X86: TDX support
> >
> >On Mon, Jun 30, 2025 at 02:17:11PM +0800, Zhenzhong Duan wrote:
> >> Hi,
> >>
> >> This series brings libvirt the x86 TDX support.
> >>
> >> * What's TDX?
> >> TDX stands for Trust Domain Extensions which isolates VMs from
> >> the virtual-machine manager (VMM)/hypervisor and any other software on
> >> the platform.
> >>
> >> This patchset extends libvirt to support TDX, with which one can start a TDX
> >> guest from high level rather than running qemu directly.
> >>
> >> * Misc
> >> As QEMU use a software emulated way to reset guest which isn't supported
> >by TDX
> >> guest for security reason. We simulate reboot for TDX guest by kill and
> >create a
> >> new one in FakeReboot framework.
> >>
> >> Complete code can be found at [1].
> >>
> >> * Test
> >> Tested with upstream qemu v10.0.0-1724-gf9a3def17b
> >> shutdown/reboot/reset with virsh
> >> shutdown/reboot trigger in guest
> >> shutdown with on_poweroff=destroy/restart
> >> reboot with on_reboot=destroy/restart
> >
> >FYI when I have a guest crash libvirt logs
> >
> >  error : qemuMonitorJSONGuestPanicExtractInfo:638 : internal error:
> >unknown panic info type 'tdx'
> >
> >this is a new type of panic data structure from upstream QEMU
> >that libvirt needs to handle in some way
> 
> Oh yes, thanks, will add processing on libvirt side.

I think it'll be useful to get the crash "message" string into the
per-VM logfile at /var/log/libvirt/qemu/$GUEST.log

I wasted alot of time trying to understand why my guest was panicking
until realizing I had mistakenly enabled SEPT #VE. The 'message' string
in the panic event would have told me that immediately, but it was not
visible, so I think its important to expose this msg in the logs out of
the box.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

RE: [PATCH v3 00/21] LIBVIRT: X86: TDX support

Posted by Duan, Zhenzhong 5 months, 1 week ago


>-----Original Message-----
>From: Daniel P. Berrangé <berrange@redhat.com>
>Subject: Re: [PATCH v3 00/21] LIBVIRT: X86: TDX support
>
>On Wed, Jul 09, 2025 at 08:55:48AM +0000, Duan, Zhenzhong wrote:
>>
>>
>> >-----Original Message-----
>> >From: Daniel P. Berrangé <berrange@redhat.com>
>> >Subject: Re: [PATCH v3 00/21] LIBVIRT: X86: TDX support
>> >
>> >On Mon, Jun 30, 2025 at 02:17:11PM +0800, Zhenzhong Duan wrote:
>> >> Hi,
>> >>
>> >> This series brings libvirt the x86 TDX support.
>> >>
>> >> * What's TDX?
>> >> TDX stands for Trust Domain Extensions which isolates VMs from
>> >> the virtual-machine manager (VMM)/hypervisor and any other software
>on
>> >> the platform.
>> >>
>> >> This patchset extends libvirt to support TDX, with which one can start a
>TDX
>> >> guest from high level rather than running qemu directly.
>> >>
>> >> * Misc
>> >> As QEMU use a software emulated way to reset guest which isn't
>supported
>> >by TDX
>> >> guest for security reason. We simulate reboot for TDX guest by kill and
>> >create a
>> >> new one in FakeReboot framework.
>> >>
>> >> Complete code can be found at [1].
>> >>
>> >> * Test
>> >> Tested with upstream qemu v10.0.0-1724-gf9a3def17b
>> >> shutdown/reboot/reset with virsh
>> >> shutdown/reboot trigger in guest
>> >> shutdown with on_poweroff=destroy/restart
>> >> reboot with on_reboot=destroy/restart
>> >
>> >FYI when I have a guest crash libvirt logs
>> >
>> >  error : qemuMonitorJSONGuestPanicExtractInfo:638 : internal error:
>> >unknown panic info type 'tdx'
>> >
>> >this is a new type of panic data structure from upstream QEMU
>> >that libvirt needs to handle in some way
>>
>> Oh yes, thanks, will add processing on libvirt side.
>
>I think it'll be useful to get the crash "message" string into the
>per-VM logfile at /var/log/libvirt/qemu/$GUEST.log

OK. I will add TDX part in qemuMonitorJSONGuestPanicExtractInfo(),
then qemuProcessGuestPanicEventInfo() will be called to write to qemu log file.
Please fix me if I understand wrong.

>
>I wasted alot of time trying to understand why my guest was panicking
>until realizing I had mistakenly enabled SEPT #VE. The 'message' string
>in the panic event would have told me that immediately, but it was not
>visible, so I think its important to expose this msg in the logs out of
>the box.

Sure.

Thanks
Zhenzhong

Re: [PATCH v3 00/21] LIBVIRT: X86: TDX support

Posted by Daniel P. Berrangé via Devel 5 months, 1 week ago

On Wed, Jul 09, 2025 at 09:35:02AM +0000, Duan, Zhenzhong wrote:
> 
> 
> >-----Original Message-----
> >From: Daniel P. Berrangé <berrange@redhat.com>
> >Subject: Re: [PATCH v3 00/21] LIBVIRT: X86: TDX support
> >
> >On Wed, Jul 09, 2025 at 08:55:48AM +0000, Duan, Zhenzhong wrote:
> >>
> >>
> >> >-----Original Message-----
> >> >From: Daniel P. Berrangé <berrange@redhat.com>
> >> >Subject: Re: [PATCH v3 00/21] LIBVIRT: X86: TDX support
> >> >
> >> >On Mon, Jun 30, 2025 at 02:17:11PM +0800, Zhenzhong Duan wrote:
> >> >> Hi,
> >> >>
> >> >> This series brings libvirt the x86 TDX support.
> >> >>
> >> >> * What's TDX?
> >> >> TDX stands for Trust Domain Extensions which isolates VMs from
> >> >> the virtual-machine manager (VMM)/hypervisor and any other software
> >on
> >> >> the platform.
> >> >>
> >> >> This patchset extends libvirt to support TDX, with which one can start a
> >TDX
> >> >> guest from high level rather than running qemu directly.
> >> >>
> >> >> * Misc
> >> >> As QEMU use a software emulated way to reset guest which isn't
> >supported
> >> >by TDX
> >> >> guest for security reason. We simulate reboot for TDX guest by kill and
> >> >create a
> >> >> new one in FakeReboot framework.
> >> >>
> >> >> Complete code can be found at [1].
> >> >>
> >> >> * Test
> >> >> Tested with upstream qemu v10.0.0-1724-gf9a3def17b
> >> >> shutdown/reboot/reset with virsh
> >> >> shutdown/reboot trigger in guest
> >> >> shutdown with on_poweroff=destroy/restart
> >> >> reboot with on_reboot=destroy/restart
> >> >
> >> >FYI when I have a guest crash libvirt logs
> >> >
> >> >  error : qemuMonitorJSONGuestPanicExtractInfo:638 : internal error:
> >> >unknown panic info type 'tdx'
> >> >
> >> >this is a new type of panic data structure from upstream QEMU
> >> >that libvirt needs to handle in some way
> >>
> >> Oh yes, thanks, will add processing on libvirt side.
> >
> >I think it'll be useful to get the crash "message" string into the
> >per-VM logfile at /var/log/libvirt/qemu/$GUEST.log
> 
> OK. I will add TDX part in qemuMonitorJSONGuestPanicExtractInfo(),
> then qemuProcessGuestPanicEventInfo() will be called to write to qemu log file.

Looks like you probably don't need to change qemuProcessGuestPanicEventInfo
but instead update qemuMonitorGuestPanicEventInfoFormatMsg() to be able
to format the info for TDX.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|