[PATCH v4 00/33] Qemu SGX virtualization

Yang Zhong posted 33 patches 2 years, 9 months ago
Test checkpatch passed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20210719112136.57018-1-yang.zhong@intel.com
Maintainers: Eduardo Habkost <ehabkost@redhat.com>, Thomas Huth <thuth@redhat.com>, Peter Xu <peterx@redhat.com>, Richard Henderson <richard.henderson@linaro.org>, David Hildenbrand <david@redhat.com>, Igor Mammedov <imammedo@redhat.com>, Eric Blake <eblake@redhat.com>, Paolo Bonzini <pbonzini@redhat.com>, Alex Williamson <alex.williamson@redhat.com>, Ani Sinha <ani@anisinha.ca>, Markus Armbruster <armbru@redhat.com>, "Daniel P. Berrangé" <berrange@redhat.com>, Laurent Vivier <lvivier@redhat.com>, "Dr. David Alan Gilbert" <dgilbert@redhat.com>, Marcel Apfelbaum <marcel.apfelbaum@gmail.com>, Marcelo Tosatti <mtosatti@redhat.com>, "Michael S. Tsirkin" <mst@redhat.com>
There is a newer version of this series
backends/hostmem-epc.c                   | 118 ++++++++++
backends/meson.build                     |   1 +
configs/devices/i386-softmmu/default.mak |   1 +
docs/intel-sgx.txt                       | 167 ++++++++++++++
hmp-commands-info.hx                     |  15 ++
hw/i386/Kconfig                          |   5 +
hw/i386/acpi-build.c                     |  22 ++
hw/i386/fw_cfg.c                         |  10 +-
hw/i386/meson.build                      |   2 +
hw/i386/pc.c                             |  15 +-
hw/i386/pc_piix.c                        |   4 +
hw/i386/pc_q35.c                         |   3 +
hw/i386/sgx-epc.c                        | 265 +++++++++++++++++++++++
hw/i386/sgx-stub.c                       |  13 ++
hw/i386/sgx.c                            | 170 +++++++++++++++
hw/i386/x86.c                            |  29 +++
hw/vfio/common.c                         |   1 +
include/exec/memory.h                    |  15 +-
include/hw/i386/pc.h                     |  10 +
include/hw/i386/sgx-epc.h                |  68 ++++++
include/hw/i386/x86.h                    |   1 +
include/monitor/hmp.h                    |   1 +
include/qemu/bitops.h                    |   7 +
monitor/hmp-cmds.c                       |  32 +++
monitor/qmp-cmds.c                       |  19 ++
qapi/machine.json                        |  52 ++++-
qapi/misc.json                           |  61 ++++++
qapi/qom.json                            |  19 ++
qemu-options.hx                          |  10 +-
softmmu/memory.c                         |   5 +
softmmu/physmem.c                        |   3 +-
stubs/meson.build                        |   1 +
stubs/sgx-stub.c                         |  12 +
target/i386/cpu.c                        | 168 +++++++++++++-
target/i386/cpu.h                        |  16 ++
target/i386/kvm/kvm.c                    |  75 +++++++
target/i386/kvm/kvm_i386.h               |   2 +
target/i386/machine.c                    |  20 ++
tests/qtest/qmp-cmd-test.c               |   2 +
39 files changed, 1430 insertions(+), 10 deletions(-)
create mode 100644 backends/hostmem-epc.c
create mode 100644 docs/intel-sgx.txt
create mode 100644 hw/i386/sgx-epc.c
create mode 100644 hw/i386/sgx-stub.c
create mode 100644 hw/i386/sgx.c
create mode 100644 include/hw/i386/sgx-epc.h
create mode 100644 stubs/sgx-stub.c
[PATCH v4 00/33] Qemu SGX virtualization
Posted by Yang Zhong 2 years, 9 months ago
Since Sean Christopherson has left Intel and i am responsible for Qemu SGX
upstream work. His @intel.com address will be bouncing and his new email(
seanjc@google.com) is also in CC lists.

This series is Qemu SGX virtualization implementation rebased on latest
Qemu release. The numa support for SGX will be sent in another patchset
once this basic SGX patchset are merged.

You can find Qemu repo here:

    https://github.com/intel/qemu-sgx.git upstream

If you want to try SGX, you can directly install the linux release(at least 5.13.0-rc1+)
since kvm SGX has been merged into linux release.

    https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

To simplify, you'd better install linux on host and guest, which can support
SGX on host and guest kernel. And to me, use below reference command to boot
SGX guest:

    #qemu-system-x86_64 \
        ...... \
        -cpu host,+sgx-provisionkey \
        -object memory-backend-epc,id=mem1,size=64M,prealloc=on \
        -object memory-backend-epc,id=mem2,size=28M \
        -M sgx-epc.0.memdev=mem1,sgx-epc.1.memdev=mem2

Overview
========

Intel Software Guard eXtensions (SGX) is a set of instructions and mechanisms
for memory accesses in order to provide security accesses for sensitive
applications and data. SGX allows an application to use it's pariticular
address space as an *enclave*, which is a protected area provides confidentiality
and integrity even in the presence of privileged malware. Accesses to the
enclave memory area from any software not resident in the enclave are prevented,
including those from privileged software.

SGX virtaulization
==================

The KVM SGX creates one new misc device, sgx_vepc, and Qemu will open '/dev/sgx_vepc'
device node to mmap() host EPC memory to guest. The Qemu also adds 'sgx-epc' device
to expose EPC sections to guest through CPUID and ACPI table.  The Qemu SGX also
supports multiple virtual EPC sections to guest, we just put them together physically
contiguous for the sake of simplicity. The kernel SGX NUMA has been merged into Linux
tip tree, we will support this function in the next phase.

Although the current host SGX subsystem can not support SGX2 feature, the KVM/Qemu
implementation still expose this feature to guest. Guest SGX2 support doesn't have
interaction with host kernel SGX driver, the SGX guest can normally use those new
instructions.

As for SGX virtualization detailed infomation, please reference docs/intel-sgx.txt
docuement(patch 33).

Changelog:
=========

(Changelog here is for global changes, please see each patch's changelog for changes
made to specific patch.)

v3-->v4:
   - Rebased the sgx patches into latest Qemu release.
   - Moved sgx compound property setter/getter from MachineState to X86MachineState(Paolo).
   - Re-defined struct SgxEPC, removed 'id' property and added struct SgxEPCList for
     sgx-epc.0.{memdev}(Paolo).
   - Removed g_malloc0(), and changed the 'SGXEPCState *sgx_epc' to 'SGXEPCState sgx_epc'
     in struct PCMachineState(Paolo).
   - Changed the SGX compound property cmdline from sgx-epc.{memdev}.0 to
     sgx-epc.0.{memdev}(Paolo).
   - Removed the signature from the 'git format-patch' command(Jarkko).

v2-->v3:
   - Rebased the sgx patches into latest Qemu release.
   - Implemented the compound property for SGX, ref patch5, the command from '-sgx-epc'
     to '-M'(Paolo).
   - Moved the sgx common code from sgx-epc.c to sgx.c. The sgx-epc.c is
     only responsible for virtual epc device.
   - Removed the previous patch13(linux-headers: Add placeholder for KVM_CAP_SGX_ATTRIBUTE)
     because ehabkost@redhat.com updated Linux headers to 5.13-rc4 with commit 278f064e452.
   - Updated the patch1 because ram_flags were changed by David Hildenbra.
   - Added one patch24, which avoid reset operation caused by bios reset.
   - Added one patch25, which make prealloc property consistent with Qemu cmdline during VM
     reset.

v1-->v2:
   - Rebased the sgx patches into latest Qemu release.
   - Unified the "share" and "protected" arguments with ram_flags in the
     memory_region_init_ram_from_fd()(Paolo).
   - Added the new MemoryBackendEpcProperties and related documents(Eric Blake).
   - Changed the KVM_CAP_SGX_ATTRIBUTE from 195 to 196(Kai).
   - Changed the version and some grammar issues(Eric Blake).

Sean Christopherson (21):
  memory: Add RAM_PROTECTED flag to skip IOMMU mappings
  hostmem: Add hostmem-epc as a backend for SGX EPC
  i386: Add 'sgx-epc' device to expose EPC sections to guest
  vl: Add sgx compound properties to expose SGX EPC sections to guest
  i386: Add primary SGX CPUID and MSR defines
  i386: Add SGX CPUID leaf FEAT_SGX_12_0_EAX
  i386: Add SGX CPUID leaf FEAT_SGX_12_0_EBX
  i386: Add SGX CPUID leaf FEAT_SGX_12_1_EAX
  i386: Add get/set/migrate support for SGX_LEPUBKEYHASH MSRs
  i386: Add feature control MSR dependency when SGX is enabled
  i386: Update SGX CPUID info according to hardware/KVM/user input
  i386: kvm: Add support for exposing PROVISIONKEY to guest
  i386: Propagate SGX CPUID sub-leafs to KVM
  Adjust min CPUID level to 0x12 when SGX is enabled
  hw/i386/fw_cfg: Set SGX bits in feature control fw_cfg accordingly
  hw/i386/pc: Account for SGX EPC sections when calculating device
    memory
  i386/pc: Add e820 entry for SGX EPC section(s)
  i386: acpi: Add SGX EPC entry to ACPI tables
  q35: Add support for SGX EPC
  i440fx: Add support for SGX EPC
  doc: Add the SGX doc

Yang Zhong (12):
  qom: Add memory-backend-epc ObjectOptions support
  hostmem-epc: Add the reset interface for EPC backend reset
  sgx-epc: Add the reset interface for sgx-epc virt device
  sgx-epc: Avoid bios reset during sgx epc initialization
  hostmem-epc: Make prealloc consistent with qemu cmdline during reset
  qmp: Add query-sgx command
  hmp: Add 'info sgx' command
  i386: Add sgx_get_info() interface
  bitops: Support 32 and 64 bit mask macro
  qmp: Add the qmp_query_sgx_capabilities()
  Kconfig: Add CONFIG_SGX support
  sgx-epc: Add the fill_device_info() callback support

 backends/hostmem-epc.c                   | 118 ++++++++++
 backends/meson.build                     |   1 +
 configs/devices/i386-softmmu/default.mak |   1 +
 docs/intel-sgx.txt                       | 167 ++++++++++++++
 hmp-commands-info.hx                     |  15 ++
 hw/i386/Kconfig                          |   5 +
 hw/i386/acpi-build.c                     |  22 ++
 hw/i386/fw_cfg.c                         |  10 +-
 hw/i386/meson.build                      |   2 +
 hw/i386/pc.c                             |  15 +-
 hw/i386/pc_piix.c                        |   4 +
 hw/i386/pc_q35.c                         |   3 +
 hw/i386/sgx-epc.c                        | 265 +++++++++++++++++++++++
 hw/i386/sgx-stub.c                       |  13 ++
 hw/i386/sgx.c                            | 170 +++++++++++++++
 hw/i386/x86.c                            |  29 +++
 hw/vfio/common.c                         |   1 +
 include/exec/memory.h                    |  15 +-
 include/hw/i386/pc.h                     |  10 +
 include/hw/i386/sgx-epc.h                |  68 ++++++
 include/hw/i386/x86.h                    |   1 +
 include/monitor/hmp.h                    |   1 +
 include/qemu/bitops.h                    |   7 +
 monitor/hmp-cmds.c                       |  32 +++
 monitor/qmp-cmds.c                       |  19 ++
 qapi/machine.json                        |  52 ++++-
 qapi/misc.json                           |  61 ++++++
 qapi/qom.json                            |  19 ++
 qemu-options.hx                          |  10 +-
 softmmu/memory.c                         |   5 +
 softmmu/physmem.c                        |   3 +-
 stubs/meson.build                        |   1 +
 stubs/sgx-stub.c                         |  12 +
 target/i386/cpu.c                        | 168 +++++++++++++-
 target/i386/cpu.h                        |  16 ++
 target/i386/kvm/kvm.c                    |  75 +++++++
 target/i386/kvm/kvm_i386.h               |   2 +
 target/i386/machine.c                    |  20 ++
 tests/qtest/qmp-cmd-test.c               |   2 +
 39 files changed, 1430 insertions(+), 10 deletions(-)
 create mode 100644 backends/hostmem-epc.c
 create mode 100644 docs/intel-sgx.txt
 create mode 100644 hw/i386/sgx-epc.c
 create mode 100644 hw/i386/sgx-stub.c
 create mode 100644 hw/i386/sgx.c
 create mode 100644 include/hw/i386/sgx-epc.h
 create mode 100644 stubs/sgx-stub.c


Re: [PATCH v4 00/33] Qemu SGX virtualization
Posted by Paolo Bonzini 2 years, 9 months ago
On 19/07/21 13:21, Yang Zhong wrote:
> Since Sean Christopherson has left Intel and i am responsible for Qemu SGX
> upstream work. His @intel.com address will be bouncing and his new email(
> seanjc@google.com) is also in CC lists.
> 
> This series is Qemu SGX virtualization implementation rebased on latest
> Qemu release. The numa support for SGX will be sent in another patchset
> once this basic SGX patchset are merged.
> 
> You can find Qemu repo here:
> 
>      https://github.com/intel/qemu-sgx.git upstream
> 
> If you want to try SGX, you can directly install the linux release(at least 5.13.0-rc1+)
> since kvm SGX has been merged into linux release.
> 
>      https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
> 
> To simplify, you'd better install linux on host and guest, which can support
> SGX on host and guest kernel. And to me, use below reference command to boot
> SGX guest:
> 
>      #qemu-system-x86_64 \
>          ...... \
>          -cpu host,+sgx-provisionkey \
>          -object memory-backend-epc,id=mem1,size=64M,prealloc=on \
>          -object memory-backend-epc,id=mem2,size=28M \
>          -M sgx-epc.0.memdev=mem1,sgx-epc.1.memdev=mem2
> 
> Overview
> ========
> 
> Intel Software Guard eXtensions (SGX) is a set of instructions and mechanisms
> for memory accesses in order to provide security accesses for sensitive
> applications and data. SGX allows an application to use it's pariticular
> address space as an *enclave*, which is a protected area provides confidentiality
> and integrity even in the presence of privileged malware. Accesses to the
> enclave memory area from any software not resident in the enclave are prevented,
> including those from privileged software.
> 
> SGX virtaulization
> ==================
> 
> The KVM SGX creates one new misc device, sgx_vepc, and Qemu will open '/dev/sgx_vepc'
> device node to mmap() host EPC memory to guest. The Qemu also adds 'sgx-epc' device
> to expose EPC sections to guest through CPUID and ACPI table.  The Qemu SGX also
> supports multiple virtual EPC sections to guest, we just put them together physically
> contiguous for the sake of simplicity. The kernel SGX NUMA has been merged into Linux
> tip tree, we will support this function in the next phase.
> 
> Although the current host SGX subsystem can not support SGX2 feature, the KVM/Qemu
> implementation still expose this feature to guest. Guest SGX2 support doesn't have
> interaction with host kernel SGX driver, the SGX guest can normally use those new
> instructions.
> 
> As for SGX virtualization detailed infomation, please reference docs/intel-sgx.txt
> docuement(patch 33).
> 
> Changelog:
> =========
> 
> (Changelog here is for global changes, please see each patch's changelog for changes
> made to specific patch.)
> 
> v3-->v4:
>     - Rebased the sgx patches into latest Qemu release.
>     - Moved sgx compound property setter/getter from MachineState to X86MachineState(Paolo).
>     - Re-defined struct SgxEPC, removed 'id' property and added struct SgxEPCList for
>       sgx-epc.0.{memdev}(Paolo).
>     - Removed g_malloc0(), and changed the 'SGXEPCState *sgx_epc' to 'SGXEPCState sgx_epc'
>       in struct PCMachineState(Paolo).
>     - Changed the SGX compound property cmdline from sgx-epc.{memdev}.0 to
>       sgx-epc.0.{memdev}(Paolo).
>     - Removed the signature from the 'git format-patch' command(Jarkko).
> 
> v2-->v3:
>     - Rebased the sgx patches into latest Qemu release.
>     - Implemented the compound property for SGX, ref patch5, the command from '-sgx-epc'
>       to '-M'(Paolo).
>     - Moved the sgx common code from sgx-epc.c to sgx.c. The sgx-epc.c is
>       only responsible for virtual epc device.
>     - Removed the previous patch13(linux-headers: Add placeholder for KVM_CAP_SGX_ATTRIBUTE)
>       because ehabkost@redhat.com updated Linux headers to 5.13-rc4 with commit 278f064e452.
>     - Updated the patch1 because ram_flags were changed by David Hildenbra.
>     - Added one patch24, which avoid reset operation caused by bios reset.
>     - Added one patch25, which make prealloc property consistent with Qemu cmdline during VM
>       reset.
> 
> v1-->v2:
>     - Rebased the sgx patches into latest Qemu release.
>     - Unified the "share" and "protected" arguments with ram_flags in the
>       memory_region_init_ram_from_fd()(Paolo).
>     - Added the new MemoryBackendEpcProperties and related documents(Eric Blake).
>     - Changed the KVM_CAP_SGX_ATTRIBUTE from 195 to 196(Kai).
>     - Changed the version and some grammar issues(Eric Blake).

Looks good, I will queue it for 6.2.

Thanks for your patience with the compound machine properties support.

Paolo

> Sean Christopherson (21):
>    memory: Add RAM_PROTECTED flag to skip IOMMU mappings
>    hostmem: Add hostmem-epc as a backend for SGX EPC
>    i386: Add 'sgx-epc' device to expose EPC sections to guest
>    vl: Add sgx compound properties to expose SGX EPC sections to guest
>    i386: Add primary SGX CPUID and MSR defines
>    i386: Add SGX CPUID leaf FEAT_SGX_12_0_EAX
>    i386: Add SGX CPUID leaf FEAT_SGX_12_0_EBX
>    i386: Add SGX CPUID leaf FEAT_SGX_12_1_EAX
>    i386: Add get/set/migrate support for SGX_LEPUBKEYHASH MSRs
>    i386: Add feature control MSR dependency when SGX is enabled
>    i386: Update SGX CPUID info according to hardware/KVM/user input
>    i386: kvm: Add support for exposing PROVISIONKEY to guest
>    i386: Propagate SGX CPUID sub-leafs to KVM
>    Adjust min CPUID level to 0x12 when SGX is enabled
>    hw/i386/fw_cfg: Set SGX bits in feature control fw_cfg accordingly
>    hw/i386/pc: Account for SGX EPC sections when calculating device
>      memory
>    i386/pc: Add e820 entry for SGX EPC section(s)
>    i386: acpi: Add SGX EPC entry to ACPI tables
>    q35: Add support for SGX EPC
>    i440fx: Add support for SGX EPC
>    doc: Add the SGX doc
> 
> Yang Zhong (12):
>    qom: Add memory-backend-epc ObjectOptions support
>    hostmem-epc: Add the reset interface for EPC backend reset
>    sgx-epc: Add the reset interface for sgx-epc virt device
>    sgx-epc: Avoid bios reset during sgx epc initialization
>    hostmem-epc: Make prealloc consistent with qemu cmdline during reset
>    qmp: Add query-sgx command
>    hmp: Add 'info sgx' command
>    i386: Add sgx_get_info() interface
>    bitops: Support 32 and 64 bit mask macro
>    qmp: Add the qmp_query_sgx_capabilities()
>    Kconfig: Add CONFIG_SGX support
>    sgx-epc: Add the fill_device_info() callback support
> 
>   backends/hostmem-epc.c                   | 118 ++++++++++
>   backends/meson.build                     |   1 +
>   configs/devices/i386-softmmu/default.mak |   1 +
>   docs/intel-sgx.txt                       | 167 ++++++++++++++
>   hmp-commands-info.hx                     |  15 ++
>   hw/i386/Kconfig                          |   5 +
>   hw/i386/acpi-build.c                     |  22 ++
>   hw/i386/fw_cfg.c                         |  10 +-
>   hw/i386/meson.build                      |   2 +
>   hw/i386/pc.c                             |  15 +-
>   hw/i386/pc_piix.c                        |   4 +
>   hw/i386/pc_q35.c                         |   3 +
>   hw/i386/sgx-epc.c                        | 265 +++++++++++++++++++++++
>   hw/i386/sgx-stub.c                       |  13 ++
>   hw/i386/sgx.c                            | 170 +++++++++++++++
>   hw/i386/x86.c                            |  29 +++
>   hw/vfio/common.c                         |   1 +
>   include/exec/memory.h                    |  15 +-
>   include/hw/i386/pc.h                     |  10 +
>   include/hw/i386/sgx-epc.h                |  68 ++++++
>   include/hw/i386/x86.h                    |   1 +
>   include/monitor/hmp.h                    |   1 +
>   include/qemu/bitops.h                    |   7 +
>   monitor/hmp-cmds.c                       |  32 +++
>   monitor/qmp-cmds.c                       |  19 ++
>   qapi/machine.json                        |  52 ++++-
>   qapi/misc.json                           |  61 ++++++
>   qapi/qom.json                            |  19 ++
>   qemu-options.hx                          |  10 +-
>   softmmu/memory.c                         |   5 +
>   softmmu/physmem.c                        |   3 +-
>   stubs/meson.build                        |   1 +
>   stubs/sgx-stub.c                         |  12 +
>   target/i386/cpu.c                        | 168 +++++++++++++-
>   target/i386/cpu.h                        |  16 ++
>   target/i386/kvm/kvm.c                    |  75 +++++++
>   target/i386/kvm/kvm_i386.h               |   2 +
>   target/i386/machine.c                    |  20 ++
>   tests/qtest/qmp-cmd-test.c               |   2 +
>   39 files changed, 1430 insertions(+), 10 deletions(-)
>   create mode 100644 backends/hostmem-epc.c
>   create mode 100644 docs/intel-sgx.txt
>   create mode 100644 hw/i386/sgx-epc.c
>   create mode 100644 hw/i386/sgx-stub.c
>   create mode 100644 hw/i386/sgx.c
>   create mode 100644 include/hw/i386/sgx-epc.h
>   create mode 100644 stubs/sgx-stub.c
> 
> 


Re: [PATCH v4 00/33] Qemu SGX virtualization
Posted by Yang Zhong 2 years, 9 months ago
On Wed, Jul 28, 2021 at 05:57:51PM +0200, Paolo Bonzini wrote:
> On 19/07/21 13:21, Yang Zhong wrote:
> >Since Sean Christopherson has left Intel and i am responsible for Qemu SGX
> >upstream work. His @intel.com address will be bouncing and his new email(
> >seanjc@google.com) is also in CC lists.
> >
> >This series is Qemu SGX virtualization implementation rebased on latest
> >Qemu release. The numa support for SGX will be sent in another patchset
> >once this basic SGX patchset are merged.
> >
> >You can find Qemu repo here:
> >
> >     https://github.com/intel/qemu-sgx.git upstream
> >
> >If you want to try SGX, you can directly install the linux release(at least 5.13.0-rc1+)
> >since kvm SGX has been merged into linux release.
> >
> >     https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
> >
> >To simplify, you'd better install linux on host and guest, which can support
> >SGX on host and guest kernel. And to me, use below reference command to boot
> >SGX guest:
> >
> >     #qemu-system-x86_64 \
> >         ...... \
> >         -cpu host,+sgx-provisionkey \
> >         -object memory-backend-epc,id=mem1,size=64M,prealloc=on \
> >         -object memory-backend-epc,id=mem2,size=28M \
> >         -M sgx-epc.0.memdev=mem1,sgx-epc.1.memdev=mem2
> >
> >Overview
> >========
> >
> >Intel Software Guard eXtensions (SGX) is a set of instructions and mechanisms
> >for memory accesses in order to provide security accesses for sensitive
> >applications and data. SGX allows an application to use it's pariticular
> >address space as an *enclave*, which is a protected area provides confidentiality
> >and integrity even in the presence of privileged malware. Accesses to the
> >enclave memory area from any software not resident in the enclave are prevented,
> >including those from privileged software.
> >
> >SGX virtaulization
> >==================
> >
> >The KVM SGX creates one new misc device, sgx_vepc, and Qemu will open '/dev/sgx_vepc'
> >device node to mmap() host EPC memory to guest. The Qemu also adds 'sgx-epc' device
> >to expose EPC sections to guest through CPUID and ACPI table.  The Qemu SGX also
> >supports multiple virtual EPC sections to guest, we just put them together physically
> >contiguous for the sake of simplicity. The kernel SGX NUMA has been merged into Linux
> >tip tree, we will support this function in the next phase.
> >
> >Although the current host SGX subsystem can not support SGX2 feature, the KVM/Qemu
> >implementation still expose this feature to guest. Guest SGX2 support doesn't have
> >interaction with host kernel SGX driver, the SGX guest can normally use those new
> >instructions.
> >
> >As for SGX virtualization detailed infomation, please reference docs/intel-sgx.txt
> >docuement(patch 33).
> >
> >Changelog:
> >=========
> >
> >(Changelog here is for global changes, please see each patch's changelog for changes
> >made to specific patch.)
> >
> >v3-->v4:
> >    - Rebased the sgx patches into latest Qemu release.
> >    - Moved sgx compound property setter/getter from MachineState to X86MachineState(Paolo).
> >    - Re-defined struct SgxEPC, removed 'id' property and added struct SgxEPCList for
> >      sgx-epc.0.{memdev}(Paolo).
> >    - Removed g_malloc0(), and changed the 'SGXEPCState *sgx_epc' to 'SGXEPCState sgx_epc'
> >      in struct PCMachineState(Paolo).
> >    - Changed the SGX compound property cmdline from sgx-epc.{memdev}.0 to
> >      sgx-epc.0.{memdev}(Paolo).
> >    - Removed the signature from the 'git format-patch' command(Jarkko).
> >
> >v2-->v3:
> >    - Rebased the sgx patches into latest Qemu release.
> >    - Implemented the compound property for SGX, ref patch5, the command from '-sgx-epc'
> >      to '-M'(Paolo).
> >    - Moved the sgx common code from sgx-epc.c to sgx.c. The sgx-epc.c is
> >      only responsible for virtual epc device.
> >    - Removed the previous patch13(linux-headers: Add placeholder for KVM_CAP_SGX_ATTRIBUTE)
> >      because ehabkost@redhat.com updated Linux headers to 5.13-rc4 with commit 278f064e452.
> >    - Updated the patch1 because ram_flags were changed by David Hildenbra.
> >    - Added one patch24, which avoid reset operation caused by bios reset.
> >    - Added one patch25, which make prealloc property consistent with Qemu cmdline during VM
> >      reset.
> >
> >v1-->v2:
> >    - Rebased the sgx patches into latest Qemu release.
> >    - Unified the "share" and "protected" arguments with ram_flags in the
> >      memory_region_init_ram_from_fd()(Paolo).
> >    - Added the new MemoryBackendEpcProperties and related documents(Eric Blake).
> >    - Changed the KVM_CAP_SGX_ATTRIBUTE from 195 to 196(Kai).
> >    - Changed the version and some grammar issues(Eric Blake).
> 
> Looks good, I will queue it for 6.2.
> 
> Thanks for your patience with the compound machine properties support.
>

  Paolo, thanks! Once this patchset is merged, i will send SGX NUMA patchset, thanks!

  Regards,

  Yang


 
> Paolo
> 
> >Sean Christopherson (21):
> >   memory: Add RAM_PROTECTED flag to skip IOMMU mappings
> >   hostmem: Add hostmem-epc as a backend for SGX EPC
> >   i386: Add 'sgx-epc' device to expose EPC sections to guest
> >   vl: Add sgx compound properties to expose SGX EPC sections to guest
> >   i386: Add primary SGX CPUID and MSR defines
> >   i386: Add SGX CPUID leaf FEAT_SGX_12_0_EAX
> >   i386: Add SGX CPUID leaf FEAT_SGX_12_0_EBX
> >   i386: Add SGX CPUID leaf FEAT_SGX_12_1_EAX
> >   i386: Add get/set/migrate support for SGX_LEPUBKEYHASH MSRs
> >   i386: Add feature control MSR dependency when SGX is enabled
> >   i386: Update SGX CPUID info according to hardware/KVM/user input
> >   i386: kvm: Add support for exposing PROVISIONKEY to guest
> >   i386: Propagate SGX CPUID sub-leafs to KVM
> >   Adjust min CPUID level to 0x12 when SGX is enabled
> >   hw/i386/fw_cfg: Set SGX bits in feature control fw_cfg accordingly
> >   hw/i386/pc: Account for SGX EPC sections when calculating device
> >     memory
> >   i386/pc: Add e820 entry for SGX EPC section(s)
> >   i386: acpi: Add SGX EPC entry to ACPI tables
> >   q35: Add support for SGX EPC
> >   i440fx: Add support for SGX EPC
> >   doc: Add the SGX doc
> >
> >Yang Zhong (12):
> >   qom: Add memory-backend-epc ObjectOptions support
> >   hostmem-epc: Add the reset interface for EPC backend reset
> >   sgx-epc: Add the reset interface for sgx-epc virt device
> >   sgx-epc: Avoid bios reset during sgx epc initialization
> >   hostmem-epc: Make prealloc consistent with qemu cmdline during reset
> >   qmp: Add query-sgx command
> >   hmp: Add 'info sgx' command
> >   i386: Add sgx_get_info() interface
> >   bitops: Support 32 and 64 bit mask macro
> >   qmp: Add the qmp_query_sgx_capabilities()
> >   Kconfig: Add CONFIG_SGX support
> >   sgx-epc: Add the fill_device_info() callback support
> >
> >  backends/hostmem-epc.c                   | 118 ++++++++++
> >  backends/meson.build                     |   1 +
> >  configs/devices/i386-softmmu/default.mak |   1 +
> >  docs/intel-sgx.txt                       | 167 ++++++++++++++
> >  hmp-commands-info.hx                     |  15 ++
> >  hw/i386/Kconfig                          |   5 +
> >  hw/i386/acpi-build.c                     |  22 ++
> >  hw/i386/fw_cfg.c                         |  10 +-
> >  hw/i386/meson.build                      |   2 +
> >  hw/i386/pc.c                             |  15 +-
> >  hw/i386/pc_piix.c                        |   4 +
> >  hw/i386/pc_q35.c                         |   3 +
> >  hw/i386/sgx-epc.c                        | 265 +++++++++++++++++++++++
> >  hw/i386/sgx-stub.c                       |  13 ++
> >  hw/i386/sgx.c                            | 170 +++++++++++++++
> >  hw/i386/x86.c                            |  29 +++
> >  hw/vfio/common.c                         |   1 +
> >  include/exec/memory.h                    |  15 +-
> >  include/hw/i386/pc.h                     |  10 +
> >  include/hw/i386/sgx-epc.h                |  68 ++++++
> >  include/hw/i386/x86.h                    |   1 +
> >  include/monitor/hmp.h                    |   1 +
> >  include/qemu/bitops.h                    |   7 +
> >  monitor/hmp-cmds.c                       |  32 +++
> >  monitor/qmp-cmds.c                       |  19 ++
> >  qapi/machine.json                        |  52 ++++-
> >  qapi/misc.json                           |  61 ++++++
> >  qapi/qom.json                            |  19 ++
> >  qemu-options.hx                          |  10 +-
> >  softmmu/memory.c                         |   5 +
> >  softmmu/physmem.c                        |   3 +-
> >  stubs/meson.build                        |   1 +
> >  stubs/sgx-stub.c                         |  12 +
> >  target/i386/cpu.c                        | 168 +++++++++++++-
> >  target/i386/cpu.h                        |  16 ++
> >  target/i386/kvm/kvm.c                    |  75 +++++++
> >  target/i386/kvm/kvm_i386.h               |   2 +
> >  target/i386/machine.c                    |  20 ++
> >  tests/qtest/qmp-cmd-test.c               |   2 +
> >  39 files changed, 1430 insertions(+), 10 deletions(-)
> >  create mode 100644 backends/hostmem-epc.c
> >  create mode 100644 docs/intel-sgx.txt
> >  create mode 100644 hw/i386/sgx-epc.c
> >  create mode 100644 hw/i386/sgx-stub.c
> >  create mode 100644 hw/i386/sgx.c
> >  create mode 100644 include/hw/i386/sgx-epc.h
> >  create mode 100644 stubs/sgx-stub.c
> >
> >

Re: [PATCH v4 00/33] Qemu SGX virtualization
Posted by Paolo Bonzini 2 years, 7 months ago
Hi,

the monitor patches did not pass the test-hmp qtest, and also they
should be in target/i386/monitor.c (see other commands that were
implemented there for SEV).  However, I've sent a pull request with
the rest.

Thanks,

Paolo

On Mon, Jul 19, 2021 at 1:27 PM Yang Zhong <yang.zhong@intel.com> wrote:
>
> Since Sean Christopherson has left Intel and i am responsible for Qemu SGX
> upstream work. His @intel.com address will be bouncing and his new email(
> seanjc@google.com) is also in CC lists.
>
> This series is Qemu SGX virtualization implementation rebased on latest
> Qemu release. The numa support for SGX will be sent in another patchset
> once this basic SGX patchset are merged.
>
> You can find Qemu repo here:
>
>     https://github.com/intel/qemu-sgx.git upstream
>
> If you want to try SGX, you can directly install the linux release(at least 5.13.0-rc1+)
> since kvm SGX has been merged into linux release.
>
>     https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
>
> To simplify, you'd better install linux on host and guest, which can support
> SGX on host and guest kernel. And to me, use below reference command to boot
> SGX guest:
>
>     #qemu-system-x86_64 \
>         ...... \
>         -cpu host,+sgx-provisionkey \
>         -object memory-backend-epc,id=mem1,size=64M,prealloc=on \
>         -object memory-backend-epc,id=mem2,size=28M \
>         -M sgx-epc.0.memdev=mem1,sgx-epc.1.memdev=mem2
>
> Overview
> ========
>
> Intel Software Guard eXtensions (SGX) is a set of instructions and mechanisms
> for memory accesses in order to provide security accesses for sensitive
> applications and data. SGX allows an application to use it's pariticular
> address space as an *enclave*, which is a protected area provides confidentiality
> and integrity even in the presence of privileged malware. Accesses to the
> enclave memory area from any software not resident in the enclave are prevented,
> including those from privileged software.
>
> SGX virtaulization
> ==================
>
> The KVM SGX creates one new misc device, sgx_vepc, and Qemu will open '/dev/sgx_vepc'
> device node to mmap() host EPC memory to guest. The Qemu also adds 'sgx-epc' device
> to expose EPC sections to guest through CPUID and ACPI table.  The Qemu SGX also
> supports multiple virtual EPC sections to guest, we just put them together physically
> contiguous for the sake of simplicity. The kernel SGX NUMA has been merged into Linux
> tip tree, we will support this function in the next phase.
>
> Although the current host SGX subsystem can not support SGX2 feature, the KVM/Qemu
> implementation still expose this feature to guest. Guest SGX2 support doesn't have
> interaction with host kernel SGX driver, the SGX guest can normally use those new
> instructions.
>
> As for SGX virtualization detailed infomation, please reference docs/intel-sgx.txt
> docuement(patch 33).
>
> Changelog:
> =========
>
> (Changelog here is for global changes, please see each patch's changelog for changes
> made to specific patch.)
>
> v3-->v4:
>    - Rebased the sgx patches into latest Qemu release.
>    - Moved sgx compound property setter/getter from MachineState to X86MachineState(Paolo).
>    - Re-defined struct SgxEPC, removed 'id' property and added struct SgxEPCList for
>      sgx-epc.0.{memdev}(Paolo).
>    - Removed g_malloc0(), and changed the 'SGXEPCState *sgx_epc' to 'SGXEPCState sgx_epc'
>      in struct PCMachineState(Paolo).
>    - Changed the SGX compound property cmdline from sgx-epc.{memdev}.0 to
>      sgx-epc.0.{memdev}(Paolo).
>    - Removed the signature from the 'git format-patch' command(Jarkko).
>
> v2-->v3:
>    - Rebased the sgx patches into latest Qemu release.
>    - Implemented the compound property for SGX, ref patch5, the command from '-sgx-epc'
>      to '-M'(Paolo).
>    - Moved the sgx common code from sgx-epc.c to sgx.c. The sgx-epc.c is
>      only responsible for virtual epc device.
>    - Removed the previous patch13(linux-headers: Add placeholder for KVM_CAP_SGX_ATTRIBUTE)
>      because ehabkost@redhat.com updated Linux headers to 5.13-rc4 with commit 278f064e452.
>    - Updated the patch1 because ram_flags were changed by David Hildenbra.
>    - Added one patch24, which avoid reset operation caused by bios reset.
>    - Added one patch25, which make prealloc property consistent with Qemu cmdline during VM
>      reset.
>
> v1-->v2:
>    - Rebased the sgx patches into latest Qemu release.
>    - Unified the "share" and "protected" arguments with ram_flags in the
>      memory_region_init_ram_from_fd()(Paolo).
>    - Added the new MemoryBackendEpcProperties and related documents(Eric Blake).
>    - Changed the KVM_CAP_SGX_ATTRIBUTE from 195 to 196(Kai).
>    - Changed the version and some grammar issues(Eric Blake).
>
> Sean Christopherson (21):
>   memory: Add RAM_PROTECTED flag to skip IOMMU mappings
>   hostmem: Add hostmem-epc as a backend for SGX EPC
>   i386: Add 'sgx-epc' device to expose EPC sections to guest
>   vl: Add sgx compound properties to expose SGX EPC sections to guest
>   i386: Add primary SGX CPUID and MSR defines
>   i386: Add SGX CPUID leaf FEAT_SGX_12_0_EAX
>   i386: Add SGX CPUID leaf FEAT_SGX_12_0_EBX
>   i386: Add SGX CPUID leaf FEAT_SGX_12_1_EAX
>   i386: Add get/set/migrate support for SGX_LEPUBKEYHASH MSRs
>   i386: Add feature control MSR dependency when SGX is enabled
>   i386: Update SGX CPUID info according to hardware/KVM/user input
>   i386: kvm: Add support for exposing PROVISIONKEY to guest
>   i386: Propagate SGX CPUID sub-leafs to KVM
>   Adjust min CPUID level to 0x12 when SGX is enabled
>   hw/i386/fw_cfg: Set SGX bits in feature control fw_cfg accordingly
>   hw/i386/pc: Account for SGX EPC sections when calculating device
>     memory
>   i386/pc: Add e820 entry for SGX EPC section(s)
>   i386: acpi: Add SGX EPC entry to ACPI tables
>   q35: Add support for SGX EPC
>   i440fx: Add support for SGX EPC
>   doc: Add the SGX doc
>
> Yang Zhong (12):
>   qom: Add memory-backend-epc ObjectOptions support
>   hostmem-epc: Add the reset interface for EPC backend reset
>   sgx-epc: Add the reset interface for sgx-epc virt device
>   sgx-epc: Avoid bios reset during sgx epc initialization
>   hostmem-epc: Make prealloc consistent with qemu cmdline during reset
>   qmp: Add query-sgx command
>   hmp: Add 'info sgx' command
>   i386: Add sgx_get_info() interface
>   bitops: Support 32 and 64 bit mask macro
>   qmp: Add the qmp_query_sgx_capabilities()
>   Kconfig: Add CONFIG_SGX support
>   sgx-epc: Add the fill_device_info() callback support
>
>  backends/hostmem-epc.c                   | 118 ++++++++++
>  backends/meson.build                     |   1 +
>  configs/devices/i386-softmmu/default.mak |   1 +
>  docs/intel-sgx.txt                       | 167 ++++++++++++++
>  hmp-commands-info.hx                     |  15 ++
>  hw/i386/Kconfig                          |   5 +
>  hw/i386/acpi-build.c                     |  22 ++
>  hw/i386/fw_cfg.c                         |  10 +-
>  hw/i386/meson.build                      |   2 +
>  hw/i386/pc.c                             |  15 +-
>  hw/i386/pc_piix.c                        |   4 +
>  hw/i386/pc_q35.c                         |   3 +
>  hw/i386/sgx-epc.c                        | 265 +++++++++++++++++++++++
>  hw/i386/sgx-stub.c                       |  13 ++
>  hw/i386/sgx.c                            | 170 +++++++++++++++
>  hw/i386/x86.c                            |  29 +++
>  hw/vfio/common.c                         |   1 +
>  include/exec/memory.h                    |  15 +-
>  include/hw/i386/pc.h                     |  10 +
>  include/hw/i386/sgx-epc.h                |  68 ++++++
>  include/hw/i386/x86.h                    |   1 +
>  include/monitor/hmp.h                    |   1 +
>  include/qemu/bitops.h                    |   7 +
>  monitor/hmp-cmds.c                       |  32 +++
>  monitor/qmp-cmds.c                       |  19 ++
>  qapi/machine.json                        |  52 ++++-
>  qapi/misc.json                           |  61 ++++++
>  qapi/qom.json                            |  19 ++
>  qemu-options.hx                          |  10 +-
>  softmmu/memory.c                         |   5 +
>  softmmu/physmem.c                        |   3 +-
>  stubs/meson.build                        |   1 +
>  stubs/sgx-stub.c                         |  12 +
>  target/i386/cpu.c                        | 168 +++++++++++++-
>  target/i386/cpu.h                        |  16 ++
>  target/i386/kvm/kvm.c                    |  75 +++++++
>  target/i386/kvm/kvm_i386.h               |   2 +
>  target/i386/machine.c                    |  20 ++
>  tests/qtest/qmp-cmd-test.c               |   2 +
>  39 files changed, 1430 insertions(+), 10 deletions(-)
>  create mode 100644 backends/hostmem-epc.c
>  create mode 100644 docs/intel-sgx.txt
>  create mode 100644 hw/i386/sgx-epc.c
>  create mode 100644 hw/i386/sgx-stub.c
>  create mode 100644 hw/i386/sgx.c
>  create mode 100644 include/hw/i386/sgx-epc.h
>  create mode 100644 stubs/sgx-stub.c
>


Re: [PATCH v4 00/33] Qemu SGX virtualization
Posted by Yang Zhong 2 years, 7 months ago
On Mon, Sep 06, 2021 at 03:13:08PM +0200, Paolo Bonzini wrote:
> Hi,
> 
> the monitor patches did not pass the test-hmp qtest, and also they
> should be in target/i386/monitor.c (see other commands that were
> implemented there for SEV).  However, I've sent a pull request with
> the rest.
>

  Paolo, thanks for pulling those patches! In fact, the first POC implemented the 
  monitor to target/i386/monitor.c, but in that time, there were lots of 'pragma GCC poison'
  errors during the build, so i had to move to common monitor. Let me check this again, thanks!

  Yang
 
> Thanks,
> 
> Paolo
> 
> On Mon, Jul 19, 2021 at 1:27 PM Yang Zhong <yang.zhong@intel.com> wrote:
> >
> > Since Sean Christopherson has left Intel and i am responsible for Qemu SGX
> > upstream work. His @intel.com address will be bouncing and his new email(
> > seanjc@google.com) is also in CC lists.
> >
> > This series is Qemu SGX virtualization implementation rebased on latest
> > Qemu release. The numa support for SGX will be sent in another patchset
> > once this basic SGX patchset are merged.
> >
> > You can find Qemu repo here:
> >
> >     https://github.com/intel/qemu-sgx.git upstream
> >
> > If you want to try SGX, you can directly install the linux release(at least 5.13.0-rc1+)
> > since kvm SGX has been merged into linux release.
> >
> >     https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
> >
> > To simplify, you'd better install linux on host and guest, which can support
> > SGX on host and guest kernel. And to me, use below reference command to boot
> > SGX guest:
> >
> >     #qemu-system-x86_64 \
> >         ...... \
> >         -cpu host,+sgx-provisionkey \
> >         -object memory-backend-epc,id=mem1,size=64M,prealloc=on \
> >         -object memory-backend-epc,id=mem2,size=28M \
> >         -M sgx-epc.0.memdev=mem1,sgx-epc.1.memdev=mem2
> >
> > Overview
> > ========
> >
> > Intel Software Guard eXtensions (SGX) is a set of instructions and mechanisms
> > for memory accesses in order to provide security accesses for sensitive
> > applications and data. SGX allows an application to use it's pariticular
> > address space as an *enclave*, which is a protected area provides confidentiality
> > and integrity even in the presence of privileged malware. Accesses to the
> > enclave memory area from any software not resident in the enclave are prevented,
> > including those from privileged software.
> >
> > SGX virtaulization
> > ==================
> >
> > The KVM SGX creates one new misc device, sgx_vepc, and Qemu will open '/dev/sgx_vepc'
> > device node to mmap() host EPC memory to guest. The Qemu also adds 'sgx-epc' device
> > to expose EPC sections to guest through CPUID and ACPI table.  The Qemu SGX also
> > supports multiple virtual EPC sections to guest, we just put them together physically
> > contiguous for the sake of simplicity. The kernel SGX NUMA has been merged into Linux
> > tip tree, we will support this function in the next phase.
> >
> > Although the current host SGX subsystem can not support SGX2 feature, the KVM/Qemu
> > implementation still expose this feature to guest. Guest SGX2 support doesn't have
> > interaction with host kernel SGX driver, the SGX guest can normally use those new
> > instructions.
> >
> > As for SGX virtualization detailed infomation, please reference docs/intel-sgx.txt
> > docuement(patch 33).
> >
> > Changelog:
> > =========
> >
> > (Changelog here is for global changes, please see each patch's changelog for changes
> > made to specific patch.)
> >
> > v3-->v4:
> >    - Rebased the sgx patches into latest Qemu release.
> >    - Moved sgx compound property setter/getter from MachineState to X86MachineState(Paolo).
> >    - Re-defined struct SgxEPC, removed 'id' property and added struct SgxEPCList for
> >      sgx-epc.0.{memdev}(Paolo).
> >    - Removed g_malloc0(), and changed the 'SGXEPCState *sgx_epc' to 'SGXEPCState sgx_epc'
> >      in struct PCMachineState(Paolo).
> >    - Changed the SGX compound property cmdline from sgx-epc.{memdev}.0 to
> >      sgx-epc.0.{memdev}(Paolo).
> >    - Removed the signature from the 'git format-patch' command(Jarkko).
> >
> > v2-->v3:
> >    - Rebased the sgx patches into latest Qemu release.
> >    - Implemented the compound property for SGX, ref patch5, the command from '-sgx-epc'
> >      to '-M'(Paolo).
> >    - Moved the sgx common code from sgx-epc.c to sgx.c. The sgx-epc.c is
> >      only responsible for virtual epc device.
> >    - Removed the previous patch13(linux-headers: Add placeholder for KVM_CAP_SGX_ATTRIBUTE)
> >      because ehabkost@redhat.com updated Linux headers to 5.13-rc4 with commit 278f064e452.
> >    - Updated the patch1 because ram_flags were changed by David Hildenbra.
> >    - Added one patch24, which avoid reset operation caused by bios reset.
> >    - Added one patch25, which make prealloc property consistent with Qemu cmdline during VM
> >      reset.
> >
> > v1-->v2:
> >    - Rebased the sgx patches into latest Qemu release.
> >    - Unified the "share" and "protected" arguments with ram_flags in the
> >      memory_region_init_ram_from_fd()(Paolo).
> >    - Added the new MemoryBackendEpcProperties and related documents(Eric Blake).
> >    - Changed the KVM_CAP_SGX_ATTRIBUTE from 195 to 196(Kai).
> >    - Changed the version and some grammar issues(Eric Blake).
> >
> > Sean Christopherson (21):
> >   memory: Add RAM_PROTECTED flag to skip IOMMU mappings
> >   hostmem: Add hostmem-epc as a backend for SGX EPC
> >   i386: Add 'sgx-epc' device to expose EPC sections to guest
> >   vl: Add sgx compound properties to expose SGX EPC sections to guest
> >   i386: Add primary SGX CPUID and MSR defines
> >   i386: Add SGX CPUID leaf FEAT_SGX_12_0_EAX
> >   i386: Add SGX CPUID leaf FEAT_SGX_12_0_EBX
> >   i386: Add SGX CPUID leaf FEAT_SGX_12_1_EAX
> >   i386: Add get/set/migrate support for SGX_LEPUBKEYHASH MSRs
> >   i386: Add feature control MSR dependency when SGX is enabled
> >   i386: Update SGX CPUID info according to hardware/KVM/user input
> >   i386: kvm: Add support for exposing PROVISIONKEY to guest
> >   i386: Propagate SGX CPUID sub-leafs to KVM
> >   Adjust min CPUID level to 0x12 when SGX is enabled
> >   hw/i386/fw_cfg: Set SGX bits in feature control fw_cfg accordingly
> >   hw/i386/pc: Account for SGX EPC sections when calculating device
> >     memory
> >   i386/pc: Add e820 entry for SGX EPC section(s)
> >   i386: acpi: Add SGX EPC entry to ACPI tables
> >   q35: Add support for SGX EPC
> >   i440fx: Add support for SGX EPC
> >   doc: Add the SGX doc
> >
> > Yang Zhong (12):
> >   qom: Add memory-backend-epc ObjectOptions support
> >   hostmem-epc: Add the reset interface for EPC backend reset
> >   sgx-epc: Add the reset interface for sgx-epc virt device
> >   sgx-epc: Avoid bios reset during sgx epc initialization
> >   hostmem-epc: Make prealloc consistent with qemu cmdline during reset
> >   qmp: Add query-sgx command
> >   hmp: Add 'info sgx' command
> >   i386: Add sgx_get_info() interface
> >   bitops: Support 32 and 64 bit mask macro
> >   qmp: Add the qmp_query_sgx_capabilities()
> >   Kconfig: Add CONFIG_SGX support
> >   sgx-epc: Add the fill_device_info() callback support
> >
> >  backends/hostmem-epc.c                   | 118 ++++++++++
> >  backends/meson.build                     |   1 +
> >  configs/devices/i386-softmmu/default.mak |   1 +
> >  docs/intel-sgx.txt                       | 167 ++++++++++++++
> >  hmp-commands-info.hx                     |  15 ++
> >  hw/i386/Kconfig                          |   5 +
> >  hw/i386/acpi-build.c                     |  22 ++
> >  hw/i386/fw_cfg.c                         |  10 +-
> >  hw/i386/meson.build                      |   2 +
> >  hw/i386/pc.c                             |  15 +-
> >  hw/i386/pc_piix.c                        |   4 +
> >  hw/i386/pc_q35.c                         |   3 +
> >  hw/i386/sgx-epc.c                        | 265 +++++++++++++++++++++++
> >  hw/i386/sgx-stub.c                       |  13 ++
> >  hw/i386/sgx.c                            | 170 +++++++++++++++
> >  hw/i386/x86.c                            |  29 +++
> >  hw/vfio/common.c                         |   1 +
> >  include/exec/memory.h                    |  15 +-
> >  include/hw/i386/pc.h                     |  10 +
> >  include/hw/i386/sgx-epc.h                |  68 ++++++
> >  include/hw/i386/x86.h                    |   1 +
> >  include/monitor/hmp.h                    |   1 +
> >  include/qemu/bitops.h                    |   7 +
> >  monitor/hmp-cmds.c                       |  32 +++
> >  monitor/qmp-cmds.c                       |  19 ++
> >  qapi/machine.json                        |  52 ++++-
> >  qapi/misc.json                           |  61 ++++++
> >  qapi/qom.json                            |  19 ++
> >  qemu-options.hx                          |  10 +-
> >  softmmu/memory.c                         |   5 +
> >  softmmu/physmem.c                        |   3 +-
> >  stubs/meson.build                        |   1 +
> >  stubs/sgx-stub.c                         |  12 +
> >  target/i386/cpu.c                        | 168 +++++++++++++-
> >  target/i386/cpu.h                        |  16 ++
> >  target/i386/kvm/kvm.c                    |  75 +++++++
> >  target/i386/kvm/kvm_i386.h               |   2 +
> >  target/i386/machine.c                    |  20 ++
> >  tests/qtest/qmp-cmd-test.c               |   2 +
> >  39 files changed, 1430 insertions(+), 10 deletions(-)
> >  create mode 100644 backends/hostmem-epc.c
> >  create mode 100644 docs/intel-sgx.txt
> >  create mode 100644 hw/i386/sgx-epc.c
> >  create mode 100644 hw/i386/sgx-stub.c
> >  create mode 100644 hw/i386/sgx.c
> >  create mode 100644 include/hw/i386/sgx-epc.h
> >  create mode 100644 stubs/sgx-stub.c
> >

Re: [PATCH v4 00/33] Qemu SGX virtualization
Posted by Yang Zhong 2 years, 7 months ago
On Mon, Sep 06, 2021 at 03:13:08PM +0200, Paolo Bonzini wrote:
> Hi,
> 
> the monitor patches did not pass the test-hmp qtest, and also they
> should be in target/i386/monitor.c (see other commands that were
> implemented there for SEV).  However, I've sent a pull request with
> the rest.
>
  
  Paolo, I have moved hmp and qmp codes to target/i386/monitor.c and also
  fixed the issue with test tool(tests/qtest/test-hmp). This issue is caused
  by 'machine none' test in the hmp, the previous patches only covered qmp
  test in 'machine none' with Libvirt.

  So, the next issue:
  1) re-send all sgx basic patches(including monitors patches) to you?
  2) only send monitor patches in the next phase when the basic sgx patches
     are merged? 

  Regards,

  Yang

 
> Thanks,
> 
> Paolo
> 
> On Mon, Jul 19, 2021 at 1:27 PM Yang Zhong <yang.zhong@intel.com> wrote:
> >
> > Since Sean Christopherson has left Intel and i am responsible for Qemu SGX
> > upstream work. His @intel.com address will be bouncing and his new email(
> > seanjc@google.com) is also in CC lists.
> >
> > This series is Qemu SGX virtualization implementation rebased on latest
> > Qemu release. The numa support for SGX will be sent in another patchset
> > once this basic SGX patchset are merged.
> >
> > You can find Qemu repo here:
> >
> >     https://github.com/intel/qemu-sgx.git upstream
> >
> > If you want to try SGX, you can directly install the linux release(at least 5.13.0-rc1+)
> > since kvm SGX has been merged into linux release.
> >
> >     https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
> >
> > To simplify, you'd better install linux on host and guest, which can support
> > SGX on host and guest kernel. And to me, use below reference command to boot
> > SGX guest:
> >
> >     #qemu-system-x86_64 \
> >         ...... \
> >         -cpu host,+sgx-provisionkey \
> >         -object memory-backend-epc,id=mem1,size=64M,prealloc=on \
> >         -object memory-backend-epc,id=mem2,size=28M \
> >         -M sgx-epc.0.memdev=mem1,sgx-epc.1.memdev=mem2
> >
> > Overview
> > ========
> >
> > Intel Software Guard eXtensions (SGX) is a set of instructions and mechanisms
> > for memory accesses in order to provide security accesses for sensitive
> > applications and data. SGX allows an application to use it's pariticular
> > address space as an *enclave*, which is a protected area provides confidentiality
> > and integrity even in the presence of privileged malware. Accesses to the
> > enclave memory area from any software not resident in the enclave are prevented,
> > including those from privileged software.
> >
> > SGX virtaulization
> > ==================
> >
> > The KVM SGX creates one new misc device, sgx_vepc, and Qemu will open '/dev/sgx_vepc'
> > device node to mmap() host EPC memory to guest. The Qemu also adds 'sgx-epc' device
> > to expose EPC sections to guest through CPUID and ACPI table.  The Qemu SGX also
> > supports multiple virtual EPC sections to guest, we just put them together physically
> > contiguous for the sake of simplicity. The kernel SGX NUMA has been merged into Linux
> > tip tree, we will support this function in the next phase.
> >
> > Although the current host SGX subsystem can not support SGX2 feature, the KVM/Qemu
> > implementation still expose this feature to guest. Guest SGX2 support doesn't have
> > interaction with host kernel SGX driver, the SGX guest can normally use those new
> > instructions.
> >
> > As for SGX virtualization detailed infomation, please reference docs/intel-sgx.txt
> > docuement(patch 33).
> >
> > Changelog:
> > =========
> >
> > (Changelog here is for global changes, please see each patch's changelog for changes
> > made to specific patch.)
> >
> > v3-->v4:
> >    - Rebased the sgx patches into latest Qemu release.
> >    - Moved sgx compound property setter/getter from MachineState to X86MachineState(Paolo).
> >    - Re-defined struct SgxEPC, removed 'id' property and added struct SgxEPCList for
> >      sgx-epc.0.{memdev}(Paolo).
> >    - Removed g_malloc0(), and changed the 'SGXEPCState *sgx_epc' to 'SGXEPCState sgx_epc'
> >      in struct PCMachineState(Paolo).
> >    - Changed the SGX compound property cmdline from sgx-epc.{memdev}.0 to
> >      sgx-epc.0.{memdev}(Paolo).
> >    - Removed the signature from the 'git format-patch' command(Jarkko).
> >
> > v2-->v3:
> >    - Rebased the sgx patches into latest Qemu release.
> >    - Implemented the compound property for SGX, ref patch5, the command from '-sgx-epc'
> >      to '-M'(Paolo).
> >    - Moved the sgx common code from sgx-epc.c to sgx.c. The sgx-epc.c is
> >      only responsible for virtual epc device.
> >    - Removed the previous patch13(linux-headers: Add placeholder for KVM_CAP_SGX_ATTRIBUTE)
> >      because ehabkost@redhat.com updated Linux headers to 5.13-rc4 with commit 278f064e452.
> >    - Updated the patch1 because ram_flags were changed by David Hildenbra.
> >    - Added one patch24, which avoid reset operation caused by bios reset.
> >    - Added one patch25, which make prealloc property consistent with Qemu cmdline during VM
> >      reset.
> >
> > v1-->v2:
> >    - Rebased the sgx patches into latest Qemu release.
> >    - Unified the "share" and "protected" arguments with ram_flags in the
> >      memory_region_init_ram_from_fd()(Paolo).
> >    - Added the new MemoryBackendEpcProperties and related documents(Eric Blake).
> >    - Changed the KVM_CAP_SGX_ATTRIBUTE from 195 to 196(Kai).
> >    - Changed the version and some grammar issues(Eric Blake).
> >
> > Sean Christopherson (21):
> >   memory: Add RAM_PROTECTED flag to skip IOMMU mappings
> >   hostmem: Add hostmem-epc as a backend for SGX EPC
> >   i386: Add 'sgx-epc' device to expose EPC sections to guest
> >   vl: Add sgx compound properties to expose SGX EPC sections to guest
> >   i386: Add primary SGX CPUID and MSR defines
> >   i386: Add SGX CPUID leaf FEAT_SGX_12_0_EAX
> >   i386: Add SGX CPUID leaf FEAT_SGX_12_0_EBX
> >   i386: Add SGX CPUID leaf FEAT_SGX_12_1_EAX
> >   i386: Add get/set/migrate support for SGX_LEPUBKEYHASH MSRs
> >   i386: Add feature control MSR dependency when SGX is enabled
> >   i386: Update SGX CPUID info according to hardware/KVM/user input
> >   i386: kvm: Add support for exposing PROVISIONKEY to guest
> >   i386: Propagate SGX CPUID sub-leafs to KVM
> >   Adjust min CPUID level to 0x12 when SGX is enabled
> >   hw/i386/fw_cfg: Set SGX bits in feature control fw_cfg accordingly
> >   hw/i386/pc: Account for SGX EPC sections when calculating device
> >     memory
> >   i386/pc: Add e820 entry for SGX EPC section(s)
> >   i386: acpi: Add SGX EPC entry to ACPI tables
> >   q35: Add support for SGX EPC
> >   i440fx: Add support for SGX EPC
> >   doc: Add the SGX doc
> >
> > Yang Zhong (12):
> >   qom: Add memory-backend-epc ObjectOptions support
> >   hostmem-epc: Add the reset interface for EPC backend reset
> >   sgx-epc: Add the reset interface for sgx-epc virt device
> >   sgx-epc: Avoid bios reset during sgx epc initialization
> >   hostmem-epc: Make prealloc consistent with qemu cmdline during reset
> >   qmp: Add query-sgx command
> >   hmp: Add 'info sgx' command
> >   i386: Add sgx_get_info() interface
> >   bitops: Support 32 and 64 bit mask macro
> >   qmp: Add the qmp_query_sgx_capabilities()
> >   Kconfig: Add CONFIG_SGX support
> >   sgx-epc: Add the fill_device_info() callback support
> >
> >  backends/hostmem-epc.c                   | 118 ++++++++++
> >  backends/meson.build                     |   1 +
> >  configs/devices/i386-softmmu/default.mak |   1 +
> >  docs/intel-sgx.txt                       | 167 ++++++++++++++
> >  hmp-commands-info.hx                     |  15 ++
> >  hw/i386/Kconfig                          |   5 +
> >  hw/i386/acpi-build.c                     |  22 ++
> >  hw/i386/fw_cfg.c                         |  10 +-
> >  hw/i386/meson.build                      |   2 +
> >  hw/i386/pc.c                             |  15 +-
> >  hw/i386/pc_piix.c                        |   4 +
> >  hw/i386/pc_q35.c                         |   3 +
> >  hw/i386/sgx-epc.c                        | 265 +++++++++++++++++++++++
> >  hw/i386/sgx-stub.c                       |  13 ++
> >  hw/i386/sgx.c                            | 170 +++++++++++++++
> >  hw/i386/x86.c                            |  29 +++
> >  hw/vfio/common.c                         |   1 +
> >  include/exec/memory.h                    |  15 +-
> >  include/hw/i386/pc.h                     |  10 +
> >  include/hw/i386/sgx-epc.h                |  68 ++++++
> >  include/hw/i386/x86.h                    |   1 +
> >  include/monitor/hmp.h                    |   1 +
> >  include/qemu/bitops.h                    |   7 +
> >  monitor/hmp-cmds.c                       |  32 +++
> >  monitor/qmp-cmds.c                       |  19 ++
> >  qapi/machine.json                        |  52 ++++-
> >  qapi/misc.json                           |  61 ++++++
> >  qapi/qom.json                            |  19 ++
> >  qemu-options.hx                          |  10 +-
> >  softmmu/memory.c                         |   5 +
> >  softmmu/physmem.c                        |   3 +-
> >  stubs/meson.build                        |   1 +
> >  stubs/sgx-stub.c                         |  12 +
> >  target/i386/cpu.c                        | 168 +++++++++++++-
> >  target/i386/cpu.h                        |  16 ++
> >  target/i386/kvm/kvm.c                    |  75 +++++++
> >  target/i386/kvm/kvm_i386.h               |   2 +
> >  target/i386/machine.c                    |  20 ++
> >  tests/qtest/qmp-cmd-test.c               |   2 +
> >  39 files changed, 1430 insertions(+), 10 deletions(-)
> >  create mode 100644 backends/hostmem-epc.c
> >  create mode 100644 docs/intel-sgx.txt
> >  create mode 100644 hw/i386/sgx-epc.c
> >  create mode 100644 hw/i386/sgx-stub.c
> >  create mode 100644 hw/i386/sgx.c
> >  create mode 100644 include/hw/i386/sgx-epc.h
> >  create mode 100644 stubs/sgx-stub.c
> >

Re: [PATCH v4 00/33] Qemu SGX virtualization
Posted by Paolo Bonzini 2 years, 7 months ago
On 07/09/21 11:51, Yang Zhong wrote:
>    
>    Paolo, I have moved hmp and qmp codes to target/i386/monitor.c and also
>    fixed the issue with test tool(tests/qtest/test-hmp). This issue is caused
>    by 'machine none' test in the hmp, the previous patches only covered qmp
>    test in 'machine none' with Libvirt.
> 
>    So, the next issue:
>    1) re-send all sgx basic patches(including monitors patches) to you?
>    2) only send monitor patches in the next phase when the basic sgx patches
>       are merged?

You can post them now, including

Based-on: <20210907071349.116329-1-pbonzini@redhat.com>

in the cover letter.

Paolo


Re: [PATCH v4 00/33] Qemu SGX virtualization
Posted by Jarkko Sakkinen 2 years, 7 months ago
On Tue, 2021-09-07 at 17:51 +0800, Yang Zhong wrote:
> On Mon, Sep 06, 2021 at 03:13:08PM +0200, Paolo Bonzini wrote:
> > Hi,
> > 
> > the monitor patches did not pass the test-hmp qtest, and also they
> > should be in target/i386/monitor.c (see other commands that were
> > implemented there for SEV).  However, I've sent a pull request with
> > the rest.
> > 
>   
>   Paolo, I have moved hmp and qmp codes to target/i386/monitor.c and also
>   fixed the issue with test tool(tests/qtest/test-hmp). This issue is caused
>   by 'machine none' test in the hmp, the previous patches only covered qmp
>   test in 'machine none' with Libvirt.
> 
>   So, the next issue:
>   1) re-send all sgx basic patches(including monitors patches) to you?
>   2) only send monitor patches in the next phase when the basic sgx patches
>      are merged? 
> 
>   Regards,
> 
>   Yang
> 
>  
> > Thanks,
> > 
> > Paolo
> > 
> > On Mon, Jul 19, 2021 at 1:27 PM Yang Zhong <yang.zhong@intel.com> wrote:
> > > Since Sean Christopherson has left Intel and i am responsible for Qemu SGX
> > > upstream work. His @intel.com address will be bouncing and his new email(
> > > seanjc@google.com) is also in CC lists.
> > > 
> > > This series is Qemu SGX virtualization implementation rebased on latest
> > > Qemu release. The numa support for SGX will be sent in another patchset
> > > once this basic SGX patchset are merged.
> > > 
> > > You can find Qemu repo here:
> > > 
> > >     https://github.com/intel/qemu-sgx.git upstream
> > > 
> > > If you want to try SGX, you can directly install the linux release(at least 5.13.0-rc1+)
> > > since kvm SGX has been merged into linux release.
> > > 
> > >     https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
> > > 
> > > To simplify, you'd better install linux on host and guest, which can support
> > > SGX on host and guest kernel. And to me, use below reference command to boot
> > > SGX guest:
> > > 
> > >     #qemu-system-x86_64 \
> > >         ...... \
> > >         -cpu host,+sgx-provisionkey \
> > >         -object memory-backend-epc,id=mem1,size=64M,prealloc=on \
> > >         -object memory-backend-epc,id=mem2,size=28M \

I would call these just "memory-backend-sgx".

> > >         -M sgx-epc.0.memdev=mem1,sgx-epc.1.memdev=mem2
> > > 
> > > Overview
> > > ========
> > > 
> > > Intel Software Guard eXtensions (SGX) is a set of instructions and mechanisms
> > > for memory accesses in order to provide security accesses for sensitive
> > > applications and data. SGX allows an application to use it's pariticular
> > > address space as an *enclave*, which is a protected area provides confidentiality
> > > and integrity even in the presence of privileged malware. Accesses to the
> > > enclave memory area from any software not resident in the enclave are prevented,
> > > including those from privileged software.
> > > 
> > > SGX virtaulization
          ~~~~~~~~~~~~~~
	  virtualization

I'm using these patches now instead of "real" hardwave. It seems that
9th gen cores are now Linux compatible SGX. Maybe some ucode update
has changed this because I don't recall this working before.


/Jarkko


Re: [PATCH v4 00/33] Qemu SGX virtualization
Posted by Philippe Mathieu-Daudé 2 years, 7 months ago
On 7/19/21 1:21 PM, Yang Zhong wrote:

> This series is Qemu SGX virtualization implementation rebased on latest
> Qemu release. The numa support for SGX will be sent in another patchset
> once this basic SGX patchset are merged.

>  backends/hostmem-epc.c                   | 118 ++++++++++
>  backends/meson.build                     |   1 +
>  configs/devices/i386-softmmu/default.mak |   1 +
>  docs/intel-sgx.txt                       | 167 ++++++++++++++
>  hmp-commands-info.hx                     |  15 ++
>  hw/i386/Kconfig                          |   5 +
>  hw/i386/acpi-build.c                     |  22 ++
>  hw/i386/fw_cfg.c                         |  10 +-
>  hw/i386/meson.build                      |   2 +
>  hw/i386/pc.c                             |  15 +-
>  hw/i386/pc_piix.c                        |   4 +
>  hw/i386/pc_q35.c                         |   3 +
>  hw/i386/sgx-epc.c                        | 265 +++++++++++++++++++++++
>  hw/i386/sgx-stub.c                       |  13 ++
>  hw/i386/sgx.c                            | 170 +++++++++++++++
>  hw/i386/x86.c                            |  29 +++
>  hw/vfio/common.c                         |   1 +
>  include/exec/memory.h                    |  15 +-
>  include/hw/i386/pc.h                     |  10 +
>  include/hw/i386/sgx-epc.h                |  68 ++++++
>  include/hw/i386/x86.h                    |   1 +
>  include/monitor/hmp.h                    |   1 +
>  include/qemu/bitops.h                    |   7 +

Consider using scripts/git.orderfile to ease reviewers workflow.


Re: [PATCH v4 00/33] Qemu SGX virtualization
Posted by Yang Zhong 2 years, 7 months ago
On Tue, Sep 14, 2021 at 08:51:47AM +0200, Philippe Mathieu-Daudé wrote:
> On 7/19/21 1:21 PM, Yang Zhong wrote:
> 
> > This series is Qemu SGX virtualization implementation rebased on latest
> > Qemu release. The numa support for SGX will be sent in another patchset
> > once this basic SGX patchset are merged.
> 
> >  backends/hostmem-epc.c                   | 118 ++++++++++
> >  backends/meson.build                     |   1 +
> >  configs/devices/i386-softmmu/default.mak |   1 +
> >  docs/intel-sgx.txt                       | 167 ++++++++++++++
> >  hmp-commands-info.hx                     |  15 ++
> >  hw/i386/Kconfig                          |   5 +
> >  hw/i386/acpi-build.c                     |  22 ++
> >  hw/i386/fw_cfg.c                         |  10 +-
> >  hw/i386/meson.build                      |   2 +
> >  hw/i386/pc.c                             |  15 +-
> >  hw/i386/pc_piix.c                        |   4 +
> >  hw/i386/pc_q35.c                         |   3 +
> >  hw/i386/sgx-epc.c                        | 265 +++++++++++++++++++++++
> >  hw/i386/sgx-stub.c                       |  13 ++
> >  hw/i386/sgx.c                            | 170 +++++++++++++++
> >  hw/i386/x86.c                            |  29 +++
> >  hw/vfio/common.c                         |   1 +
> >  include/exec/memory.h                    |  15 +-
> >  include/hw/i386/pc.h                     |  10 +
> >  include/hw/i386/sgx-epc.h                |  68 ++++++
> >  include/hw/i386/x86.h                    |   1 +
> >  include/monitor/hmp.h                    |   1 +
> >  include/qemu/bitops.h                    |   7 +
> 
> Consider using scripts/git.orderfile to ease reviewers workflow.

  Thanks! I never noticed this issue before, thanks!

  My ref command:
  git format-patch -32 --no-signature --cover-letter -O scripts/git.orderfile --subject-prefix="PATCH"

  Yang