accel/kvm/kvm-all.c | 73 ++++++++++++++++++++++++++ accel/stubs/kvm-stub.c | 5 ++ hw/i386/pc_sysfw.c | 10 +++- include/sysemu/cpus.h | 2 + include/sysemu/hw_accel.h | 5 ++ include/sysemu/kvm.h | 18 +++++++ include/sysemu/sev.h | 3 ++ softmmu/cpus.c | 5 ++ softmmu/vl.c | 5 +- target/i386/cpu.c | 1 + target/i386/kvm.c | 2 + target/i386/sev-stub.c | 5 ++ target/i386/sev.c | 105 +++++++++++++++++++++++++++++++++++++- target/i386/sev_i386.h | 1 + 14 files changed, 236 insertions(+), 4 deletions(-)
From: Tom Lendacky <thomas.lendacky@amd.com>
This patch series provides support for launching an SEV-ES guest.
Secure Encrypted Virtualization - Encrypted State (SEV-ES) expands on the
SEV support to protect the guest register state from the hypervisor. See
"AMD64 Architecture Programmer's Manual Volume 2: System Programming",
section "15.35 Encrypted State (SEV-ES)" [1].
In order to allow a hypervisor to perform functions on behalf of a guest,
there is architectural support for notifying a guest's operating system
when certain types of VMEXITs are about to occur. This allows the guest to
selectively share information with the hypervisor to satisfy the requested
function. The notification is performed using a new exception, the VMM
Communication exception (#VC). The information is shared through the
Guest-Hypervisor Communication Block (GHCB) using the VMGEXIT instruction.
The GHCB format and the protocol for using it is documented in "SEV-ES
Guest-Hypervisor Communication Block Standardization" [2].
The main areas of the Qemu code that are updated to support SEV-ES are
around the SEV guest launch process and AP booting in order to support
booting multiple vCPUs.
There are no new command line switches required. Instead, the desire for
SEV-ES is presented using the SEV policy object. Bit 2 of the SEV policy
object indicates that SEV-ES is required.
The SEV launch process is updated in two ways. The first is that a the
KVM_SEV_ES_INIT ioctl is used to initialize the guest instead of the
standard KVM_SEV_INIT ioctl. The second is that before the SEV launch
measurement is calculated, the LAUNCH_UPDATE_VMSA SEV API is invoked for
each vCPU that Qemu has created. Once the LAUNCH_UPDATE_VMSA API has been
invoked, no direct changes to the guest register state can be made.
AP booting poses some interesting challenges. The INIT-SIPI-SIPI sequence
is typically used to boot the APs. However, the hypervisor is not allowed
to update the guest registers. For the APs, the reset vector must be known
in advance. An OVMF method to provide a known reset vector address exists
by providing an SEV information block, identified by UUID, near the end of
the firmware [3]. OVMF will program the jump to the actual reset vector in
this area of memory. Since the memory location is known in advance, an AP
can be created with the known reset vector address as its starting CS:IP.
The GHCB document [2] talks about how SMP booting under SEV-ES is
performed. SEV-ES also requires the use of the in-kernel irqchip support
in order to minimize the changes required to Qemu to support AP booting.
[1] https://www.amd.com/system/files/TechDocs/24593.pdf
[2] https://developer.amd.com/wp-content/resources/56421.pdf
[3] 30937f2f98c4 ("OvmfPkg: Use the SEV-ES work area for the SEV-ES AP reset vector")
https://github.com/tianocore/edk2/commit/30937f2f98c42496f2f143fe8374ae7f7e684847
---
These patches are based on commit:
d0ed6a69d3 ("Update version for v5.1.0 release")
(I tried basing on the latest Qemu commit, but I was having build issues
that level)
A version of the tree can be found at:
https://github.com/AMDESE/qemu/tree/sev-es-v11
Changes since v2:
- Add in-kernel irqchip requirement for SEV-ES guests
Changes since v1:
- Fixed checkpatch.pl errors/warnings
Tom Lendacky (5):
sev/i386: Add initial support for SEV-ES
sev/i386: Require in-kernel irqchip support for SEV-ES guests
sev/i386: Allow AP booting under SEV-ES
sev/i386: Don't allow a system reset under an SEV-ES guest
sev/i386: Enable an SEV-ES guest based on SEV policy
accel/kvm/kvm-all.c | 73 ++++++++++++++++++++++++++
accel/stubs/kvm-stub.c | 5 ++
hw/i386/pc_sysfw.c | 10 +++-
include/sysemu/cpus.h | 2 +
include/sysemu/hw_accel.h | 5 ++
include/sysemu/kvm.h | 18 +++++++
include/sysemu/sev.h | 3 ++
softmmu/cpus.c | 5 ++
softmmu/vl.c | 5 +-
target/i386/cpu.c | 1 +
target/i386/kvm.c | 2 +
target/i386/sev-stub.c | 5 ++
target/i386/sev.c | 105 +++++++++++++++++++++++++++++++++++++-
target/i386/sev_i386.h | 1 +
14 files changed, 236 insertions(+), 4 deletions(-)
--
2.28.0
* Tom Lendacky (thomas.lendacky@amd.com) wrote:
> From: Tom Lendacky <thomas.lendacky@amd.com>
>
> This patch series provides support for launching an SEV-ES guest.
>
> Secure Encrypted Virtualization - Encrypted State (SEV-ES) expands on the
> SEV support to protect the guest register state from the hypervisor. See
> "AMD64 Architecture Programmer's Manual Volume 2: System Programming",
> section "15.35 Encrypted State (SEV-ES)" [1].
>
> In order to allow a hypervisor to perform functions on behalf of a guest,
> there is architectural support for notifying a guest's operating system
> when certain types of VMEXITs are about to occur. This allows the guest to
> selectively share information with the hypervisor to satisfy the requested
> function. The notification is performed using a new exception, the VMM
> Communication exception (#VC). The information is shared through the
> Guest-Hypervisor Communication Block (GHCB) using the VMGEXIT instruction.
> The GHCB format and the protocol for using it is documented in "SEV-ES
> Guest-Hypervisor Communication Block Standardization" [2].
>
> The main areas of the Qemu code that are updated to support SEV-ES are
> around the SEV guest launch process and AP booting in order to support
> booting multiple vCPUs.
>
> There are no new command line switches required. Instead, the desire for
> SEV-ES is presented using the SEV policy object. Bit 2 of the SEV policy
> object indicates that SEV-ES is required.
>
> The SEV launch process is updated in two ways. The first is that a the
> KVM_SEV_ES_INIT ioctl is used to initialize the guest instead of the
> standard KVM_SEV_INIT ioctl. The second is that before the SEV launch
> measurement is calculated, the LAUNCH_UPDATE_VMSA SEV API is invoked for
> each vCPU that Qemu has created. Once the LAUNCH_UPDATE_VMSA API has been
> invoked, no direct changes to the guest register state can be made.
>
> AP booting poses some interesting challenges. The INIT-SIPI-SIPI sequence
> is typically used to boot the APs. However, the hypervisor is not allowed
> to update the guest registers. For the APs, the reset vector must be known
> in advance. An OVMF method to provide a known reset vector address exists
> by providing an SEV information block, identified by UUID, near the end of
> the firmware [3]. OVMF will program the jump to the actual reset vector in
> this area of memory. Since the memory location is known in advance, an AP
> can be created with the known reset vector address as its starting CS:IP.
> The GHCB document [2] talks about how SMP booting under SEV-ES is
> performed. SEV-ES also requires the use of the in-kernel irqchip support
> in order to minimize the changes required to Qemu to support AP booting.
Some random thoughts:
a) Is there something that explicitly disallows SMM?
b) I think all the interfaces you're using are already defined in
Linux header files - even if the code to implement them isn't actually
upstream in the kernel yet (the launch_update in particular) - we
normally wait for the kernel interface to be accepted before taking the
QEMU patches, but if the constants are in the headers already I'm not
sure what the rule is.
c) What happens if QEMU reads the register values from the state if
the guest is paused - does it just see junk? I'm just wondering if you
need to add checks in places it might try to.
Dave
> [1] https://www.amd.com/system/files/TechDocs/24593.pdf
> [2] https://developer.amd.com/wp-content/resources/56421.pdf
> [3] 30937f2f98c4 ("OvmfPkg: Use the SEV-ES work area for the SEV-ES AP reset vector")
> https://github.com/tianocore/edk2/commit/30937f2f98c42496f2f143fe8374ae7f7e684847
>
> ---
>
> These patches are based on commit:
> d0ed6a69d3 ("Update version for v5.1.0 release")
>
> (I tried basing on the latest Qemu commit, but I was having build issues
> that level)
>
> A version of the tree can be found at:
> https://github.com/AMDESE/qemu/tree/sev-es-v11
>
> Changes since v2:
> - Add in-kernel irqchip requirement for SEV-ES guests
>
> Changes since v1:
> - Fixed checkpatch.pl errors/warnings
>
> Tom Lendacky (5):
> sev/i386: Add initial support for SEV-ES
> sev/i386: Require in-kernel irqchip support for SEV-ES guests
> sev/i386: Allow AP booting under SEV-ES
> sev/i386: Don't allow a system reset under an SEV-ES guest
> sev/i386: Enable an SEV-ES guest based on SEV policy
>
> accel/kvm/kvm-all.c | 73 ++++++++++++++++++++++++++
> accel/stubs/kvm-stub.c | 5 ++
> hw/i386/pc_sysfw.c | 10 +++-
> include/sysemu/cpus.h | 2 +
> include/sysemu/hw_accel.h | 5 ++
> include/sysemu/kvm.h | 18 +++++++
> include/sysemu/sev.h | 3 ++
> softmmu/cpus.c | 5 ++
> softmmu/vl.c | 5 +-
> target/i386/cpu.c | 1 +
> target/i386/kvm.c | 2 +
> target/i386/sev-stub.c | 5 ++
> target/i386/sev.c | 105 +++++++++++++++++++++++++++++++++++++-
> target/i386/sev_i386.h | 1 +
> 14 files changed, 236 insertions(+), 4 deletions(-)
>
> --
> 2.28.0
>
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
On 9/17/20 12:28 PM, Dr. David Alan Gilbert wrote:
> * Tom Lendacky (thomas.lendacky@amd.com) wrote:
>> From: Tom Lendacky <thomas.lendacky@amd.com>
>>
>> This patch series provides support for launching an SEV-ES guest.
>>
>> Secure Encrypted Virtualization - Encrypted State (SEV-ES) expands on the
>> SEV support to protect the guest register state from the hypervisor. See
>> "AMD64 Architecture Programmer's Manual Volume 2: System Programming",
>> section "15.35 Encrypted State (SEV-ES)" [1].
>>
>> In order to allow a hypervisor to perform functions on behalf of a guest,
>> there is architectural support for notifying a guest's operating system
>> when certain types of VMEXITs are about to occur. This allows the guest to
>> selectively share information with the hypervisor to satisfy the requested
>> function. The notification is performed using a new exception, the VMM
>> Communication exception (#VC). The information is shared through the
>> Guest-Hypervisor Communication Block (GHCB) using the VMGEXIT instruction.
>> The GHCB format and the protocol for using it is documented in "SEV-ES
>> Guest-Hypervisor Communication Block Standardization" [2].
>>
>> The main areas of the Qemu code that are updated to support SEV-ES are
>> around the SEV guest launch process and AP booting in order to support
>> booting multiple vCPUs.
>>
>> There are no new command line switches required. Instead, the desire for
>> SEV-ES is presented using the SEV policy object. Bit 2 of the SEV policy
>> object indicates that SEV-ES is required.
>>
>> The SEV launch process is updated in two ways. The first is that a the
>> KVM_SEV_ES_INIT ioctl is used to initialize the guest instead of the
>> standard KVM_SEV_INIT ioctl. The second is that before the SEV launch
>> measurement is calculated, the LAUNCH_UPDATE_VMSA SEV API is invoked for
>> each vCPU that Qemu has created. Once the LAUNCH_UPDATE_VMSA API has been
>> invoked, no direct changes to the guest register state can be made.
>>
>> AP booting poses some interesting challenges. The INIT-SIPI-SIPI sequence
>> is typically used to boot the APs. However, the hypervisor is not allowed
>> to update the guest registers. For the APs, the reset vector must be known
>> in advance. An OVMF method to provide a known reset vector address exists
>> by providing an SEV information block, identified by UUID, near the end of
>> the firmware [3]. OVMF will program the jump to the actual reset vector in
>> this area of memory. Since the memory location is known in advance, an AP
>> can be created with the known reset vector address as its starting CS:IP.
>> The GHCB document [2] talks about how SMP booting under SEV-ES is
>> performed. SEV-ES also requires the use of the in-kernel irqchip support
>> in order to minimize the changes required to Qemu to support AP booting.
>
> Some random thoughts:
> a) Is there something that explicitly disallows SMM?
There isn't currently. Is there a way to know early on that SMM is
enabled? Could I just call x86_machine_is_smm_enabled() to check that?
> b) I think all the interfaces you're using are already defined in
> Linux header files - even if the code to implement them isn't actually
> upstream in the kernel yet (the launch_update in particular) - we
> normally wait for the kernel interface to be accepted before taking the
> QEMU patches, but if the constants are in the headers already I'm not
> sure what the rule is.
Correct, everything was already present from a Linux header perspective.
> c) What happens if QEMU reads the register values from the state if
> the guest is paused - does it just see junk? I'm just wondering if you
> need to add checks in places it might try to.
I thought about what to do about calls to read the registers once the
guest state has become encrypted. I think it would take a lot of changes
to make Qemu "protected state aware" for what I see as little gain. Qemu
is likely to see a lot of zeroes or actual register values from the GHCB
protocol for previous VMGEXITs that took place.
Thanks,
Tom
>
> Dave
>
>> [1] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.amd.com%2Fsystem%2Ffiles%2FTechDocs%2F24593.pdf&data=02%7C01%7Cthomas.lendacky%40amd.com%7Cb07b788e09054a91143308d85b2f1a89%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637359606292398926&sdata=B2naGIEXuhD7a%2Fi4NDsRzeHwvDvNJ%2FP7nf5HmAzk9CU%3D&reserved=0
>> [2] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdeveloper.amd.com%2Fwp-content%2Fresources%2F56421.pdf&data=02%7C01%7Cthomas.lendacky%40amd.com%7Cb07b788e09054a91143308d85b2f1a89%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637359606292398926&sdata=0HrHZxdTEK%2FWM1KxxasMAghpzTNGvuKKSlg6nBgPjJY%3D&reserved=0
>> [3] 30937f2f98c4 ("OvmfPkg: Use the SEV-ES work area for the SEV-ES AP reset vector")
>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fcommit%2F30937f2f98c42496f2f143fe8374ae7f7e684847&data=02%7C01%7Cthomas.lendacky%40amd.com%7Cb07b788e09054a91143308d85b2f1a89%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637359606292408916&sdata=ISAjIahZH4izDHnXgdWDX0GK61kwgtTw%2BEE%2BS8FBls0%3D&reserved=0
>>
>> ---
>>
>> These patches are based on commit:
>> d0ed6a69d3 ("Update version for v5.1.0 release")
>>
>> (I tried basing on the latest Qemu commit, but I was having build issues
>> that level)
>>
>> A version of the tree can be found at:
>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAMDESE%2Fqemu%2Ftree%2Fsev-es-v11&data=02%7C01%7Cthomas.lendacky%40amd.com%7Cb07b788e09054a91143308d85b2f1a89%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637359606292408916&sdata=pWd8HAZkAILIMRb1i5TNz9XoHyrhCgRu%2Bq%2BXN2NJ4ag%3D&reserved=0
>>
>> Changes since v2:
>> - Add in-kernel irqchip requirement for SEV-ES guests
>>
>> Changes since v1:
>> - Fixed checkpatch.pl errors/warnings
>>
>> Tom Lendacky (5):
>> sev/i386: Add initial support for SEV-ES
>> sev/i386: Require in-kernel irqchip support for SEV-ES guests
>> sev/i386: Allow AP booting under SEV-ES
>> sev/i386: Don't allow a system reset under an SEV-ES guest
>> sev/i386: Enable an SEV-ES guest based on SEV policy
>>
>> accel/kvm/kvm-all.c | 73 ++++++++++++++++++++++++++
>> accel/stubs/kvm-stub.c | 5 ++
>> hw/i386/pc_sysfw.c | 10 +++-
>> include/sysemu/cpus.h | 2 +
>> include/sysemu/hw_accel.h | 5 ++
>> include/sysemu/kvm.h | 18 +++++++
>> include/sysemu/sev.h | 3 ++
>> softmmu/cpus.c | 5 ++
>> softmmu/vl.c | 5 +-
>> target/i386/cpu.c | 1 +
>> target/i386/kvm.c | 2 +
>> target/i386/sev-stub.c | 5 ++
>> target/i386/sev.c | 105 +++++++++++++++++++++++++++++++++++++-
>> target/i386/sev_i386.h | 1 +
>> 14 files changed, 236 insertions(+), 4 deletions(-)
>>
>> --
>> 2.28.0
>>
On Thu, Sep 17, 2020 at 01:56:21PM -0500, Tom Lendacky wrote:
> On 9/17/20 12:28 PM, Dr. David Alan Gilbert wrote:
> > * Tom Lendacky (thomas.lendacky@amd.com) wrote:
> > > From: Tom Lendacky <thomas.lendacky@amd.com>
> > >
> > > This patch series provides support for launching an SEV-ES guest.
> > >
> > > Secure Encrypted Virtualization - Encrypted State (SEV-ES) expands on the
> > > SEV support to protect the guest register state from the hypervisor. See
> > > "AMD64 Architecture Programmer's Manual Volume 2: System Programming",
> > > section "15.35 Encrypted State (SEV-ES)" [1].
> > >
> > > In order to allow a hypervisor to perform functions on behalf of a guest,
> > > there is architectural support for notifying a guest's operating system
> > > when certain types of VMEXITs are about to occur. This allows the guest to
> > > selectively share information with the hypervisor to satisfy the requested
> > > function. The notification is performed using a new exception, the VMM
> > > Communication exception (#VC). The information is shared through the
> > > Guest-Hypervisor Communication Block (GHCB) using the VMGEXIT instruction.
> > > The GHCB format and the protocol for using it is documented in "SEV-ES
> > > Guest-Hypervisor Communication Block Standardization" [2].
> > >
> > > The main areas of the Qemu code that are updated to support SEV-ES are
> > > around the SEV guest launch process and AP booting in order to support
> > > booting multiple vCPUs.
> > >
> > > There are no new command line switches required. Instead, the desire for
> > > SEV-ES is presented using the SEV policy object. Bit 2 of the SEV policy
> > > object indicates that SEV-ES is required.
> > >
> > > The SEV launch process is updated in two ways. The first is that a the
> > > KVM_SEV_ES_INIT ioctl is used to initialize the guest instead of the
> > > standard KVM_SEV_INIT ioctl. The second is that before the SEV launch
> > > measurement is calculated, the LAUNCH_UPDATE_VMSA SEV API is invoked for
> > > each vCPU that Qemu has created. Once the LAUNCH_UPDATE_VMSA API has been
> > > invoked, no direct changes to the guest register state can be made.
> > >
> > > AP booting poses some interesting challenges. The INIT-SIPI-SIPI sequence
> > > is typically used to boot the APs. However, the hypervisor is not allowed
> > > to update the guest registers. For the APs, the reset vector must be known
> > > in advance. An OVMF method to provide a known reset vector address exists
> > > by providing an SEV information block, identified by UUID, near the end of
> > > the firmware [3]. OVMF will program the jump to the actual reset vector in
> > > this area of memory. Since the memory location is known in advance, an AP
> > > can be created with the known reset vector address as its starting CS:IP.
> > > The GHCB document [2] talks about how SMP booting under SEV-ES is
> > > performed. SEV-ES also requires the use of the in-kernel irqchip support
> > > in order to minimize the changes required to Qemu to support AP booting.
> >
> > Some random thoughts:
> > a) Is there something that explicitly disallows SMM?
>
> There isn't currently. Is there a way to know early on that SMM is enabled?
> Could I just call x86_machine_is_smm_enabled() to check that?
KVM_CAP_X86_SMM is currently checked as a KVM-wide capability. One option
is to change that to use a per-VM ioctl() and then have KVM return 0 for
SEV-ES VMs.
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index 416c82048a..4d7f84ed1b 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -145,7 +145,7 @@ int kvm_has_pit_state2(void)
bool kvm_has_smm(void)
{
- return kvm_check_extension(kvm_state, KVM_CAP_X86_SMM);
+ return kvm_vm_check_extension(kvm_state, KVM_CAP_X86_SMM);
}
bool kvm_has_adjust_clock_stable(void)
> > b) I think all the interfaces you're using are already defined in
> > Linux header files - even if the code to implement them isn't actually
> > upstream in the kernel yet (the launch_update in particular) - we
> > normally wait for the kernel interface to be accepted before taking the
> > QEMU patches, but if the constants are in the headers already I'm not
> > sure what the rule is.
>
> Correct, everything was already present from a Linux header perspective.
>
> > c) What happens if QEMU reads the register values from the state if
> > the guest is paused - does it just see junk? I'm just wondering if you
> > need to add checks in places it might try to.
>
> I thought about what to do about calls to read the registers once the guest
> state has become encrypted. I think it would take a lot of changes to make
> Qemu "protected state aware" for what I see as little gain. Qemu is likely
> to see a lot of zeroes or actual register values from the GHCB protocol for
> previous VMGEXITs that took place.
Yeah, we more or less came to the same conclusion for TDX. It's easy enough
to throw an error if QEMU attempts to read protected state, but without
other invasive changes, it's too easy to unintentionally kill the VM. MSRs
are a bit of a special case, but for REGS, SREGS, and whatever other state
is read out, simply letting KVM return zeros/garbage seems like the lesser
of all evils.
On 9/17/20 10:40 PM, Sean Christopherson wrote:
> On Thu, Sep 17, 2020 at 01:56:21PM -0500, Tom Lendacky wrote:
>> On 9/17/20 12:28 PM, Dr. David Alan Gilbert wrote:
>>> * Tom Lendacky (thomas.lendacky@amd.com) wrote:
>>>> From: Tom Lendacky <thomas.lendacky@amd.com>
>>>>
>>>> This patch series provides support for launching an SEV-ES guest.
>>>>
>>>> Secure Encrypted Virtualization - Encrypted State (SEV-ES) expands on the
>>>> SEV support to protect the guest register state from the hypervisor. See
>>>> "AMD64 Architecture Programmer's Manual Volume 2: System Programming",
>>>> section "15.35 Encrypted State (SEV-ES)" [1].
>>>>
>>>> In order to allow a hypervisor to perform functions on behalf of a guest,
>>>> there is architectural support for notifying a guest's operating system
>>>> when certain types of VMEXITs are about to occur. This allows the guest to
>>>> selectively share information with the hypervisor to satisfy the requested
>>>> function. The notification is performed using a new exception, the VMM
>>>> Communication exception (#VC). The information is shared through the
>>>> Guest-Hypervisor Communication Block (GHCB) using the VMGEXIT instruction.
>>>> The GHCB format and the protocol for using it is documented in "SEV-ES
>>>> Guest-Hypervisor Communication Block Standardization" [2].
>>>>
>>>> The main areas of the Qemu code that are updated to support SEV-ES are
>>>> around the SEV guest launch process and AP booting in order to support
>>>> booting multiple vCPUs.
>>>>
>>>> There are no new command line switches required. Instead, the desire for
>>>> SEV-ES is presented using the SEV policy object. Bit 2 of the SEV policy
>>>> object indicates that SEV-ES is required.
>>>>
>>>> The SEV launch process is updated in two ways. The first is that a the
>>>> KVM_SEV_ES_INIT ioctl is used to initialize the guest instead of the
>>>> standard KVM_SEV_INIT ioctl. The second is that before the SEV launch
>>>> measurement is calculated, the LAUNCH_UPDATE_VMSA SEV API is invoked for
>>>> each vCPU that Qemu has created. Once the LAUNCH_UPDATE_VMSA API has been
>>>> invoked, no direct changes to the guest register state can be made.
>>>>
>>>> AP booting poses some interesting challenges. The INIT-SIPI-SIPI sequence
>>>> is typically used to boot the APs. However, the hypervisor is not allowed
>>>> to update the guest registers. For the APs, the reset vector must be known
>>>> in advance. An OVMF method to provide a known reset vector address exists
>>>> by providing an SEV information block, identified by UUID, near the end of
>>>> the firmware [3]. OVMF will program the jump to the actual reset vector in
>>>> this area of memory. Since the memory location is known in advance, an AP
>>>> can be created with the known reset vector address as its starting CS:IP.
>>>> The GHCB document [2] talks about how SMP booting under SEV-ES is
>>>> performed. SEV-ES also requires the use of the in-kernel irqchip support
>>>> in order to minimize the changes required to Qemu to support AP booting.
>>>
>>> Some random thoughts:
>>> a) Is there something that explicitly disallows SMM?
>>
>> There isn't currently. Is there a way to know early on that SMM is enabled?
>> Could I just call x86_machine_is_smm_enabled() to check that?
>
> KVM_CAP_X86_SMM is currently checked as a KVM-wide capability. One option
> is to change that to use a per-VM ioctl() and then have KVM return 0 for
> SEV-ES VMs.
>
> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
> index 416c82048a..4d7f84ed1b 100644
> --- a/target/i386/kvm.c
> +++ b/target/i386/kvm.c
> @@ -145,7 +145,7 @@ int kvm_has_pit_state2(void)
>
> bool kvm_has_smm(void)
> {
> - return kvm_check_extension(kvm_state, KVM_CAP_X86_SMM);
> + return kvm_vm_check_extension(kvm_state, KVM_CAP_X86_SMM);
> }
This will work. I'll have to modify the has_emulated_msr() op in the
kernel as part of the the SEV-ES support to take a struct kvm argument.
I'll be sure to include a comment that the struct kvm argument could be
NULL, since that op is also used during KVM module initialization and is
called before VM initialization (and therefore a struct kvm instance), too.
Thanks,
Tom
>
> bool kvm_has_adjust_clock_stable(void)
>
>>> b) I think all the interfaces you're using are already defined in
>>> Linux header files - even if the code to implement them isn't actually
>>> upstream in the kernel yet (the launch_update in particular) - we
>>> normally wait for the kernel interface to be accepted before taking the
>>> QEMU patches, but if the constants are in the headers already I'm not
>>> sure what the rule is.
>>
>> Correct, everything was already present from a Linux header perspective.
>>
>>> c) What happens if QEMU reads the register values from the state if
>>> the guest is paused - does it just see junk? I'm just wondering if you
>>> need to add checks in places it might try to.
>>
>> I thought about what to do about calls to read the registers once the guest
>> state has become encrypted. I think it would take a lot of changes to make
>> Qemu "protected state aware" for what I see as little gain. Qemu is likely
>> to see a lot of zeroes or actual register values from the GHCB protocol for
>> previous VMGEXITs that took place.
>
> Yeah, we more or less came to the same conclusion for TDX. It's easy enough
> to throw an error if QEMU attempts to read protected state, but without
> other invasive changes, it's too easy to unintentionally kill the VM. MSRs
> are a bit of a special case, but for REGS, SREGS, and whatever other state
> is read out, simply letting KVM return zeros/garbage seems like the lesser
> of all evils.
>
* Tom Lendacky (thomas.lendacky@amd.com) wrote:
> On 9/17/20 12:28 PM, Dr. David Alan Gilbert wrote:
> > * Tom Lendacky (thomas.lendacky@amd.com) wrote:
> > > From: Tom Lendacky <thomas.lendacky@amd.com>
> > >
> > > This patch series provides support for launching an SEV-ES guest.
> > >
> > > Secure Encrypted Virtualization - Encrypted State (SEV-ES) expands on the
> > > SEV support to protect the guest register state from the hypervisor. See
> > > "AMD64 Architecture Programmer's Manual Volume 2: System Programming",
> > > section "15.35 Encrypted State (SEV-ES)" [1].
> > >
> > > In order to allow a hypervisor to perform functions on behalf of a guest,
> > > there is architectural support for notifying a guest's operating system
> > > when certain types of VMEXITs are about to occur. This allows the guest to
> > > selectively share information with the hypervisor to satisfy the requested
> > > function. The notification is performed using a new exception, the VMM
> > > Communication exception (#VC). The information is shared through the
> > > Guest-Hypervisor Communication Block (GHCB) using the VMGEXIT instruction.
> > > The GHCB format and the protocol for using it is documented in "SEV-ES
> > > Guest-Hypervisor Communication Block Standardization" [2].
> > >
> > > The main areas of the Qemu code that are updated to support SEV-ES are
> > > around the SEV guest launch process and AP booting in order to support
> > > booting multiple vCPUs.
> > >
> > > There are no new command line switches required. Instead, the desire for
> > > SEV-ES is presented using the SEV policy object. Bit 2 of the SEV policy
> > > object indicates that SEV-ES is required.
> > >
> > > The SEV launch process is updated in two ways. The first is that a the
> > > KVM_SEV_ES_INIT ioctl is used to initialize the guest instead of the
> > > standard KVM_SEV_INIT ioctl. The second is that before the SEV launch
> > > measurement is calculated, the LAUNCH_UPDATE_VMSA SEV API is invoked for
> > > each vCPU that Qemu has created. Once the LAUNCH_UPDATE_VMSA API has been
> > > invoked, no direct changes to the guest register state can be made.
> > >
> > > AP booting poses some interesting challenges. The INIT-SIPI-SIPI sequence
> > > is typically used to boot the APs. However, the hypervisor is not allowed
> > > to update the guest registers. For the APs, the reset vector must be known
> > > in advance. An OVMF method to provide a known reset vector address exists
> > > by providing an SEV information block, identified by UUID, near the end of
> > > the firmware [3]. OVMF will program the jump to the actual reset vector in
> > > this area of memory. Since the memory location is known in advance, an AP
> > > can be created with the known reset vector address as its starting CS:IP.
> > > The GHCB document [2] talks about how SMP booting under SEV-ES is
> > > performed. SEV-ES also requires the use of the in-kernel irqchip support
> > > in order to minimize the changes required to Qemu to support AP booting.
> >
> > Some random thoughts:
> > a) Is there something that explicitly disallows SMM?
>
> There isn't currently. Is there a way to know early on that SMM is enabled?
> Could I just call x86_machine_is_smm_enabled() to check that?
>
> > b) I think all the interfaces you're using are already defined in
> > Linux header files - even if the code to implement them isn't actually
> > upstream in the kernel yet (the launch_update in particular) - we
> > normally wait for the kernel interface to be accepted before taking the
> > QEMU patches, but if the constants are in the headers already I'm not
> > sure what the rule is.
>
> Correct, everything was already present from a Linux header perspective.
>
> > c) What happens if QEMU reads the register values from the state if
> > the guest is paused - does it just see junk? I'm just wondering if you
> > need to add checks in places it might try to.
>
> I thought about what to do about calls to read the registers once the guest
> state has become encrypted. I think it would take a lot of changes to make
> Qemu "protected state aware" for what I see as little gain. Qemu is likely
> to see a lot of zeroes or actual register values from the GHCB protocol for
> previous VMGEXITs that took place.
Yep, that's fair enough - I was curious if we'll hit anything
accidentally still reading it.
How does SEV-ES interact with the 'NODBG' flag of the guest policy - if
that's 0, and 'debugging of the guest' is allowed, what can you actually
do?
Dave
> Thanks,
> Tom
>
> >
> > Dave
> >
> > > [1] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.amd.com%2Fsystem%2Ffiles%2FTechDocs%2F24593.pdf&data=02%7C01%7Cthomas.lendacky%40amd.com%7Cb07b788e09054a91143308d85b2f1a89%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637359606292398926&sdata=B2naGIEXuhD7a%2Fi4NDsRzeHwvDvNJ%2FP7nf5HmAzk9CU%3D&reserved=0
> > > [2] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdeveloper.amd.com%2Fwp-content%2Fresources%2F56421.pdf&data=02%7C01%7Cthomas.lendacky%40amd.com%7Cb07b788e09054a91143308d85b2f1a89%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637359606292398926&sdata=0HrHZxdTEK%2FWM1KxxasMAghpzTNGvuKKSlg6nBgPjJY%3D&reserved=0
> > > [3] 30937f2f98c4 ("OvmfPkg: Use the SEV-ES work area for the SEV-ES AP reset vector")
> > > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fcommit%2F30937f2f98c42496f2f143fe8374ae7f7e684847&data=02%7C01%7Cthomas.lendacky%40amd.com%7Cb07b788e09054a91143308d85b2f1a89%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637359606292408916&sdata=ISAjIahZH4izDHnXgdWDX0GK61kwgtTw%2BEE%2BS8FBls0%3D&reserved=0
> > >
> > > ---
> > >
> > > These patches are based on commit:
> > > d0ed6a69d3 ("Update version for v5.1.0 release")
> > >
> > > (I tried basing on the latest Qemu commit, but I was having build issues
> > > that level)
> > >
> > > A version of the tree can be found at:
> > > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAMDESE%2Fqemu%2Ftree%2Fsev-es-v11&data=02%7C01%7Cthomas.lendacky%40amd.com%7Cb07b788e09054a91143308d85b2f1a89%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637359606292408916&sdata=pWd8HAZkAILIMRb1i5TNz9XoHyrhCgRu%2Bq%2BXN2NJ4ag%3D&reserved=0
> > >
> > > Changes since v2:
> > > - Add in-kernel irqchip requirement for SEV-ES guests
> > >
> > > Changes since v1:
> > > - Fixed checkpatch.pl errors/warnings
> > >
> > > Tom Lendacky (5):
> > > sev/i386: Add initial support for SEV-ES
> > > sev/i386: Require in-kernel irqchip support for SEV-ES guests
> > > sev/i386: Allow AP booting under SEV-ES
> > > sev/i386: Don't allow a system reset under an SEV-ES guest
> > > sev/i386: Enable an SEV-ES guest based on SEV policy
> > >
> > > accel/kvm/kvm-all.c | 73 ++++++++++++++++++++++++++
> > > accel/stubs/kvm-stub.c | 5 ++
> > > hw/i386/pc_sysfw.c | 10 +++-
> > > include/sysemu/cpus.h | 2 +
> > > include/sysemu/hw_accel.h | 5 ++
> > > include/sysemu/kvm.h | 18 +++++++
> > > include/sysemu/sev.h | 3 ++
> > > softmmu/cpus.c | 5 ++
> > > softmmu/vl.c | 5 +-
> > > target/i386/cpu.c | 1 +
> > > target/i386/kvm.c | 2 +
> > > target/i386/sev-stub.c | 5 ++
> > > target/i386/sev.c | 105 +++++++++++++++++++++++++++++++++++++-
> > > target/i386/sev_i386.h | 1 +
> > > 14 files changed, 236 insertions(+), 4 deletions(-)
> > >
> > > --
> > > 2.28.0
> > >
>
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
On 9/18/20 5:00 AM, Dr. David Alan Gilbert wrote:
> * Tom Lendacky (thomas.lendacky@amd.com) wrote:
>> On 9/17/20 12:28 PM, Dr. David Alan Gilbert wrote:
>>> * Tom Lendacky (thomas.lendacky@amd.com) wrote:
>>>> From: Tom Lendacky <thomas.lendacky@amd.com>
>>>>
>>>> This patch series provides support for launching an SEV-ES guest.
>>>>
>>>> Secure Encrypted Virtualization - Encrypted State (SEV-ES) expands on the
>>>> SEV support to protect the guest register state from the hypervisor. See
>>>> "AMD64 Architecture Programmer's Manual Volume 2: System Programming",
>>>> section "15.35 Encrypted State (SEV-ES)" [1].
>>>>
>>>> In order to allow a hypervisor to perform functions on behalf of a guest,
>>>> there is architectural support for notifying a guest's operating system
>>>> when certain types of VMEXITs are about to occur. This allows the guest to
>>>> selectively share information with the hypervisor to satisfy the requested
>>>> function. The notification is performed using a new exception, the VMM
>>>> Communication exception (#VC). The information is shared through the
>>>> Guest-Hypervisor Communication Block (GHCB) using the VMGEXIT instruction.
>>>> The GHCB format and the protocol for using it is documented in "SEV-ES
>>>> Guest-Hypervisor Communication Block Standardization" [2].
>>>>
>>>> The main areas of the Qemu code that are updated to support SEV-ES are
>>>> around the SEV guest launch process and AP booting in order to support
>>>> booting multiple vCPUs.
>>>>
>>>> There are no new command line switches required. Instead, the desire for
>>>> SEV-ES is presented using the SEV policy object. Bit 2 of the SEV policy
>>>> object indicates that SEV-ES is required.
>>>>
>>>> The SEV launch process is updated in two ways. The first is that a the
>>>> KVM_SEV_ES_INIT ioctl is used to initialize the guest instead of the
>>>> standard KVM_SEV_INIT ioctl. The second is that before the SEV launch
>>>> measurement is calculated, the LAUNCH_UPDATE_VMSA SEV API is invoked for
>>>> each vCPU that Qemu has created. Once the LAUNCH_UPDATE_VMSA API has been
>>>> invoked, no direct changes to the guest register state can be made.
>>>>
>>>> AP booting poses some interesting challenges. The INIT-SIPI-SIPI sequence
>>>> is typically used to boot the APs. However, the hypervisor is not allowed
>>>> to update the guest registers. For the APs, the reset vector must be known
>>>> in advance. An OVMF method to provide a known reset vector address exists
>>>> by providing an SEV information block, identified by UUID, near the end of
>>>> the firmware [3]. OVMF will program the jump to the actual reset vector in
>>>> this area of memory. Since the memory location is known in advance, an AP
>>>> can be created with the known reset vector address as its starting CS:IP.
>>>> The GHCB document [2] talks about how SMP booting under SEV-ES is
>>>> performed. SEV-ES also requires the use of the in-kernel irqchip support
>>>> in order to minimize the changes required to Qemu to support AP booting.
>>>
>>> Some random thoughts:
>>> a) Is there something that explicitly disallows SMM?
>>
>> There isn't currently. Is there a way to know early on that SMM is enabled?
>> Could I just call x86_machine_is_smm_enabled() to check that?
>>
>>> b) I think all the interfaces you're using are already defined in
>>> Linux header files - even if the code to implement them isn't actually
>>> upstream in the kernel yet (the launch_update in particular) - we
>>> normally wait for the kernel interface to be accepted before taking the
>>> QEMU patches, but if the constants are in the headers already I'm not
>>> sure what the rule is.
>>
>> Correct, everything was already present from a Linux header perspective.
>>
>>> c) What happens if QEMU reads the register values from the state if
>>> the guest is paused - does it just see junk? I'm just wondering if you
>>> need to add checks in places it might try to.
>>
>> I thought about what to do about calls to read the registers once the guest
>> state has become encrypted. I think it would take a lot of changes to make
>> Qemu "protected state aware" for what I see as little gain. Qemu is likely
>> to see a lot of zeroes or actual register values from the GHCB protocol for
>> previous VMGEXITs that took place.
>
> Yep, that's fair enough - I was curious if we'll hit anything
> accidentally still reading it.
>
> How does SEV-ES interact with the 'NODBG' flag of the guest policy - if
> that's 0, and 'debugging of the guest' is allowed, what can you actually
> do?
The SEV-ES KVM patches will disallow debugging of the guest, or at least
setting breakpoints using the debug registers. Gdb can still break in, but
you wont get anything reasonable with register dumps and memory dumps.
The NODBG policy bit enables or disables the DBG_DECRYPT and DBG_ENCRYPT
APIs. So if the guest has allowed debugging, memory dumps could be done
using those APIs (for encrypted pages). Registers are a different story
because you simply can't update from the hypervisor side under SEV-ES.
Under SEV you could do actual debugging if the support was developed and
in place.
Thanks,
Tom
>
> Dave
>
>> Thanks,
>> Tom
>>
>>>
>>> Dave
>>>
>>>> [1] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.amd.com%2Fsystem%2Ffiles%2FTechDocs%2F24593.pdf&data=02%7C01%7Cthomas.lendacky%40amd.com%7Cecf88d6f7bd0494d1b0e08d85bb9c19b%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637360201727448609&sdata=e6CbpjDDvCUG2q9pk6OSsty0QB5HuhueVAM4t8iygT8%3D&reserved=0
>>>> [2] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdeveloper.amd.com%2Fwp-content%2Fresources%2F56421.pdf&data=02%7C01%7Cthomas.lendacky%40amd.com%7Cecf88d6f7bd0494d1b0e08d85bb9c19b%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637360201727448609&sdata=%2FUzJB5K%2F8rOMF%2B%2BVPXjg%2BJBLgD4uLW6U82Wvf8pXq%2BA%3D&reserved=0
>>>> [3] 30937f2f98c4 ("OvmfPkg: Use the SEV-ES work area for the SEV-ES AP reset vector")
>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fcommit%2F30937f2f98c42496f2f143fe8374ae7f7e684847&data=02%7C01%7Cthomas.lendacky%40amd.com%7Cecf88d6f7bd0494d1b0e08d85bb9c19b%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637360201727458605&sdata=0FmiYEdIEtDjw1VIGaeeRrto%2FZpvH1esIgE93gXyagM%3D&reserved=0
>>>>
>>>> ---
>>>>
>>>> These patches are based on commit:
>>>> d0ed6a69d3 ("Update version for v5.1.0 release")
>>>>
>>>> (I tried basing on the latest Qemu commit, but I was having build issues
>>>> that level)
>>>>
>>>> A version of the tree can be found at:
>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAMDESE%2Fqemu%2Ftree%2Fsev-es-v11&data=02%7C01%7Cthomas.lendacky%40amd.com%7Cecf88d6f7bd0494d1b0e08d85bb9c19b%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637360201727458605&sdata=w1tfrMDgruZBDxNHgKLhpKtQ50Ua%2FMy9IfkSXfne2xg%3D&reserved=0
>>>>
>>>> Changes since v2:
>>>> - Add in-kernel irqchip requirement for SEV-ES guests
>>>>
>>>> Changes since v1:
>>>> - Fixed checkpatch.pl errors/warnings
>>>>
>>>> Tom Lendacky (5):
>>>> sev/i386: Add initial support for SEV-ES
>>>> sev/i386: Require in-kernel irqchip support for SEV-ES guests
>>>> sev/i386: Allow AP booting under SEV-ES
>>>> sev/i386: Don't allow a system reset under an SEV-ES guest
>>>> sev/i386: Enable an SEV-ES guest based on SEV policy
>>>>
>>>> accel/kvm/kvm-all.c | 73 ++++++++++++++++++++++++++
>>>> accel/stubs/kvm-stub.c | 5 ++
>>>> hw/i386/pc_sysfw.c | 10 +++-
>>>> include/sysemu/cpus.h | 2 +
>>>> include/sysemu/hw_accel.h | 5 ++
>>>> include/sysemu/kvm.h | 18 +++++++
>>>> include/sysemu/sev.h | 3 ++
>>>> softmmu/cpus.c | 5 ++
>>>> softmmu/vl.c | 5 +-
>>>> target/i386/cpu.c | 1 +
>>>> target/i386/kvm.c | 2 +
>>>> target/i386/sev-stub.c | 5 ++
>>>> target/i386/sev.c | 105 +++++++++++++++++++++++++++++++++++++-
>>>> target/i386/sev_i386.h | 1 +
>>>> 14 files changed, 236 insertions(+), 4 deletions(-)
>>>>
>>>> --
>>>> 2.28.0
>>>>
>>
* Tom Lendacky (thomas.lendacky@amd.com) wrote:
> On 9/18/20 5:00 AM, Dr. David Alan Gilbert wrote:
> > * Tom Lendacky (thomas.lendacky@amd.com) wrote:
> > > On 9/17/20 12:28 PM, Dr. David Alan Gilbert wrote:
> > > > * Tom Lendacky (thomas.lendacky@amd.com) wrote:
> > > > > From: Tom Lendacky <thomas.lendacky@amd.com>
> > > > >
> > > > > This patch series provides support for launching an SEV-ES guest.
> > > > >
> > > > > Secure Encrypted Virtualization - Encrypted State (SEV-ES) expands on the
> > > > > SEV support to protect the guest register state from the hypervisor. See
> > > > > "AMD64 Architecture Programmer's Manual Volume 2: System Programming",
> > > > > section "15.35 Encrypted State (SEV-ES)" [1].
> > > > >
> > > > > In order to allow a hypervisor to perform functions on behalf of a guest,
> > > > > there is architectural support for notifying a guest's operating system
> > > > > when certain types of VMEXITs are about to occur. This allows the guest to
> > > > > selectively share information with the hypervisor to satisfy the requested
> > > > > function. The notification is performed using a new exception, the VMM
> > > > > Communication exception (#VC). The information is shared through the
> > > > > Guest-Hypervisor Communication Block (GHCB) using the VMGEXIT instruction.
> > > > > The GHCB format and the protocol for using it is documented in "SEV-ES
> > > > > Guest-Hypervisor Communication Block Standardization" [2].
> > > > >
> > > > > The main areas of the Qemu code that are updated to support SEV-ES are
> > > > > around the SEV guest launch process and AP booting in order to support
> > > > > booting multiple vCPUs.
> > > > >
> > > > > There are no new command line switches required. Instead, the desire for
> > > > > SEV-ES is presented using the SEV policy object. Bit 2 of the SEV policy
> > > > > object indicates that SEV-ES is required.
> > > > >
> > > > > The SEV launch process is updated in two ways. The first is that a the
> > > > > KVM_SEV_ES_INIT ioctl is used to initialize the guest instead of the
> > > > > standard KVM_SEV_INIT ioctl. The second is that before the SEV launch
> > > > > measurement is calculated, the LAUNCH_UPDATE_VMSA SEV API is invoked for
> > > > > each vCPU that Qemu has created. Once the LAUNCH_UPDATE_VMSA API has been
> > > > > invoked, no direct changes to the guest register state can be made.
> > > > >
> > > > > AP booting poses some interesting challenges. The INIT-SIPI-SIPI sequence
> > > > > is typically used to boot the APs. However, the hypervisor is not allowed
> > > > > to update the guest registers. For the APs, the reset vector must be known
> > > > > in advance. An OVMF method to provide a known reset vector address exists
> > > > > by providing an SEV information block, identified by UUID, near the end of
> > > > > the firmware [3]. OVMF will program the jump to the actual reset vector in
> > > > > this area of memory. Since the memory location is known in advance, an AP
> > > > > can be created with the known reset vector address as its starting CS:IP.
> > > > > The GHCB document [2] talks about how SMP booting under SEV-ES is
> > > > > performed. SEV-ES also requires the use of the in-kernel irqchip support
> > > > > in order to minimize the changes required to Qemu to support AP booting.
> > > >
> > > > Some random thoughts:
> > > > a) Is there something that explicitly disallows SMM?
> > >
> > > There isn't currently. Is there a way to know early on that SMM is enabled?
> > > Could I just call x86_machine_is_smm_enabled() to check that?
> > >
> > > > b) I think all the interfaces you're using are already defined in
> > > > Linux header files - even if the code to implement them isn't actually
> > > > upstream in the kernel yet (the launch_update in particular) - we
> > > > normally wait for the kernel interface to be accepted before taking the
> > > > QEMU patches, but if the constants are in the headers already I'm not
> > > > sure what the rule is.
> > >
> > > Correct, everything was already present from a Linux header perspective.
> > >
> > > > c) What happens if QEMU reads the register values from the state if
> > > > the guest is paused - does it just see junk? I'm just wondering if you
> > > > need to add checks in places it might try to.
> > >
> > > I thought about what to do about calls to read the registers once the guest
> > > state has become encrypted. I think it would take a lot of changes to make
> > > Qemu "protected state aware" for what I see as little gain. Qemu is likely
> > > to see a lot of zeroes or actual register values from the GHCB protocol for
> > > previous VMGEXITs that took place.
> >
> > Yep, that's fair enough - I was curious if we'll hit anything
> > accidentally still reading it.
> >
> > How does SEV-ES interact with the 'NODBG' flag of the guest policy - if
> > that's 0, and 'debugging of the guest' is allowed, what can you actually
> > do?
>
> The SEV-ES KVM patches will disallow debugging of the guest, or at least
> setting breakpoints using the debug registers. Gdb can still break in, but
> you wont get anything reasonable with register dumps and memory dumps.
>
> The NODBG policy bit enables or disables the DBG_DECRYPT and DBG_ENCRYPT
> APIs. So if the guest has allowed debugging, memory dumps could be done
> using those APIs (for encrypted pages). Registers are a different story
> because you simply can't update from the hypervisor side under SEV-ES.
>
> Under SEV you could do actual debugging if the support was developed and in
> place.
Thanks for the explanation - it might be interesting to wire the
DBG_DECRYPT support up to dump/dump.c for doing full guest memory dumps
- if the policy has it enabled.
Dave
> Thanks,
> Tom
>
> >
> > Dave
> >
> > > Thanks,
> > > Tom
> > >
> > > >
> > > > Dave
> > > >
> > > > > [1] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.amd.com%2Fsystem%2Ffiles%2FTechDocs%2F24593.pdf&data=02%7C01%7Cthomas.lendacky%40amd.com%7Cecf88d6f7bd0494d1b0e08d85bb9c19b%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637360201727448609&sdata=e6CbpjDDvCUG2q9pk6OSsty0QB5HuhueVAM4t8iygT8%3D&reserved=0
> > > > > [2] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdeveloper.amd.com%2Fwp-content%2Fresources%2F56421.pdf&data=02%7C01%7Cthomas.lendacky%40amd.com%7Cecf88d6f7bd0494d1b0e08d85bb9c19b%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637360201727448609&sdata=%2FUzJB5K%2F8rOMF%2B%2BVPXjg%2BJBLgD4uLW6U82Wvf8pXq%2BA%3D&reserved=0
> > > > > [3] 30937f2f98c4 ("OvmfPkg: Use the SEV-ES work area for the SEV-ES AP reset vector")
> > > > > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fcommit%2F30937f2f98c42496f2f143fe8374ae7f7e684847&data=02%7C01%7Cthomas.lendacky%40amd.com%7Cecf88d6f7bd0494d1b0e08d85bb9c19b%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637360201727458605&sdata=0FmiYEdIEtDjw1VIGaeeRrto%2FZpvH1esIgE93gXyagM%3D&reserved=0
> > > > >
> > > > > ---
> > > > >
> > > > > These patches are based on commit:
> > > > > d0ed6a69d3 ("Update version for v5.1.0 release")
> > > > >
> > > > > (I tried basing on the latest Qemu commit, but I was having build issues
> > > > > that level)
> > > > >
> > > > > A version of the tree can be found at:
> > > > > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAMDESE%2Fqemu%2Ftree%2Fsev-es-v11&data=02%7C01%7Cthomas.lendacky%40amd.com%7Cecf88d6f7bd0494d1b0e08d85bb9c19b%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637360201727458605&sdata=w1tfrMDgruZBDxNHgKLhpKtQ50Ua%2FMy9IfkSXfne2xg%3D&reserved=0
> > > > >
> > > > > Changes since v2:
> > > > > - Add in-kernel irqchip requirement for SEV-ES guests
> > > > >
> > > > > Changes since v1:
> > > > > - Fixed checkpatch.pl errors/warnings
> > > > >
> > > > > Tom Lendacky (5):
> > > > > sev/i386: Add initial support for SEV-ES
> > > > > sev/i386: Require in-kernel irqchip support for SEV-ES guests
> > > > > sev/i386: Allow AP booting under SEV-ES
> > > > > sev/i386: Don't allow a system reset under an SEV-ES guest
> > > > > sev/i386: Enable an SEV-ES guest based on SEV policy
> > > > >
> > > > > accel/kvm/kvm-all.c | 73 ++++++++++++++++++++++++++
> > > > > accel/stubs/kvm-stub.c | 5 ++
> > > > > hw/i386/pc_sysfw.c | 10 +++-
> > > > > include/sysemu/cpus.h | 2 +
> > > > > include/sysemu/hw_accel.h | 5 ++
> > > > > include/sysemu/kvm.h | 18 +++++++
> > > > > include/sysemu/sev.h | 3 ++
> > > > > softmmu/cpus.c | 5 ++
> > > > > softmmu/vl.c | 5 +-
> > > > > target/i386/cpu.c | 1 +
> > > > > target/i386/kvm.c | 2 +
> > > > > target/i386/sev-stub.c | 5 ++
> > > > > target/i386/sev.c | 105 +++++++++++++++++++++++++++++++++++++-
> > > > > target/i386/sev_i386.h | 1 +
> > > > > 14 files changed, 236 insertions(+), 4 deletions(-)
> > > > >
> > > > > --
> > > > > 2.28.0
> > > > >
> > >
>
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
On 9/21/20 6:48 AM, Dr. David Alan Gilbert wrote:
> * Tom Lendacky (thomas.lendacky@amd.com) wrote:
>> On 9/18/20 5:00 AM, Dr. David Alan Gilbert wrote:
>>> * Tom Lendacky (thomas.lendacky@amd.com) wrote:
>>>> On 9/17/20 12:28 PM, Dr. David Alan Gilbert wrote:
>>>>> * Tom Lendacky (thomas.lendacky@amd.com) wrote:
>>>>>> From: Tom Lendacky <thomas.lendacky@amd.com>
>>>>>>
>>>>>> This patch series provides support for launching an SEV-ES guest.
>>>>>>
>>>>>> Secure Encrypted Virtualization - Encrypted State (SEV-ES) expands on the
>>>>>> SEV support to protect the guest register state from the hypervisor. See
>>>>>> "AMD64 Architecture Programmer's Manual Volume 2: System Programming",
>>>>>> section "15.35 Encrypted State (SEV-ES)" [1].
>>>>>>
>>>>>> In order to allow a hypervisor to perform functions on behalf of a guest,
>>>>>> there is architectural support for notifying a guest's operating system
>>>>>> when certain types of VMEXITs are about to occur. This allows the guest to
>>>>>> selectively share information with the hypervisor to satisfy the requested
>>>>>> function. The notification is performed using a new exception, the VMM
>>>>>> Communication exception (#VC). The information is shared through the
>>>>>> Guest-Hypervisor Communication Block (GHCB) using the VMGEXIT instruction.
>>>>>> The GHCB format and the protocol for using it is documented in "SEV-ES
>>>>>> Guest-Hypervisor Communication Block Standardization" [2].
>>>>>>
>>>>>> The main areas of the Qemu code that are updated to support SEV-ES are
>>>>>> around the SEV guest launch process and AP booting in order to support
>>>>>> booting multiple vCPUs.
>>>>>>
>>>>>> There are no new command line switches required. Instead, the desire for
>>>>>> SEV-ES is presented using the SEV policy object. Bit 2 of the SEV policy
>>>>>> object indicates that SEV-ES is required.
>>>>>>
>>>>>> The SEV launch process is updated in two ways. The first is that a the
>>>>>> KVM_SEV_ES_INIT ioctl is used to initialize the guest instead of the
>>>>>> standard KVM_SEV_INIT ioctl. The second is that before the SEV launch
>>>>>> measurement is calculated, the LAUNCH_UPDATE_VMSA SEV API is invoked for
>>>>>> each vCPU that Qemu has created. Once the LAUNCH_UPDATE_VMSA API has been
>>>>>> invoked, no direct changes to the guest register state can be made.
>>>>>>
>>>>>> AP booting poses some interesting challenges. The INIT-SIPI-SIPI sequence
>>>>>> is typically used to boot the APs. However, the hypervisor is not allowed
>>>>>> to update the guest registers. For the APs, the reset vector must be known
>>>>>> in advance. An OVMF method to provide a known reset vector address exists
>>>>>> by providing an SEV information block, identified by UUID, near the end of
>>>>>> the firmware [3]. OVMF will program the jump to the actual reset vector in
>>>>>> this area of memory. Since the memory location is known in advance, an AP
>>>>>> can be created with the known reset vector address as its starting CS:IP.
>>>>>> The GHCB document [2] talks about how SMP booting under SEV-ES is
>>>>>> performed. SEV-ES also requires the use of the in-kernel irqchip support
>>>>>> in order to minimize the changes required to Qemu to support AP booting.
>>>>>
>>>>> Some random thoughts:
>>>>> a) Is there something that explicitly disallows SMM?
>>>>
>>>> There isn't currently. Is there a way to know early on that SMM is enabled?
>>>> Could I just call x86_machine_is_smm_enabled() to check that?
>>>>
>>>>> b) I think all the interfaces you're using are already defined in
>>>>> Linux header files - even if the code to implement them isn't actually
>>>>> upstream in the kernel yet (the launch_update in particular) - we
>>>>> normally wait for the kernel interface to be accepted before taking the
>>>>> QEMU patches, but if the constants are in the headers already I'm not
>>>>> sure what the rule is.
>>>>
>>>> Correct, everything was already present from a Linux header perspective.
>>>>
>>>>> c) What happens if QEMU reads the register values from the state if
>>>>> the guest is paused - does it just see junk? I'm just wondering if you
>>>>> need to add checks in places it might try to.
>>>>
>>>> I thought about what to do about calls to read the registers once the guest
>>>> state has become encrypted. I think it would take a lot of changes to make
>>>> Qemu "protected state aware" for what I see as little gain. Qemu is likely
>>>> to see a lot of zeroes or actual register values from the GHCB protocol for
>>>> previous VMGEXITs that took place.
>>>
>>> Yep, that's fair enough - I was curious if we'll hit anything
>>> accidentally still reading it.
>>>
>>> How does SEV-ES interact with the 'NODBG' flag of the guest policy - if
>>> that's 0, and 'debugging of the guest' is allowed, what can you actually
>>> do?
>>
>> The SEV-ES KVM patches will disallow debugging of the guest, or at least
>> setting breakpoints using the debug registers. Gdb can still break in, but
>> you wont get anything reasonable with register dumps and memory dumps.
>>
>> The NODBG policy bit enables or disables the DBG_DECRYPT and DBG_ENCRYPT
>> APIs. So if the guest has allowed debugging, memory dumps could be done
>> using those APIs (for encrypted pages). Registers are a different story
>> because you simply can't update from the hypervisor side under SEV-ES.
>>
>> Under SEV you could do actual debugging if the support was developed and in
>> place.
>
> Thanks for the explanation - it might be interesting to wire the
> DBG_DECRYPT support up to dump/dump.c for doing full guest memory dumps
> - if the policy has it enabled.
Someone on our team is looking at hooking in those APIs for displaying /
dumping memory. I'll forward this on to them.
Thanks,
Tom
>
> Dave
>
>> Thanks,
>> Tom
>>
>>>
>>> Dave
>>>
>>>> Thanks,
>>>> Tom
>>>>
>>>>>
>>>>> Dave
>>>>>
>>>>>> [1] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.amd.com%2Fsystem%2Ffiles%2FTechDocs%2F24593.pdf&data=02%7C01%7Cthomas.lendacky%40amd.com%7C8287b3e99fdc41c682fe08d85e244ee0%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637362858215616687&sdata=d9n0rRa2AvVS1dmKyBoCF%2BapDhV6UUc8mfGpWD2%2FxMU%3D&reserved=0
>>>>>> [2] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdeveloper.amd.com%2Fwp-content%2Fresources%2F56421.pdf&data=02%7C01%7Cthomas.lendacky%40amd.com%7C8287b3e99fdc41c682fe08d85e244ee0%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637362858215616687&sdata=84imjHi1kXbEf2omr1sS%2Brl7gKYeGFGKk%2BeTavyluN4%3D&reserved=0
>>>>>> [3] 30937f2f98c4 ("OvmfPkg: Use the SEV-ES work area for the SEV-ES AP reset vector")
>>>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fcommit%2F30937f2f98c42496f2f143fe8374ae7f7e684847&data=02%7C01%7Cthomas.lendacky%40amd.com%7C8287b3e99fdc41c682fe08d85e244ee0%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637362858215616687&sdata=BTWlq%2BqgpjRtUrbEZE1wasyXxs3YFmfKioEvzvWlsYc%3D&reserved=0
>>>>>>
>>>>>> ---
>>>>>>
>>>>>> These patches are based on commit:
>>>>>> d0ed6a69d3 ("Update version for v5.1.0 release")
>>>>>>
>>>>>> (I tried basing on the latest Qemu commit, but I was having build issues
>>>>>> that level)
>>>>>>
>>>>>> A version of the tree can be found at:
>>>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAMDESE%2Fqemu%2Ftree%2Fsev-es-v11&data=02%7C01%7Cthomas.lendacky%40amd.com%7C8287b3e99fdc41c682fe08d85e244ee0%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637362858215626678&sdata=1o51C3GUrPwodt4yP6ZlkORcrtSfxLUytJtC66OSjSQ%3D&reserved=0
>>>>>>
>>>>>> Changes since v2:
>>>>>> - Add in-kernel irqchip requirement for SEV-ES guests
>>>>>>
>>>>>> Changes since v1:
>>>>>> - Fixed checkpatch.pl errors/warnings
>>>>>>
>>>>>> Tom Lendacky (5):
>>>>>> sev/i386: Add initial support for SEV-ES
>>>>>> sev/i386: Require in-kernel irqchip support for SEV-ES guests
>>>>>> sev/i386: Allow AP booting under SEV-ES
>>>>>> sev/i386: Don't allow a system reset under an SEV-ES guest
>>>>>> sev/i386: Enable an SEV-ES guest based on SEV policy
>>>>>>
>>>>>> accel/kvm/kvm-all.c | 73 ++++++++++++++++++++++++++
>>>>>> accel/stubs/kvm-stub.c | 5 ++
>>>>>> hw/i386/pc_sysfw.c | 10 +++-
>>>>>> include/sysemu/cpus.h | 2 +
>>>>>> include/sysemu/hw_accel.h | 5 ++
>>>>>> include/sysemu/kvm.h | 18 +++++++
>>>>>> include/sysemu/sev.h | 3 ++
>>>>>> softmmu/cpus.c | 5 ++
>>>>>> softmmu/vl.c | 5 +-
>>>>>> target/i386/cpu.c | 1 +
>>>>>> target/i386/kvm.c | 2 +
>>>>>> target/i386/sev-stub.c | 5 ++
>>>>>> target/i386/sev.c | 105 +++++++++++++++++++++++++++++++++++++-
>>>>>> target/i386/sev_i386.h | 1 +
>>>>>> 14 files changed, 236 insertions(+), 4 deletions(-)
>>>>>>
>>>>>> --
>>>>>> 2.28.0
>>>>>>
>>>>
>>
© 2016 - 2024 Red Hat, Inc.