From nobody Sat May 4 08:52:15 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1561971604; cv=none; d=zoho.com; s=zohoarc; b=JXNc0SSQAm/QNy2eAX5/HLPrwjWXzuXrAf3dnkH4AIPitMT11cAjj9iPNZd2H95mPWQg0kztDq2y1Ow3NctwAzp3oScmcqOQmBPhtGsRAZZPDm0UWEPZFORp7VTawjEPWlo4JWEM3NSMYeCp64p9jQuaVX+jkwh6RnPcJKlXYTs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1561971604; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:To:ARC-Authentication-Results; bh=M18Xs6yNB5C5t6A6VnlT7G/TGiGW+g/AbSroHQ7PxCc=; b=PakYFx5Yk5TWRsw4oheM64QgL4/dKIO0kP2W+EX4oKyQsrtDy23rlkxW3wtf8sN647QGfWCEVGYZk06Cz3bvrPGBMTTIa71vh5N9wxokLXjhf11My3014Bt/3HeKM+ZwpagzYLMjFvkGMEwr5hYjhGUmhN54j3rL/D4Em5VCCTw= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1561971604772683.575970218434; Mon, 1 Jul 2019 02:00:04 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 4C98B85546; Mon, 1 Jul 2019 08:59:52 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 1698917CFF; Mon, 1 Jul 2019 08:59:48 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 9F7DE3D87; Mon, 1 Jul 2019 08:59:41 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id x618xSJ0028539 for ; Mon, 1 Jul 2019 04:59:28 -0400 Received: by smtp.corp.redhat.com (Postfix) id 7A59D63B82; Mon, 1 Jul 2019 08:59:28 +0000 (UTC) Received: from beluga.usersys.redhat.com (unknown [10.43.2.166]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8728F6012C; Mon, 1 Jul 2019 08:59:25 +0000 (UTC) From: Erik Skultety To: libvir-list@redhat.com Date: Mon, 1 Jul 2019 10:59:21 +0200 Message-Id: <65612b8d7e409c5f35911cfe0f19a61edaffe8f8.1561971468.git.eskultet@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-loop: libvir-list@redhat.com Cc: Erik Skultety Subject: [libvirt] [PATCH v2] docs: Provide documentation for SEV launch security X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Mon, 01 Jul 2019 08:59:57 +0000 (UTC) Content-Type: text/plain; charset="utf-8" Signed-off-by: Erik Skultety --- docs/launch_security_sev.html.in | 516 +++++++++++++++++++++++++++++++ 1 file changed, 516 insertions(+) create mode 100644 docs/launch_security_sev.html.in diff --git a/docs/launch_security_sev.html.in b/docs/launch_security_sev.ht= ml.in new file mode 100644 index 0000000000..2f0f89d714 --- /dev/null +++ b/docs/launch_security_sev.html.in @@ -0,0 +1,516 @@ + + + + +

Launch security with AMD SEV

+ +
    + +

    + Storage encryption in modern public cloud computing is a common pr= actice. + However, from the point of view of a user of these cloud workloads= , a + significant amount of trust needs to be put in the cloud platform = security as + well as integrity (was the hypervisor tampered?). For this reason = there's ever + rising demand for securing data in use, i.e. memory encryption. + One of the solutions addressing this matter is AMD SEV. +

    + +

    AMD SEV

    +

    + SEV (Secure Encrypted Virtualization) is a feature extension of AM= D's SME (Secure + Memory Encryption) intended for KVM virtual machines which is supp= orted + primarily on AMD's EPYC CPU line. In contrast to SME, SEV uses a u= nique memory encryption + key for each VM. The whole encryption of memory pages is completel= y transparent + to the hypervisor and happens in the AMD firmware. + For more details about the technology itself, you can visit + AMD's developer portal. +

    + +

    Enabling SEV on the host

    +

    + Before VMs can make use of the SEV feature you need to make sure= your + AMD CPU does support SEV. You can check whether SEV is among the= CPU + flags with: +

    + + 
    +$ cat /proc/cpuinfo | grep sev
+...
+sme ssbd sev ibpb
    + +

    + Next step is to enable SEV in the kernel, because it is disabled= by default. + This is done by putting the following onto the kernel command li= ne: +

    + + 
    +mem_encrypt=3Don kvm_amd.sev=3D1
+
    + +

    + To make the changes persistent, append the above to the variable= holding + parameters of the kernel command line in + /etc/default/grub to preserve SEV settings across r= eboots +

    + + 
    +$ cat /etc/default/grub
+...
+GRUB_CMDLINE_LINUX=3D"... mem_encrypt=3Don kvm_amd.sev=3D1"
+$ grub2-mkconfig -o /boot/efi/EFI/<distro>/grub.cfg
    + +

    + mem_encrypt=3Don turns on the SME memory encryption f= eature on + the host which is required for SEV to work. The kvm_amd.sev<= /code> + parameter actually enables SEV in the kvm module. It can be set on= the + command line alongside mem_encrypt like shown above, = or it + can be put into a module config under /etc/modprobe.d/ +

    + + 
    +$ cat /etc/modprobe.d/sev.conf
+options kvm_amd sev=3D1
+
    + +

    + After rebooting the host, you should see SEV being enabled in th= e kernel: +

    + + 
    +$ cat /sys/module/kvm_amd/parameters/sev
+1
+
    + +

    Checking SEV support in the virt stack

    +

    + Note: All of the commands bellow need to be run with root privi= leges. +

    + +

    + First make sure you have the following packages in the specified= versions: +

    + +
      +
    • + libvirt >=3D 4.5.0 (>5.1.0 recommended due to additional SEV b= ugfixes) +
      • +
    • + QEMU >=3D 2.12.0 +
      • +
    +

    + To confirm that the virtualization stack supports SEV, run the f= ollowing: +

    + + 
    +# virsh domcapabilities
+<domainCapabilities>
+...
+  <features>
+    ...
+    <sev supported=3D'yes'>
+      <cbitpos>47</cbitpos>
+      <reducedPhysBits>1</reducedPhysBits>
+    </sev>
+    ...
+  </features>
+</domainCapabilities>
    +

    + Note that if libvirt was already installed and libvirtd running = before enabling SEV in the kernel followed by the host reboot you need to f= orce libvirtd + to re-probe both the host and QEMU capabilities. First stop libv= irtd: +

    + + 
    +# systemctl stop libvirtd.service
+
    + +

    + Now you need to clean the capabilities cache: +

    + + 
    +# rm -f /var/cache/libvirt/qemu/capabilities/*
+
    + +

    + If you now restart libvirtd, it will re-probe the capabilities a= nd if + you now run: +

    + + 
    +# virsh domcapabilities
+
    + +

    + SEV should be listed as supported. If you still see: +

    + + 
    +<sev supported=3D'no'/>
+
    + +

    + it means one of two things: +

      +
    1. + libvirt does support SEV, but either QEMU or the host does n= ot +
      2. +
    2. + you have libvirt <=3D5.1.0 which suffered from getting a + 'Permission denied' on /dev/sev beca= use + of the default permissions on the character device which preve= nted + QEMU from opening it during capabilities probing - you can eit= her + manually tweak the permissions so that QEMU has access to it or + preferably install libvirt 5.1.0 or higher +
      3. +
    +

    + +

    VM Configuration

    +

    + SEV is enabled in the XML by specifying the + <= ;launchSecurity> element. However, specifying launchSecurity<= /code> isn't + enough to boot an SEV VM. Further configuration requirements are d= iscussed + below. +

    + +

    Machine type

    +

    + Even though both Q35 and legacy PC machine types (for PC see also + "virtio") can be used with SEV, usage of the legacy PC machine t= ype is + strongly discouraged, since depending on how your OVMF package w= as + built (e.g. including features like SecureBoot or SMM) Q35 may e= ven be + required. +

    + +
    Q35
    +
    +...
+<os>
+  <type arch=3D'x86_64' machine=3D'pc-q35-3.0'>hvm</type>
+  ...
+</os>
+...
    + +
    i440fx (discouraged)
    + 
    +...
+<os>
+  <type arch=3D'x86_64' machine=3D'pc-i440fx-3.0'>hvm</type>
+  ...
+</os>
+...
+
    + +

    Boot loader

    +

    + SEV is only going to work with OVMF (UEFI), so you'll need to po= int libvirt to + the correct OVMF binary. +

    + 
    +...
+<os>
+  <type arch=3D'x86_64' machine=3D'pc-q35-3.0'>hvm</type>
+  <loader readonly=3D'yes' type=3D'pflash'>/usr/share/edk2/ovmf/OVMF=
_CODE.fd</loader>
+</os>
+...
    + +

    Memory

    +

    + Internally, SEV expects that the encrypted memory pages won't be= swapped out or move + around so the VM memory needs to be pinned in physical RAM which= will be + handled by QEMU. Apart from that, certain memory regions allocat= ed by QEMU + itself (UEFI pflash, device ROMs, video RAM, etc.) have to be en= crypted as + well. This causes a conflict in how libvirt tries to protect the= host. + By default, libvirt enforces a memory hard limit on each VM's cg= roup in order + to protect the host from malicious QEMU to allocate and lock all= the available + memory. This limit corresponds to the total memory allocation fo= r the VM given + by <currentMemory> element. However, trying t= o account for the additional + memory regions QEMU allocates when calculating the limit in an a= utomated manner + is non-deterministic. One way to resolve this is to set the hard= limit manually. + +

    + Note: Figuring out the right number so that your guest boots and= isn't killed is + challenging, but 256MiB extra memory over the total guest RAM sh= ould suffice for + most workloads and may serve as a good starting point. + + For example, a domain with 4GB memory with a 256MiB extra hard l= imit would look + like this: +

    +

    + + 
    +# virsh edit <domain>
+<domain>
+  ...
+  <currentMemory unit=3D'KiB'>4194304</currentMemory>
+  <memtune>
+    <hard_limit unit=3D'KiB'>4456448</hard_limit>
+  </memtune>
+  ...
+</domain>
    +

    + There's another, preferred method of taking care of the limits by + using the<memoryBacking> element along with t= he + <locked/> subelement: +

    + + 
    +<domain>
+  ...
+  <memoryBacking>
+    <locked/>
+  </memoryBacking>
+  ...
+</domain>
    + +

    + What that does is that it tells libvirt not to force any hard li= mit (well, + unlimited) upon the VM cgroup. The obvious advantage is that one= doesn't need + to determine the hard limit for every single SEV-enabled VM. How= ever, there is + a significant security-related drawback to this approach. Since = no hard limit + is applied, a malicious QEMU could perform a DoS attack by locki= ng all of the + host's available memory. The way to avoid this issue and to prot= ect the host is + to enforce a bigger hard limit on the master cgroup containing a= ll of the VMs + - on systemd this is machine.slice. +

    + + 
    +# systemctl set-property machine.slice MemoryHigh=3D<value>
    + +

    + To put even stricter measures in place which would involve the O= OM killer, use + 

    +# systemctl set-property machine.slice MemoryMax=3D<value>
    + instead. Alternatively, you can create a systemd config (don't f= orget + to reload systemd configuration in this case): + 
    +# cat << EOF > /etc/systemd/system.control/machine.slice.d/90-Mem=
oryMax.conf
+MemoryMax=3D<value>
+EOF
    + The trade-off to keep in mind with the second approach is that t= he VMs + can still perform DoS on each other. +

    + +

    Virtio

    +

    + In order to make virtio devices work, we need to enable emulated= IOMMU + on the devices so that virtual DMA can work. +

    + + 
    +# virsh edit <domain>
+<domain>
+  ...
+  <controller type=3D'virtio-serial' index=3D'0'>
+    <driver iommu=3D'on'/>
+  </controller>
+  <controller type=3D'scsi' index=3D'0' model=3D'virtio-scsi'>
+    <driver iommu=3D'on'/>
+  </controller>
+  ...
+  <memballoon model=3D'virtio'>
+    <driver iommu=3D'on'/>
+  </memballoon>
+  <rng model=3D'virtio'>
+    <backend model=3D'random'>/dev/urandom</backend>
+    <driver iommu=3D'on'/>
+  </rng>
+  ...
+<domain>
    + +

    + If you for some reason want to use the legacy PC machine type, fur= ther changes + to the virtio + configuration is required, because SEV will not work with Virtio &= lt;1.0. In + libvirt, this is handled by using the virtio-non-transitional devi= ce model + (libvirt >=3D 5.2.0 required). + +

    + Note: some devices like video devices don't + support non-transitional model, which means that virtio GPU cann= ot be used. +

    +

    + + 
    +<domain>
+  ...
+  <devices>
+    ...
+    <memballoon model=3D'virtio-non-transitional'>
+      <driver iommu=3D'on'/>
+    </memballoon>
+  </devices>
+  ...
+</domain>
    + +

    Limitations

    +

    + Currently, the boot disk cannot be of type virtio-blk, instead, vi= rtio-scsi + needs to be used if virtio is desired. This limitation is expected= to be lifted + with future releases of kernel (the kernel used at the time of wri= ting the + article is 5.0.14). + If you still cannot start an SEV VM, it could be because of wrong = SELinux label on the /dev/sev device with selinux-policy <3= .14.2.40 which prevents QEMU from touching the device. This can be resolved= by upgrading the package, tuning the selinux policy rules manually to allo= w svirt_t to access the device (see audit2allow on how to do t= hat) or putting SELinux into permissive mode (discouraged). +

    + +

    Full domain XML examples

    + +
    Q35 machine
    + 
    +<domain type=3D'kvm'>
+  <name>sev-dummy</name>
+  <memory unit=3D'KiB'>4194304</memory>
+  <currentMemory unit=3D'KiB'>4194304</currentMemory>
+  <memoryBacking>
+    <locked/>
+  </memoryBacking>
+  <vcpu placement=3D'static'>4</vcpu>
+  <os>
+    <type arch=3D'x86_64' machine=3D'pc-q35-3.0'>hvm</type>
+    <loader readonly=3D'yes' type=3D'pflash'>/usr/share/edk2/ovmf/OV=
MF_CODE.fd</loader>
+    <nvram>/var/lib/libvirt/qemu/nvram/sev-dummy_VARS.fd</nvram&g=
t;
+  </os>
+  <features>
+    <acpi/>
+    <apic/>
+    <vmport state=3D'off'/>
+  </features>
+  <cpu mode=3D'host-model' check=3D'partial'>
+    <model fallback=3D'allow'/>
+  </cpu>
+  <clock offset=3D'utc'>
+    <timer name=3D'rtc' tickpolicy=3D'catchup'/>
+    <timer name=3D'pit' tickpolicy=3D'delay'/>
+    <timer name=3D'hpet' present=3D'no'/>
+  </clock>
+  <on_poweroff>destroy</on_poweroff>
+  <on_reboot>restart</on_reboot>
+  <on_crash>destroy</on_crash>
+  <pm>
+    <suspend-to-mem enabled=3D'no'/>
+    <suspend-to-disk enabled=3D'no'/>
+  </pm>
+  <devices>
+    <emulator>/usr/bin/qemu-kvm</emulator>
+    <disk type=3D'file' device=3D'disk'>
+      <driver name=3D'qemu' type=3D'qcow2'/>
+      <source file=3D'/var/lib/libvirt/images/sev-dummy.qcow2'/>
+      <target dev=3D'sda' bus=3D'scsi'/>
+      <boot order=3D'1'/>
+    </disk>
+    <controller type=3D'virtio-serial' index=3D'0'>
+      <driver iommu=3D'on'/>
+    </controller>
+    <controller type=3D'scsi' index=3D'0' model=3D'virtio-scsi'>
+      <driver iommu=3D'on'/>
+    </controller>
+    <interface type=3D'network'>
+      <mac address=3D'52:54:00:cc:56:90'/>
+      <source network=3D'default'/>
+      <model type=3D'virtio'/>
+      <driver iommu=3D'on'/>
+    </interface>
+    <graphics type=3D'spice' autoport=3D'yes'>
+      <listen type=3D'address'/>
+      <gl enable=3D'no'/>
+    </graphics>
+    <video>
+      <model type=3D'qxl'/>
+    </video>
+    <memballoon model=3D'virtio'>
+      <driver iommu=3D'on'/>
+    </memballoon>
+    <rng model=3D'virtio'>
+      <driver iommu=3D'on'/>
+    </rng>
+  </devices>
+  <launchSecurity type=3D'sev'>
+    <cbitpos>47</cbitpos>
+    <reducedPhysBits>1</reducedPhysBits>
+    <policy>0x0003</policy>
+  </launchSecurity>
+</domain>
    + +
    PC-i440fx machine:
    + 
    +<domain type=3D'kvm'>
+  <name>sev-dummy-legacy</name>
+  <memory unit=3D'KiB'>4194304</memory>
+  <currentMemory unit=3D'KiB'>4194304</currentMemory>
+  <memtune>
+    <hard_limit unit=3D'KiB'>5242880</hard_limit>
+  </memtune>
+  <vcpu placement=3D'static'>4</vcpu>
+  <os>
+    <type arch=3D'x86_64' machine=3D'pc-i440fx-3.0'>hvm</type>
+    <loader readonly=3D'yes' type=3D'pflash'>/usr/share/edk2/ovmf/OV=
MF_CODE.fd</loader>
+    <nvram>/var/lib/libvirt/qemu/nvram/sev-dummy_VARS.fd</nvram&g=
t;
+    <boot dev=3D'hd'/>
+  </os>
+  <features>
+  <acpi/>
+  <apic/>
+  <vmport state=3D'off'/>
+  </features>
+  <cpu mode=3D'host-model' check=3D'partial'>
+    <model fallback=3D'allow'/>
+  </cpu>
+  <clock offset=3D'utc'>
+    <timer name=3D'rtc' tickpolicy=3D'catchup'/>
+    <timer name=3D'pit' tickpolicy=3D'delay'/>
+    <timer name=3D'hpet' present=3D'no'/>
+  </clock>
+  <on_poweroff>destroy</on_poweroff>
+  <on_reboot>restart</on_reboot>
+  <on_crash>destroy</on_crash>
+  <pm>
+    <suspend-to-mem enabled=3D'no'/>
+    <suspend-to-disk enabled=3D'no'/>
+  </pm>
+  <devices>
+    <emulator>/usr/bin/qemu-kvm</emulator>
+    <disk type=3D'file' device=3D'disk'>
+      <driver name=3D'qemu' type=3D'qcow2'/>
+      <source file=3D'/var/lib/libvirt/images/sev-dummy-seabios.qcow2'/=
>
+      <target dev=3D'sda' bus=3D'sata'/>
+    </disk>
+    <interface type=3D'network'>
+      <mac address=3D'52:54:00:d8:96:c8'/>
+      <source network=3D'default'/>
+      <model type=3D'virtio-non-transitional'/>
+    </interface>
+    <serial type=3D'pty'>
+      <target type=3D'isa-serial' port=3D'0'>
+        <model name=3D'isa-serial'/>
+      </target>
+    </serial>
+    <console type=3D'pty'>
+      <target type=3D'serial' port=3D'0'/>
+    </console>
+    <input type=3D'tablet' bus=3D'usb'>
+      <address type=3D'usb' bus=3D'0' port=3D'1'/>
+    </input>
+    <input type=3D'mouse' bus=3D'ps2'/>
+    <input type=3D'keyboard' bus=3D'ps2'/>
+    <graphics type=3D'spice' autoport=3D'yes'>
+      <listen type=3D'address'/>
+      <gl enable=3D'no'/>
+    </graphics>
+    <video>
+      <model type=3D'qxl' ram=3D'65536' vram=3D'65536' vgamem=3D'16384'=
 heads=3D'1' primary=3D'yes'/>
+    </video>
+    <memballoon model=3D'virtio-non-transitional'>
+      <driver iommu=3D'on'/>
+    </memballoon>
+      <rng model=3D'virtio-non-transitional'>
+    <driver iommu=3D'on'/>
+    </rng>
+  </devices>
+  <launchSecurity type=3D'sev'>
+    <cbitpos>47</cbitpos>
+    <reducedPhysBits>1</reducedPhysBits>
+    <policy>0x0003</policy>
+  </launchSecurity>
+</domain>
    + + --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list