From nobody Sat Oct 25 11:10:43 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1521156464380958.6526707012238; Thu, 15 Mar 2018 16:27:44 -0700 (PDT) Received: from localhost ([::1]:53798 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ewcHf-0002h6-Ih for importer@patchew.org; Thu, 15 Mar 2018 19:27:43 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48532) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ewcFy-0001hy-JB for qemu-devel@nongnu.org; Thu, 15 Mar 2018 19:25:59 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ewcFu-0000VD-K6 for qemu-devel@nongnu.org; Thu, 15 Mar 2018 19:25:58 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:44672) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ewcFu-0000US-B0 for qemu-devel@nongnu.org; Thu, 15 Mar 2018 19:25:54 -0400 Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w2FNM7KG133537 for ; Thu, 15 Mar 2018 19:25:50 -0400 Received: from e32.co.us.ibm.com (e32.co.us.ibm.com [32.97.110.150]) by mx0a-001b2d01.pphosted.com with ESMTP id 2gqxyf88h9-1 (version=TLSv1.2 cipher=AES256-SHA256 bits=256 verify=NOT) for ; Thu, 15 Mar 2018 19:25:49 -0400 Received: from localhost by e32.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 15 Mar 2018 17:25:14 -0600 Received: from b03cxnp07028.gho.boulder.ibm.com (9.17.130.15) by e32.co.us.ibm.com (192.168.1.132) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Thu, 15 Mar 2018 17:25:10 -0600 Received: from b03ledav002.gho.boulder.ibm.com (b03ledav002.gho.boulder.ibm.com [9.17.130.233]) by b03cxnp07028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w2FNP9Le12910924; Thu, 15 Mar 2018 16:25:09 -0700 Received: from b03ledav002.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E341013603A; Thu, 15 Mar 2018 17:25:08 -0600 (MDT) Received: from localhost.localdomain (unknown [9.80.217.151]) by b03ledav002.gho.boulder.ibm.com (Postfix) with ESMTPS id A9C0D136043; Thu, 15 Mar 2018 17:25:06 -0600 (MDT) From: Tony Krowiak To: qemu-devel@nongnu.org Date: Thu, 15 Mar 2018 19:24:54 -0400 X-Mailer: git-send-email 1.7.1 In-Reply-To: <1521156300-19296-1-git-send-email-akrowiak@linux.vnet.ibm.com> References: <1521156300-19296-1-git-send-email-akrowiak@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 18031523-0004-0000-0000-000013CEE144 X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00008681; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000254; SDB=6.01003605; UDB=6.00510771; IPR=6.00782930; MB=3.00020058; MTD=3.00000008; XFM=3.00000015; UTC=2018-03-15 23:25:13 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18031523-0005-0000-0000-0000867ADFD4 Message-Id: <1521156300-19296-2-git-send-email-akrowiak@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2018-03-15_11:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1803150253 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 Subject: [Qemu-devel] [PATCH v3 1/7] linux-headers: linux header updates for AP support X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mjrosato@linux.vnet.ibm.com, peter.maydell@linaro.org, pasic@linux.vnet.ibm.com, alifm@linux.vnet.ibm.com, eskultet@redhat.com, david@redhat.com, pmorel@linux.vnet.ibm.com, cohuck@redhat.com, heiko.carstens@de.ibm.com, alex.williamson@redhat.com, agraf@suse.de, borntraeger@de.ibm.com, qemu-s390x@nongnu.org, akrowiak@linux.vnet.ibm.com, jjherne@linux.vnet.ibm.com, schwidefsky@de.ibm.com, pbonzini@redhat.com, bjsdjshi@linux.vnet.ibm.com, eric.auger@redhat.com, rth@twiddle.net Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Updates the linux header files in preparation for introduction of the VFIO AP device: * Added a feature ID to indicate AP facilities are installed * Added a device attribute to the KVM_S390_VM_CRYPTO group to indicate whether AP instructions are to be interpreted * Added VFIO device information for AP devices Signed-off-by: Tony Krowiak --- linux-headers/asm-s390/kvm.h | 2 ++ linux-headers/linux/vfio.h | 2 ++ 2 files changed, 4 insertions(+), 0 deletions(-) diff --git a/linux-headers/asm-s390/kvm.h b/linux-headers/asm-s390/kvm.h index 11def14..391b250 100644 --- a/linux-headers/asm-s390/kvm.h +++ b/linux-headers/asm-s390/kvm.h @@ -130,6 +130,7 @@ struct kvm_s390_vm_cpu_machine { #define KVM_S390_VM_CPU_FEAT_PFMFI 11 #define KVM_S390_VM_CPU_FEAT_SIGPIF 12 #define KVM_S390_VM_CPU_FEAT_KSS 13 +#define KVM_S390_VM_CPU_FEAT_AP 14 struct kvm_s390_vm_cpu_feat { __u64 feat[16]; }; @@ -160,6 +161,7 @@ struct kvm_s390_vm_cpu_subfunc { #define KVM_S390_VM_CRYPTO_ENABLE_DEA_KW 1 #define KVM_S390_VM_CRYPTO_DISABLE_AES_KW 2 #define KVM_S390_VM_CRYPTO_DISABLE_DEA_KW 3 +#define KVM_S390_VM_CRYPTO_INTERPRET_AP 4 =20 /* kvm attributes for migration mode */ #define KVM_S390_VM_MIGRATION_STOP 0 diff --git a/linux-headers/linux/vfio.h b/linux-headers/linux/vfio.h index 4312e96..91298dc 100644 --- a/linux-headers/linux/vfio.h +++ b/linux-headers/linux/vfio.h @@ -200,6 +200,7 @@ struct vfio_device_info { #define VFIO_DEVICE_FLAGS_PLATFORM (1 << 2) /* vfio-platform device */ #define VFIO_DEVICE_FLAGS_AMBA (1 << 3) /* vfio-amba device */ #define VFIO_DEVICE_FLAGS_CCW (1 << 4) /* vfio-ccw device */ +#define VFIO_DEVICE_FLAGS_AP (1 << 5) /* vfio-ap device */ __u32 num_regions; /* Max region index + 1 */ __u32 num_irqs; /* Max IRQ index + 1 */ }; @@ -215,6 +216,7 @@ struct vfio_device_info { #define VFIO_DEVICE_API_PLATFORM_STRING "vfio-platform" #define VFIO_DEVICE_API_AMBA_STRING "vfio-amba" #define VFIO_DEVICE_API_CCW_STRING "vfio-ccw" +#define VFIO_DEVICE_API_AP_STRING "vfio-ap" =20 /** * VFIO_DEVICE_GET_REGION_INFO - _IOWR(VFIO_TYPE, VFIO_BASE + 8, --=20 1.7.1 From nobody Sat Oct 25 11:10:43 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1521156422596380.81586188813696; Thu, 15 Mar 2018 16:27:02 -0700 (PDT) Received: from localhost ([::1]:53796 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ewcGq-00022A-5t for importer@patchew.org; Thu, 15 Mar 2018 19:26:52 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48289) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ewcFQ-0001TT-N1 for qemu-devel@nongnu.org; Thu, 15 Mar 2018 19:25:25 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ewcFM-0008Uy-Iu for qemu-devel@nongnu.org; Thu, 15 Mar 2018 19:25:24 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:41042 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ewcFM-0008Ul-E4 for qemu-devel@nongnu.org; Thu, 15 Mar 2018 19:25:20 -0400 Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w2FNLLwl002405 for ; Thu, 15 Mar 2018 19:25:19 -0400 Received: from e32.co.us.ibm.com (e32.co.us.ibm.com [32.97.110.150]) by mx0a-001b2d01.pphosted.com with ESMTP id 2gr0f348yv-1 (version=TLSv1.2 cipher=AES256-SHA256 bits=256 verify=NOT) for ; Thu, 15 Mar 2018 19:25:19 -0400 Received: from localhost by e32.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 15 Mar 2018 17:25:18 -0600 Received: from b03cxnp08027.gho.boulder.ibm.com (9.17.130.19) by e32.co.us.ibm.com (192.168.1.132) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Thu, 15 Mar 2018 17:25:14 -0600 Received: from b03ledav002.gho.boulder.ibm.com (b03ledav002.gho.boulder.ibm.com [9.17.130.233]) by b03cxnp08027.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w2FNPCkB12124596; Thu, 15 Mar 2018 16:25:12 -0700 Received: from b03ledav002.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6FD4A136049; Thu, 15 Mar 2018 17:25:12 -0600 (MDT) Received: from localhost.localdomain (unknown [9.80.217.151]) by b03ledav002.gho.boulder.ibm.com (Postfix) with ESMTPS id 1E593136044; Thu, 15 Mar 2018 17:25:10 -0600 (MDT) From: Tony Krowiak To: qemu-devel@nongnu.org Date: Thu, 15 Mar 2018 19:24:55 -0400 X-Mailer: git-send-email 1.7.1 In-Reply-To: <1521156300-19296-1-git-send-email-akrowiak@linux.vnet.ibm.com> References: <1521156300-19296-1-git-send-email-akrowiak@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 18031523-0004-0000-0000-000013CEE148 X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00008681; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000254; SDB=6.01003605; UDB=6.00510771; IPR=6.00782930; MB=3.00020058; MTD=3.00000008; XFM=3.00000015; UTC=2018-03-15 23:25:16 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18031523-0005-0000-0000-0000867ADFE1 Message-Id: <1521156300-19296-3-git-send-email-akrowiak@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2018-03-15_11:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1803150253 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 Subject: [Qemu-devel] [PATCH v3 2/7] s390x/ap: base Adjunct Processor (AP) object X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mjrosato@linux.vnet.ibm.com, peter.maydell@linaro.org, pasic@linux.vnet.ibm.com, alifm@linux.vnet.ibm.com, eskultet@redhat.com, david@redhat.com, pmorel@linux.vnet.ibm.com, cohuck@redhat.com, heiko.carstens@de.ibm.com, alex.williamson@redhat.com, agraf@suse.de, borntraeger@de.ibm.com, qemu-s390x@nongnu.org, akrowiak@linux.vnet.ibm.com, jjherne@linux.vnet.ibm.com, schwidefsky@de.ibm.com, pbonzini@redhat.com, bjsdjshi@linux.vnet.ibm.com, eric.auger@redhat.com, rth@twiddle.net Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This patch introduces the base object for an AP device. Signed-off-by: Tony Krowiak --- hw/s390x/Makefile.objs | 1 + hw/s390x/ap-device.c | 38 ++++++++++++++++++++++++++++++++++++++ include/hw/s390x/ap-device.h | 38 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 77 insertions(+), 0 deletions(-) create mode 100644 hw/s390x/ap-device.c create mode 100644 include/hw/s390x/ap-device.h diff --git a/hw/s390x/Makefile.objs b/hw/s390x/Makefile.objs index dc704b5..3247a07 100644 --- a/hw/s390x/Makefile.objs +++ b/hw/s390x/Makefile.objs @@ -17,3 +17,4 @@ obj-y +=3D s390-stattrib.o obj-$(CONFIG_KVM) +=3D s390-skeys-kvm.o obj-$(CONFIG_KVM) +=3D s390-stattrib-kvm.o obj-y +=3D s390-ccw.o +obj-y +=3D ap-device.o diff --git a/hw/s390x/ap-device.c b/hw/s390x/ap-device.c new file mode 100644 index 0000000..448f6db --- /dev/null +++ b/hw/s390x/ap-device.c @@ -0,0 +1,38 @@ +/* + * Adjunct Processor (AP) matrix device + * + * Copyright 2018 IBM Corp. + * Author(s): Tony Krowiak + * + * This work is licensed under the terms of the GNU GPL, version 2 or (at + * your option) any later version. See the COPYING file in the top-level + * directory. + */ +#include "qemu/osdep.h" +#include "qemu/module.h" +#include "qapi/error.h" +#include "hw/qdev.h" +#include "hw/s390x/ap-device.h" + +static void ap_class_init(ObjectClass *klass, void *data) +{ + DeviceClass *dc =3D DEVICE_CLASS(klass); + + dc->desc =3D "AP device class"; +} + +static const TypeInfo ap_device_info =3D { + .name =3D AP_DEVICE_TYPE, + .parent =3D TYPE_DEVICE, + .instance_size =3D sizeof(APDevice), + .class_size =3D sizeof(APDeviceClass), + .class_init =3D ap_class_init, + .abstract =3D true, +}; + +static void ap_device_register(void) +{ + type_register_static(&ap_device_info); +} + +type_init(ap_device_register) diff --git a/include/hw/s390x/ap-device.h b/include/hw/s390x/ap-device.h new file mode 100644 index 0000000..693df90 --- /dev/null +++ b/include/hw/s390x/ap-device.h @@ -0,0 +1,38 @@ +/* + * Adjunct Processor (AP) matrix device interfaces + * + * Copyright 2018 IBM Corp. + * Author(s): Tony Krowiak + * + * This work is licensed under the terms of the GNU GPL, version 2 or (at + * your option) any later version. See the COPYING file in the top-level + * directory. + */ +#ifndef HW_S390X_AP_DEVICE_H +#define HW_S390X_AP_DEVICE_H + +#define AP_DEVICE_TYPE "ap-device" + +typedef struct APDevice { + DeviceState parent_obj; +} APDevice; + +typedef struct APDeviceClass { + DeviceClass parent_class; +} APDeviceClass; + +static inline APDevice *to_ap_dev(DeviceState *dev) +{ + return container_of(dev, APDevice, parent_obj); +} + +#define AP_DEVICE(obj) \ + OBJECT_CHECK(APDevice, (obj), AP_DEVICE_TYPE) + +#define AP_DEVICE_GET_CLASS(obj) \ + OBJECT_GET_CLASS(APDeviceClass, (obj), AP_DEVICE_TYPE) + +#define AP_DEVICE_CLASS(klass) \ + OBJECT_CLASS_CHECK(APDeviceClass, (klass), AP_DEVICE_TYPE) + +#endif /* HW_S390X_AP_DEVICE_H */ --=20 1.7.1 From nobody Sat Oct 25 11:10:43 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 15211566520031016.8213163070234; Thu, 15 Mar 2018 16:30:52 -0700 (PDT) Received: from localhost ([::1]:53826 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ewcKh-0005DU-6q for importer@patchew.org; Thu, 15 Mar 2018 19:30:51 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49282) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ewcJG-0004I3-4E for qemu-devel@nongnu.org; Thu, 15 Mar 2018 19:29:26 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ewcJB-0002TQ-VF for qemu-devel@nongnu.org; Thu, 15 Mar 2018 19:29:22 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:52496) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ewcJB-0002T4-NE for qemu-devel@nongnu.org; Thu, 15 Mar 2018 19:29:17 -0400 Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w2FNLuma060341 for ; Thu, 15 Mar 2018 19:29:16 -0400 Received: from e35.co.us.ibm.com (e35.co.us.ibm.com [32.97.110.153] (may be forged)) by mx0a-001b2d01.pphosted.com with ESMTP id 2gr05pvwsf-1 (version=TLSv1.2 cipher=AES256-SHA256 bits=256 verify=NOT) for ; Thu, 15 Mar 2018 19:29:16 -0400 Received: from localhost by e35.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 15 Mar 2018 17:25:21 -0600 Received: from b03cxnp08025.gho.boulder.ibm.com (9.17.130.17) by e35.co.us.ibm.com (192.168.1.135) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Thu, 15 Mar 2018 17:25:17 -0600 Received: from b03ledav002.gho.boulder.ibm.com (b03ledav002.gho.boulder.ibm.com [9.17.130.233]) by b03cxnp08025.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w2FNPFCU12190016; Thu, 15 Mar 2018 16:25:15 -0700 Received: from b03ledav002.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CF6C213604D; Thu, 15 Mar 2018 17:25:15 -0600 (MDT) Received: from localhost.localdomain (unknown [9.80.217.151]) by b03ledav002.gho.boulder.ibm.com (Postfix) with ESMTPS id 953D9136043; Thu, 15 Mar 2018 17:25:13 -0600 (MDT) From: Tony Krowiak To: qemu-devel@nongnu.org Date: Thu, 15 Mar 2018 19:24:56 -0400 X-Mailer: git-send-email 1.7.1 In-Reply-To: <1521156300-19296-1-git-send-email-akrowiak@linux.vnet.ibm.com> References: <1521156300-19296-1-git-send-email-akrowiak@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 18031523-0012-0000-0000-000015E7DC24 X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00008681; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000254; SDB=6.01003605; UDB=6.00510771; IPR=6.00782931; MB=3.00020058; MTD=3.00000008; XFM=3.00000015; UTC=2018-03-15 23:25:20 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18031523-0013-0000-0000-000051E32AF4 Message-Id: <1521156300-19296-4-git-send-email-akrowiak@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2018-03-15_11:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1803150253 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 Subject: [Qemu-devel] [PATCH v3 3/7] s390x/cpumodel: Set up CPU model for AP device support X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mjrosato@linux.vnet.ibm.com, peter.maydell@linaro.org, pasic@linux.vnet.ibm.com, alifm@linux.vnet.ibm.com, eskultet@redhat.com, david@redhat.com, pmorel@linux.vnet.ibm.com, cohuck@redhat.com, heiko.carstens@de.ibm.com, alex.williamson@redhat.com, agraf@suse.de, borntraeger@de.ibm.com, qemu-s390x@nongnu.org, akrowiak@linux.vnet.ibm.com, jjherne@linux.vnet.ibm.com, schwidefsky@de.ibm.com, pbonzini@redhat.com, bjsdjshi@linux.vnet.ibm.com, eric.auger@redhat.com, rth@twiddle.net Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" A new CPU model feature and two new CPU model facilities are introduced to support AP devices for a KVM guest. CPU model features: 1. The KVM_S390_VM_CPU_FEAT_AP CPU model feature indicates that AP facilities are installed. This feature will be enabled by the kernel only if the AP facilities are installed on the linux host. This feature must be turned on from userspace to access AP devices from the KVM guest. The QEMU command line to turn this feature looks something like this: qemu-system-s390x ... -cpu xxx,ap=3Don CPU model facilities: 1. The S390_FEAT_AP_QUERY_CONFIG_INFO feature indicates the AP Query Configuration Information (QCI) facility is installed. This feature will be enabled by the kernel only if the QCI is installed on the host. 2. The S390_FEAT_AP_FACILITY_TEST feature indicates that the AP Facility Test (APFT) facility is installed. This feature will be enabled by the kernel only if the APFT facility is installed on the host. Signed-off-by: Tony Krowiak --- target/s390x/cpu_features.c | 3 +++ target/s390x/cpu_features_def.h | 3 +++ target/s390x/cpu_models.c | 2 ++ target/s390x/gen-features.c | 3 +++ target/s390x/kvm.c | 1 + 5 files changed, 12 insertions(+), 0 deletions(-) diff --git a/target/s390x/cpu_features.c b/target/s390x/cpu_features.c index a5619f2..1abe987 100644 --- a/target/s390x/cpu_features.c +++ b/target/s390x/cpu_features.c @@ -36,8 +36,10 @@ static const S390FeatDef s390_features[] =3D { FEAT_INIT("srs", S390_FEAT_TYPE_STFL, 9, "Sense-running-status facilit= y"), FEAT_INIT("csske", S390_FEAT_TYPE_STFL, 10, "Conditional-SSKE facility= "), FEAT_INIT("ctop", S390_FEAT_TYPE_STFL, 11, "Configuration-topology fac= ility"), + FEAT_INIT("qci", S390_FEAT_TYPE_STFL, 12, "Query AP Configuration faci= lity"), FEAT_INIT("ipter", S390_FEAT_TYPE_STFL, 13, "IPTE-range facility"), FEAT_INIT("nonqks", S390_FEAT_TYPE_STFL, 14, "Nonquiescing key-setting= facility"), + FEAT_INIT("apft", S390_FEAT_TYPE_STFL, 15, "Adjunct Processor Faciliti= es Test facility"), FEAT_INIT("etf2", S390_FEAT_TYPE_STFL, 16, "Extended-translation facil= ity 2"), FEAT_INIT("msa-base", S390_FEAT_TYPE_STFL, 17, "Message-security-assis= t facility (excluding subfunctions)"), FEAT_INIT("ldisp", S390_FEAT_TYPE_STFL, 18, "Long-displacement facilit= y"), @@ -125,6 +127,7 @@ static const S390FeatDef s390_features[] =3D { =20 FEAT_INIT("dateh2", S390_FEAT_TYPE_MISC, 0, "DAT-enhancement facility = 2"), FEAT_INIT("cmm", S390_FEAT_TYPE_MISC, 0, "Collaborative-memory-managem= ent facility"), + FEAT_INIT("ap", S390_FEAT_TYPE_MISC, 0, "AP facilities installed"), =20 FEAT_INIT("plo-cl", S390_FEAT_TYPE_PLO, 0, "PLO Compare and load (32 b= it in general registers)"), FEAT_INIT("plo-clg", S390_FEAT_TYPE_PLO, 1, "PLO Compare and load (64 = bit in parameter list)"), diff --git a/target/s390x/cpu_features_def.h b/target/s390x/cpu_features_de= f.h index 7c5915c..8998b65 100644 --- a/target/s390x/cpu_features_def.h +++ b/target/s390x/cpu_features_def.h @@ -27,8 +27,10 @@ typedef enum { S390_FEAT_SENSE_RUNNING_STATUS, S390_FEAT_CONDITIONAL_SSKE, S390_FEAT_CONFIGURATION_TOPOLOGY, + S390_FEAT_AP_QUERY_CONFIG_INFO, S390_FEAT_IPTE_RANGE, S390_FEAT_NONQ_KEY_SETTING, + S390_FEAT_AP_FACILITIES_TEST, S390_FEAT_EXTENDED_TRANSLATION_2, S390_FEAT_MSA, S390_FEAT_LONG_DISPLACEMENT, @@ -118,6 +120,7 @@ typedef enum { /* Misc */ S390_FEAT_DAT_ENH_2, S390_FEAT_CMM, + S390_FEAT_AP, =20 /* PLO */ S390_FEAT_PLO_CL, diff --git a/target/s390x/cpu_models.c b/target/s390x/cpu_models.c index cf82589..7e2af09 100644 --- a/target/s390x/cpu_models.c +++ b/target/s390x/cpu_models.c @@ -769,6 +769,8 @@ static void check_consistency(const S390CPUModel *model) { S390_FEAT_PRNO_TRNG_QRTCR, S390_FEAT_MSA_EXT_5 }, { S390_FEAT_PRNO_TRNG, S390_FEAT_MSA_EXT_5 }, { S390_FEAT_SIE_KSS, S390_FEAT_SIE_F2 }, + { S390_FEAT_AP_QUERY_CONFIG_INFO, S390_FEAT_AP }, + { S390_FEAT_AP_FACILITIES_TEST, S390_FEAT_AP }, }; int i; =20 diff --git a/target/s390x/gen-features.c b/target/s390x/gen-features.c index 0cdbc15..0d5b0f7 100644 --- a/target/s390x/gen-features.c +++ b/target/s390x/gen-features.c @@ -447,6 +447,9 @@ static uint16_t full_GEN12_GA1[] =3D { S390_FEAT_ADAPTER_INT_SUPPRESSION, S390_FEAT_EDAT_2, S390_FEAT_SIDE_EFFECT_ACCESS_ESOP2, + S390_FEAT_AP_QUERY_CONFIG_INFO, + S390_FEAT_AP_FACILITIES_TEST, + S390_FEAT_AP, }; =20 static uint16_t full_GEN12_GA2[] =3D { diff --git a/target/s390x/kvm.c b/target/s390x/kvm.c index fbc887e..33e5ec3 100644 --- a/target/s390x/kvm.c +++ b/target/s390x/kvm.c @@ -2178,6 +2178,7 @@ static int kvm_to_feat[][2] =3D { { KVM_S390_VM_CPU_FEAT_PFMFI, S390_FEAT_SIE_PFMFI}, { KVM_S390_VM_CPU_FEAT_SIGPIF, S390_FEAT_SIE_SIGPIF}, { KVM_S390_VM_CPU_FEAT_KSS, S390_FEAT_SIE_KSS}, + { KVM_S390_VM_CPU_FEAT_AP, S390_FEAT_AP}, }; =20 static int query_cpu_feat(S390FeatBitmap features) --=20 1.7.1 From nobody Sat Oct 25 11:10:43 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1521156596156493.01531110910446; Thu, 15 Mar 2018 16:29:56 -0700 (PDT) Received: from localhost ([::1]:53809 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ewcJn-0004KO-BN for importer@patchew.org; Thu, 15 Mar 2018 19:29:55 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48688) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ewcGc-0002DG-Qw for qemu-devel@nongnu.org; Thu, 15 Mar 2018 19:26:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ewcGY-0000qY-Rl for qemu-devel@nongnu.org; Thu, 15 Mar 2018 19:26:38 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:43452) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ewcGY-0000lq-Jv for qemu-devel@nongnu.org; Thu, 15 Mar 2018 19:26:34 -0400 Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w2FNLt5Q060338 for ; Thu, 15 Mar 2018 19:26:18 -0400 Received: from e35.co.us.ibm.com (e35.co.us.ibm.com [32.97.110.153] (may be forged)) by mx0a-001b2d01.pphosted.com with ESMTP id 2gr05pvwvs-1 (version=TLSv1.2 cipher=AES256-SHA256 bits=256 verify=NOT) for ; Thu, 15 Mar 2018 19:26:17 -0400 Received: from localhost by e35.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 15 Mar 2018 17:25:25 -0600 Received: from b03cxnp07028.gho.boulder.ibm.com (9.17.130.15) by e35.co.us.ibm.com (192.168.1.135) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Thu, 15 Mar 2018 17:25:20 -0600 Received: from b03ledav002.gho.boulder.ibm.com (b03ledav002.gho.boulder.ibm.com [9.17.130.233]) by b03cxnp07028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w2FNPJ5s14745906; Thu, 15 Mar 2018 16:25:19 -0700 Received: from b03ledav002.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4DC3313603A; Thu, 15 Mar 2018 17:25:19 -0600 (MDT) Received: from localhost.localdomain (unknown [9.80.217.151]) by b03ledav002.gho.boulder.ibm.com (Postfix) with ESMTPS id 0987F136049; Thu, 15 Mar 2018 17:25:16 -0600 (MDT) From: Tony Krowiak To: qemu-devel@nongnu.org Date: Thu, 15 Mar 2018 19:24:57 -0400 X-Mailer: git-send-email 1.7.1 In-Reply-To: <1521156300-19296-1-git-send-email-akrowiak@linux.vnet.ibm.com> References: <1521156300-19296-1-git-send-email-akrowiak@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 18031523-0012-0000-0000-000015E7DC2B X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00008681; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000254; SDB=6.01003605; UDB=6.00510771; IPR=6.00782931; MB=3.00020058; MTD=3.00000008; XFM=3.00000015; UTC=2018-03-15 23:25:24 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18031523-0013-0000-0000-000051E32B01 Message-Id: <1521156300-19296-5-git-send-email-akrowiak@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2018-03-15_11:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1803150253 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 Subject: [Qemu-devel] [PATCH v3 4/7] s390x/kvm: interface to interpret AP instructions X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mjrosato@linux.vnet.ibm.com, peter.maydell@linaro.org, pasic@linux.vnet.ibm.com, alifm@linux.vnet.ibm.com, eskultet@redhat.com, david@redhat.com, pmorel@linux.vnet.ibm.com, cohuck@redhat.com, heiko.carstens@de.ibm.com, alex.williamson@redhat.com, agraf@suse.de, borntraeger@de.ibm.com, qemu-s390x@nongnu.org, akrowiak@linux.vnet.ibm.com, jjherne@linux.vnet.ibm.com, schwidefsky@de.ibm.com, pbonzini@redhat.com, bjsdjshi@linux.vnet.ibm.com, eric.auger@redhat.com, rth@twiddle.net Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The VFIO AP device exploits interpretive execution of AP instructions (APIE). APIE is enabled by setting a device attribute via the KVM_SET_DEVICE_ATTR ioctl. Signed-off-by: Tony Krowiak --- target/s390x/kvm.c | 16 ++++++++++++++++ target/s390x/kvm_s390x.h | 2 ++ 2 files changed, 18 insertions(+), 0 deletions(-) diff --git a/target/s390x/kvm.c b/target/s390x/kvm.c index 33e5ec3..2812e28 100644 --- a/target/s390x/kvm.c +++ b/target/s390x/kvm.c @@ -277,6 +277,22 @@ static void kvm_s390_init_dea_kw(void) } } =20 +int kvm_s390_set_interpret_ap(uint8_t enable) +{ + struct kvm_device_attr attribute =3D { + .group =3D KVM_S390_VM_CRYPTO, + .attr =3D KVM_S390_VM_CRYPTO_INTERPRET_AP, + .addr =3D 1, + }; + + if (!kvm_vm_check_attr(kvm_state, KVM_S390_VM_CRYPTO, + KVM_S390_VM_CRYPTO_INTERPRET_AP)) { + return -EOPNOTSUPP; + } + + return kvm_vm_ioctl(kvm_state, KVM_SET_DEVICE_ATTR, &attribute); +} + void kvm_s390_crypto_reset(void) { if (s390_has_feat(S390_FEAT_MSA_EXT_3)) { diff --git a/target/s390x/kvm_s390x.h b/target/s390x/kvm_s390x.h index 34ee7e7..0d6c6e7 100644 --- a/target/s390x/kvm_s390x.h +++ b/target/s390x/kvm_s390x.h @@ -40,4 +40,6 @@ void kvm_s390_crypto_reset(void); void kvm_s390_restart_interrupt(S390CPU *cpu); void kvm_s390_stop_interrupt(S390CPU *cpu); =20 +int kvm_s390_set_interpret_ap(uint8_t enable); + #endif /* KVM_S390X_H */ --=20 1.7.1 From nobody Sat Oct 25 11:10:43 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1521156466604271.94884219742664; Thu, 15 Mar 2018 16:27:46 -0700 (PDT) Received: from localhost ([::1]:53799 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ewcHh-0002hy-Ld for importer@patchew.org; Thu, 15 Mar 2018 19:27:45 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48534) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ewcFy-0001iA-SM for qemu-devel@nongnu.org; Thu, 15 Mar 2018 19:26:00 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ewcFu-0000VK-Pw for qemu-devel@nongnu.org; Thu, 15 Mar 2018 19:25:58 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:44698) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ewcFu-0000UW-H1 for qemu-devel@nongnu.org; Thu, 15 Mar 2018 19:25:54 -0400 Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w2FNOqhc137806 for ; Thu, 15 Mar 2018 19:25:53 -0400 Received: from e37.co.us.ibm.com (e37.co.us.ibm.com [32.97.110.158]) by mx0a-001b2d01.pphosted.com with ESMTP id 2gqxyf88y5-1 (version=TLSv1.2 cipher=AES256-SHA256 bits=256 verify=NOT) for ; Thu, 15 Mar 2018 19:25:52 -0400 Received: from localhost by e37.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 15 Mar 2018 17:25:29 -0600 Received: from b03cxnp08026.gho.boulder.ibm.com (9.17.130.18) by e37.co.us.ibm.com (192.168.1.137) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Thu, 15 Mar 2018 17:25:24 -0600 Received: from b03ledav002.gho.boulder.ibm.com (b03ledav002.gho.boulder.ibm.com [9.17.130.233]) by b03cxnp08026.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w2FNPM3U12190206; Thu, 15 Mar 2018 16:25:22 -0700 Received: from b03ledav002.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BD61413603A; Thu, 15 Mar 2018 17:25:22 -0600 (MDT) Received: from localhost.localdomain (unknown [9.80.217.151]) by b03ledav002.gho.boulder.ibm.com (Postfix) with ESMTPS id 72594136043; Thu, 15 Mar 2018 17:25:20 -0600 (MDT) From: Tony Krowiak To: qemu-devel@nongnu.org Date: Thu, 15 Mar 2018 19:24:58 -0400 X-Mailer: git-send-email 1.7.1 In-Reply-To: <1521156300-19296-1-git-send-email-akrowiak@linux.vnet.ibm.com> References: <1521156300-19296-1-git-send-email-akrowiak@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 18031523-0024-0000-0000-00001815E353 X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00008681; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000254; SDB=6.01003604; UDB=6.00510770; IPR=6.00782931; MB=3.00020058; MTD=3.00000008; XFM=3.00000015; UTC=2018-03-15 23:25:28 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18031523-0025-0000-0000-00004F1ED59E Message-Id: <1521156300-19296-6-git-send-email-akrowiak@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2018-03-15_11:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=4 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1803150253 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 Subject: [Qemu-devel] [PATCH v3 5/7] s390x/vfio: ap: Introduce VFIO AP device X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mjrosato@linux.vnet.ibm.com, peter.maydell@linaro.org, pasic@linux.vnet.ibm.com, alifm@linux.vnet.ibm.com, eskultet@redhat.com, david@redhat.com, pmorel@linux.vnet.ibm.com, cohuck@redhat.com, heiko.carstens@de.ibm.com, alex.williamson@redhat.com, agraf@suse.de, borntraeger@de.ibm.com, qemu-s390x@nongnu.org, akrowiak@linux.vnet.ibm.com, jjherne@linux.vnet.ibm.com, schwidefsky@de.ibm.com, pbonzini@redhat.com, bjsdjshi@linux.vnet.ibm.com, eric.auger@redhat.com, rth@twiddle.net Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Introduces a VFIO based AP device. The device is defined via the QEMU command line by specifying: -device vfio-ap,sysfsdev=3D The mediated matrix device is created by the VFIO AP device driver by writing a UUID to a sysfs attribute file (see docs/vfio-ap.txt). The mediated matrix device will be named after the UUID. Symbolic links to the $uuid are created in many places, so the path to the mediated matrix device $uuid can be specified in any of the following ways: /sys/devices/vfio_ap/matrix/$uuid /sys/devices/vfio_ap/matrix/mdev_supported_types/vfio_ap-passthrough/device= s/$uuid /sys/bus/mdev/devices/$uuid /sys/bus/mdev/drivers/vfio_mdev/$uuid When the vfio-ap device is realized, it acquires and opens the VFIO iommu group to which the mediated matrix device is bound. This causes a VFIO group notification event to be signaled. The vfio_ap device driver's group notification handler will get called at which time the device driver will configure the the AP devices to which the guest will be granted access. Signed-off-by: Tony Krowiak --- default-configs/s390x-softmmu.mak | 1 + hw/vfio/Makefile.objs | 1 + hw/vfio/ap.c | 184 +++++++++++++++++++++++++++++++++= ++++ include/hw/vfio/vfio-common.h | 1 + 4 files changed, 187 insertions(+), 0 deletions(-) create mode 100644 hw/vfio/ap.c diff --git a/default-configs/s390x-softmmu.mak b/default-configs/s390x-soft= mmu.mak index 2f4bfe7..0b784b6 100644 --- a/default-configs/s390x-softmmu.mak +++ b/default-configs/s390x-softmmu.mak @@ -9,3 +9,4 @@ CONFIG_S390_FLIC=3Dy CONFIG_S390_FLIC_KVM=3D$(CONFIG_KVM) CONFIG_VFIO_CCW=3D$(CONFIG_LINUX) CONFIG_WDT_DIAG288=3Dy +CONFIG_VFIO_AP=3D$(CONFIG_LINUX) diff --git a/hw/vfio/Makefile.objs b/hw/vfio/Makefile.objs index c3ab909..7300860 100644 --- a/hw/vfio/Makefile.objs +++ b/hw/vfio/Makefile.objs @@ -6,4 +6,5 @@ obj-$(CONFIG_SOFTMMU) +=3D platform.o obj-$(CONFIG_VFIO_XGMAC) +=3D calxeda-xgmac.o obj-$(CONFIG_VFIO_AMD_XGBE) +=3D amd-xgbe.o obj-$(CONFIG_SOFTMMU) +=3D spapr.o +obj-$(CONFIG_VFIO_AP) +=3D ap.o endif diff --git a/hw/vfio/ap.c b/hw/vfio/ap.c new file mode 100644 index 0000000..b397bb1 --- /dev/null +++ b/hw/vfio/ap.c @@ -0,0 +1,184 @@ +/* + * VFIO based AP matrix device assignment + * + * Copyright 2018 IBM Corp. + * Author(s): Tony Krowiak + * + * This work is licensed under the terms of the GNU GPL, version 2 or (at + * your option) any later version. See the COPYING file in the top-level + * directory. + */ + +#include +#include +#include "qemu/osdep.h" +#include "qapi/error.h" +#include "hw/sysbus.h" +#include "hw/vfio/vfio.h" +#include "hw/vfio/vfio-common.h" +#include "hw/s390x/ap-device.h" +#include "qemu/error-report.h" +#include "qemu/queue.h" +#include "cpu.h" +#include "kvm_s390x.h" + +#define VFIO_AP_DEVICE_TYPE "vfio-ap" + +typedef struct VFIOAPDevice { + APDevice apdev; + VFIODevice vdev; + QTAILQ_ENTRY(VFIOAPDevice) sibling; +} VFIOAPDevice; + +static void vfio_ap_compute_needs_reset(VFIODevice *vdev) +{ + vdev->needs_reset =3D false; +} + +/* + * We don't need vfio_hot_reset_multi and vfio_eoi operations for + * vfio-ap-matrix device now. + */ +struct VFIODeviceOps vfio_ap_ops =3D { + .vfio_compute_needs_reset =3D vfio_ap_compute_needs_reset, +}; + +static QTAILQ_HEAD(, VFIOAPDevice) vfio_ap_devices =3D + QTAILQ_HEAD_INITIALIZER(vfio_ap_devices); + +static void vfio_put_device(VFIOAPDevice *apdev) +{ + g_free(apdev->vdev.name); + vfio_put_base_device(&apdev->vdev); +} + +static VFIOGroup *vfio_ap_get_group(VFIOAPDevice *vapdev, Error **errp) +{ + char *tmp, group_path[PATH_MAX]; + ssize_t len; + int groupid; + + tmp =3D g_strdup_printf("%s/iommu_group", vapdev->vdev.sysfsdev); + len =3D readlink(tmp, group_path, sizeof(group_path)); + g_free(tmp); + + if (len <=3D 0 || len >=3D sizeof(group_path)) { + error_setg(errp, "%s: no iommu_group found for %s", + VFIO_AP_DEVICE_TYPE, vapdev->vdev.sysfsdev); + return NULL; + } + + group_path[len] =3D 0; + + if (sscanf(basename(group_path), "%d", &groupid) !=3D 1) { + error_setg(errp, "vfio: failed to read %s", group_path); + return NULL; + } + + return vfio_get_group(groupid, &address_space_memory, errp); +} + +static void vfio_ap_realize(DeviceState *dev, Error **errp) +{ + VFIODevice *vbasedev; + VFIOGroup *vfio_group; + APDevice *apdev =3D DO_UPCAST(APDevice, parent_obj, dev); + VFIOAPDevice *vapdev =3D DO_UPCAST(VFIOAPDevice, apdev, apdev); + char *mdevid; + Error *local_err =3D NULL; + int ret; + + if (!s390_has_feat(S390_FEAT_AP)) { + error_setg(&local_err, "AP support not enabled"); + goto out_err; + } + + ret =3D kvm_s390_set_interpret_ap(1); + if (ret) { + error_setg_errno(&local_err, errno, + "error setting interpretive execution of AP instr= uctions"); + goto out_err; + } + + vfio_group =3D vfio_ap_get_group(vapdev, &local_err); + if (!vfio_group) { + goto out_group_err; + } + + vapdev->vdev.ops =3D &vfio_ap_ops; + vapdev->vdev.type =3D VFIO_DEVICE_TYPE_AP; + mdevid =3D basename(vapdev->vdev.sysfsdev); + vapdev->vdev.name =3D g_strdup_printf("%s", mdevid); + vapdev->vdev.dev =3D dev; + QLIST_FOREACH(vbasedev, &vfio_group->device_list, next) { + if (strcmp(vbasedev->name, vapdev->vdev.name) =3D=3D 0) { + error_setg(&local_err, + "%s: AP device %s has already been realized", + VFIO_AP_DEVICE_TYPE, vapdev->vdev.name); + goto out_device_err; + } + } + + ret =3D vfio_get_device(vfio_group, mdevid, &vapdev->vdev, &local_err); + if (ret) { + goto out_device_err; + } + + QTAILQ_INSERT_TAIL(&vfio_ap_devices, vapdev, sibling); + + return; + + +out_device_err: + vfio_put_group(vfio_group); +out_group_err: + kvm_s390_set_interpret_ap(0); +out_err: + error_propagate(errp, local_err); +} + +static void vfio_ap_unrealize(DeviceState *dev, Error **errp) +{ + APDevice *apdev =3D DO_UPCAST(APDevice, parent_obj, dev); + VFIOAPDevice *vapdev =3D DO_UPCAST(VFIOAPDevice, apdev, apdev); + VFIOGroup *group =3D vapdev->vdev.group; + + vfio_put_device(vapdev); + vfio_put_group(group); + kvm_s390_set_interpret_ap(0); +} + +static Property vfio_ap_properties[] =3D { + DEFINE_PROP_STRING("sysfsdev", VFIOAPDevice, vdev.sysfsdev), + DEFINE_PROP_END_OF_LIST(), +}; + +static const VMStateDescription vfio_ap_vmstate =3D { + .name =3D VFIO_AP_DEVICE_TYPE, + .unmigratable =3D 1, +}; + +static void vfio_ap_class_init(ObjectClass *klass, void *data) +{ + DeviceClass *dc =3D DEVICE_CLASS(klass); + + dc->props =3D vfio_ap_properties; + dc->vmsd =3D &vfio_ap_vmstate; + dc->desc =3D "VFIO-based AP device assignment"; + dc->realize =3D vfio_ap_realize; + dc->unrealize =3D vfio_ap_unrealize; +} + +static const TypeInfo vfio_ap_info =3D { + .name =3D VFIO_AP_DEVICE_TYPE, + .parent =3D AP_DEVICE_TYPE, + .instance_size =3D sizeof(VFIOAPDevice), + .class_init =3D vfio_ap_class_init, +}; + +static void register_vfio_ap_type(void) +{ + type_register_static(&vfio_ap_info); +} + +type_init(register_vfio_ap_type) diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index f3a2ac9..f1f22d9 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -46,6 +46,7 @@ enum { VFIO_DEVICE_TYPE_PCI =3D 0, VFIO_DEVICE_TYPE_PLATFORM =3D 1, VFIO_DEVICE_TYPE_CCW =3D 2, + VFIO_DEVICE_TYPE_AP =3D 3, }; =20 typedef struct VFIOMmap { --=20 1.7.1 From nobody Sat Oct 25 11:10:43 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1521156553346539.8617174429999; Thu, 15 Mar 2018 16:29:13 -0700 (PDT) Received: from localhost ([::1]:53807 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ewcJ6-0003rD-HI for importer@patchew.org; Thu, 15 Mar 2018 19:29:12 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48399) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ewcFd-0001Y0-Rs for qemu-devel@nongnu.org; Thu, 15 Mar 2018 19:25:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ewcFa-0000EZ-MA for qemu-devel@nongnu.org; Thu, 15 Mar 2018 19:25:37 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:42778 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ewcFa-0000EK-FE for qemu-devel@nongnu.org; Thu, 15 Mar 2018 19:25:34 -0400 Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w2FNLLwt002405 for ; Thu, 15 Mar 2018 19:25:34 -0400 Received: from e33.co.us.ibm.com (e33.co.us.ibm.com [32.97.110.151]) by mx0a-001b2d01.pphosted.com with ESMTP id 2gr0f3497s-1 (version=TLSv1.2 cipher=AES256-SHA256 bits=256 verify=NOT) for ; Thu, 15 Mar 2018 19:25:33 -0400 Received: from localhost by e33.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 15 Mar 2018 17:25:31 -0600 Received: from b03cxnp08026.gho.boulder.ibm.com (9.17.130.18) by e33.co.us.ibm.com (192.168.1.133) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Thu, 15 Mar 2018 17:25:27 -0600 Received: from b03ledav002.gho.boulder.ibm.com (b03ledav002.gho.boulder.ibm.com [9.17.130.233]) by b03cxnp08026.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w2FNPQI911403546; Thu, 15 Mar 2018 16:25:26 -0700 Received: from b03ledav002.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 430EB136044; Thu, 15 Mar 2018 17:25:26 -0600 (MDT) Received: from localhost.localdomain (unknown [9.80.217.151]) by b03ledav002.gho.boulder.ibm.com (Postfix) with ESMTPS id E31FD13603A; Thu, 15 Mar 2018 17:25:23 -0600 (MDT) From: Tony Krowiak To: qemu-devel@nongnu.org Date: Thu, 15 Mar 2018 19:24:59 -0400 X-Mailer: git-send-email 1.7.1 In-Reply-To: <1521156300-19296-1-git-send-email-akrowiak@linux.vnet.ibm.com> References: <1521156300-19296-1-git-send-email-akrowiak@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 18031523-0008-0000-0000-00000976DF58 X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00008681; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000254; SDB=6.01003605; UDB=6.00510771; IPR=6.00782931; MB=3.00020058; MTD=3.00000008; XFM=3.00000015; UTC=2018-03-15 23:25:30 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18031523-0009-0000-0000-00004668E032 Message-Id: <1521156300-19296-7-git-send-email-akrowiak@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2018-03-15_11:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1803150253 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 Subject: [Qemu-devel] [PATCH v3 6/7] s390x/kvm: handle AP instruction interception X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mjrosato@linux.vnet.ibm.com, peter.maydell@linaro.org, pasic@linux.vnet.ibm.com, alifm@linux.vnet.ibm.com, eskultet@redhat.com, david@redhat.com, pmorel@linux.vnet.ibm.com, cohuck@redhat.com, heiko.carstens@de.ibm.com, alex.williamson@redhat.com, agraf@suse.de, borntraeger@de.ibm.com, qemu-s390x@nongnu.org, akrowiak@linux.vnet.ibm.com, jjherne@linux.vnet.ibm.com, schwidefsky@de.ibm.com, pbonzini@redhat.com, bjsdjshi@linux.vnet.ibm.com, eric.auger@redhat.com, rth@twiddle.net Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" If the CPU model indicates that AP facility is installed on the guest (i.e., -cpu xxxx,ap=3Don), then the expectation is that the AP bus running in the guest will initialize; however, if the AP instructions are not being interpreted by the firmware, then they will be intercepted and routed back to QEMU for handling. If a handler is not defined to process the intercepted instruciton, then an operation exception will be injected into the guest, in which case the AP bus will not initialize. There are two situations where AP instructions will not be interpreted: 1. The guest is not configured with a vfio-ap device (i.e., -device vfio-ap,sysfsdev=3D$path-to-mdev). The realize function for the vfio-ap device enables interpretive execution of AP instructions. 2. The guest is a second level guest but the first level guest has not enabled interpretive execution. This patch introduces AP instruction handlers to ensure the AP bus module initializes on the guest when the AP facility is installed on the guest but AP instructions are not being interpreted. The logic incorporated is: * If the CPU model indicates AP instructions are installed * Set the status response code for the instruction to indicate that the APQN contained in the instruction is not valid. This is a valid response because there will be no devices configured for the guest in any of the above scenarios. * Else return an error from the handler. This will result in an operation being injected into the guest and the AP bus will not initialize on the guest. That is commensurate with how things work today. Signed-off-by: Tony Krowiak --- hw/vfio/ap.c | 45 ++++++++++++++++++++++++++++++++++++++= ++++ include/hw/s390x/ap-device.h | 6 +++++ target/s390x/kvm.c | 14 +++++++++++++ 3 files changed, 65 insertions(+), 0 deletions(-) diff --git a/hw/vfio/ap.c b/hw/vfio/ap.c index b397bb1..88e744d 100644 --- a/hw/vfio/ap.c +++ b/hw/vfio/ap.c @@ -148,6 +148,51 @@ static void vfio_ap_unrealize(DeviceState *dev, Error = **errp) kvm_s390_set_interpret_ap(0); } =20 +int ap_device_handle_nqap(S390CPU *cpu) +{ + CPUS390XState *env =3D &cpu->env; + + if (s390_has_feat(S390_FEAT_AP)) { + env->regs[1] =3D 0x10000; + + return 0; + } + + return -EOPNOTSUPP; +} + +int ap_device_handle_dqap(S390CPU *cpu) +{ + CPUS390XState *env =3D &cpu->env; + + if (s390_has_feat(S390_FEAT_AP)) { + env->regs[1] =3D 0x10000; + + return 0; + } + + return -EOPNOTSUPP; +} + +int ap_device_handle_pqap(S390CPU *cpu) +{ + CPUS390XState *env =3D &cpu->env; + int fc =3D 4 & (env->regs[0] >> 24); + + /* + * The Query Configuration Information (QCI) function (fc =3D=3D 4) do= es not + * set a response code in reg 1, so check for that along with the + * AP feature. + */ + if ((fc !=3D 4) && s390_has_feat(S390_FEAT_AP)) { + env->regs[1] =3D 0x10000; + + return 0; + } + + return -EOPNOTSUPP; +} + static Property vfio_ap_properties[] =3D { DEFINE_PROP_STRING("sysfsdev", VFIOAPDevice, vdev.sysfsdev), DEFINE_PROP_END_OF_LIST(), diff --git a/include/hw/s390x/ap-device.h b/include/hw/s390x/ap-device.h index 693df90..d45ae38 100644 --- a/include/hw/s390x/ap-device.h +++ b/include/hw/s390x/ap-device.h @@ -11,6 +11,8 @@ #ifndef HW_S390X_AP_DEVICE_H #define HW_S390X_AP_DEVICE_H =20 +#include "cpu.h" + #define AP_DEVICE_TYPE "ap-device" =20 typedef struct APDevice { @@ -35,4 +37,8 @@ static inline APDevice *to_ap_dev(DeviceState *dev) #define AP_DEVICE_CLASS(klass) \ OBJECT_CLASS_CHECK(APDeviceClass, (klass), AP_DEVICE_TYPE) =20 +int ap_device_handle_nqap(S390CPU *cpu); +int ap_device_handle_dqap(S390CPU *cpu); +int ap_device_handle_pqap(S390CPU *cpu); + #endif /* HW_S390X_AP_DEVICE_H */ diff --git a/target/s390x/kvm.c b/target/s390x/kvm.c index 2812e28..a636394 100644 --- a/target/s390x/kvm.c +++ b/target/s390x/kvm.c @@ -50,6 +50,7 @@ #include "exec/memattrs.h" #include "hw/s390x/s390-virtio-ccw.h" #include "hw/s390x/s390-virtio-hcall.h" +#include "hw/s390x/ap-device.h" =20 #ifndef DEBUG_KVM #define DEBUG_KVM 0 @@ -88,6 +89,9 @@ #define PRIV_B2_CHSC 0x5f #define PRIV_B2_SIGA 0x74 #define PRIV_B2_XSCH 0x76 +#define PRIV_B2_NQAP 0xad +#define PRIV_B2_DQAP 0xae +#define PRIV_B2_PQAP 0xaf =20 #define PRIV_EB_SQBS 0x8a #define PRIV_EB_PCISTB 0xd0 @@ -1245,6 +1249,16 @@ static int handle_b2(S390CPU *cpu, struct kvm_run *r= un, uint8_t ipa1) case PRIV_B2_SCLP_CALL: rc =3D kvm_sclp_service_call(cpu, run, ipbh0); break; + case PRIV_B2_NQAP: + rc =3D ap_device_handle_nqap(cpu); + break; + case PRIV_B2_DQAP: + rc =3D ap_device_handle_dqap(cpu); + break; + case PRIV_B2_PQAP: + rc =3D ap_device_handle_pqap(cpu); + break; + break; default: rc =3D -1; DPRINTF("KVM: unhandled PRIV: 0xb2%x\n", ipa1); --=20 1.7.1 From nobody Sat Oct 25 11:10:43 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1521156959291951.6591699733095; Thu, 15 Mar 2018 16:35:59 -0700 (PDT) Received: from localhost ([::1]:53837 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ewcPZ-0006ZO-EN for importer@patchew.org; Thu, 15 Mar 2018 19:35:53 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:50209) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ewcOG-0005x5-8G for qemu-devel@nongnu.org; Thu, 15 Mar 2018 19:34:35 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ewcOD-0005kz-1M for qemu-devel@nongnu.org; Thu, 15 Mar 2018 19:34:32 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:50248) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ewcOC-0005kT-Kb for qemu-devel@nongnu.org; Thu, 15 Mar 2018 19:34:28 -0400 Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w2FNXO8K080025 for ; Thu, 15 Mar 2018 19:34:27 -0400 Received: from e33.co.us.ibm.com (e33.co.us.ibm.com [32.97.110.151]) by mx0a-001b2d01.pphosted.com with ESMTP id 2gr05pvxm0-1 (version=TLSv1.2 cipher=AES256-SHA256 bits=256 verify=NOT) for ; Thu, 15 Mar 2018 19:34:26 -0400 Received: from localhost by e33.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 15 Mar 2018 17:25:36 -0600 Received: from b03cxnp08026.gho.boulder.ibm.com (9.17.130.18) by e33.co.us.ibm.com (192.168.1.133) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Thu, 15 Mar 2018 17:25:31 -0600 Received: from b03ledav002.gho.boulder.ibm.com (b03ledav002.gho.boulder.ibm.com [9.17.130.233]) by b03cxnp08026.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w2FNPTc112189974; Thu, 15 Mar 2018 16:25:29 -0700 Received: from b03ledav002.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C00F3136046; Thu, 15 Mar 2018 17:25:29 -0600 (MDT) Received: from localhost.localdomain (unknown [9.80.217.151]) by b03ledav002.gho.boulder.ibm.com (Postfix) with ESMTPS id 67D1F136043; Thu, 15 Mar 2018 17:25:27 -0600 (MDT) From: Tony Krowiak To: qemu-devel@nongnu.org Date: Thu, 15 Mar 2018 19:25:00 -0400 X-Mailer: git-send-email 1.7.1 In-Reply-To: <1521156300-19296-1-git-send-email-akrowiak@linux.vnet.ibm.com> References: <1521156300-19296-1-git-send-email-akrowiak@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 18031523-0008-0000-0000-00000976DF5F X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00008681; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000254; SDB=6.01003605; UDB=6.00510771; IPR=6.00782931; MB=3.00020058; MTD=3.00000008; XFM=3.00000015; UTC=2018-03-15 23:25:34 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18031523-0009-0000-0000-00004668E03E Message-Id: <1521156300-19296-8-git-send-email-akrowiak@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2018-03-15_11:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1803150254 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.156.1 Subject: [Qemu-devel] [PATCH v3 7/7] s390: doc: detailed specifications for AP virtualization X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mjrosato@linux.vnet.ibm.com, peter.maydell@linaro.org, pasic@linux.vnet.ibm.com, alifm@linux.vnet.ibm.com, eskultet@redhat.com, david@redhat.com, pmorel@linux.vnet.ibm.com, cohuck@redhat.com, heiko.carstens@de.ibm.com, alex.williamson@redhat.com, agraf@suse.de, borntraeger@de.ibm.com, qemu-s390x@nongnu.org, akrowiak@linux.vnet.ibm.com, jjherne@linux.vnet.ibm.com, schwidefsky@de.ibm.com, pbonzini@redhat.com, bjsdjshi@linux.vnet.ibm.com, eric.auger@redhat.com, rth@twiddle.net Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This patch provides documentation describing the AP architecture and design concepts behind the virtualization of AP devices. It also includes an example of how to configure AP devices for exclusive use of KVM guests. Signed-off-by: Tony Krowiak --- docs/vfio-ap.txt | 624 ++++++++++++++++++++++++++++++++++++++++++++++++++= ++++ 1 files changed, 624 insertions(+), 0 deletions(-) create mode 100644 docs/vfio-ap.txt diff --git a/docs/vfio-ap.txt b/docs/vfio-ap.txt new file mode 100644 index 0000000..54e7523 --- /dev/null +++ b/docs/vfio-ap.txt @@ -0,0 +1,624 @@ +Adjunct Processor (AP) Device +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D + +Contents: +=3D=3D=3D=3D=3D=3D=3D=3D=3D +* Introduction +* AP Architectural Overview +* Start Interpretive Execution (SIE) Instruction +* AP Matrix Configuration on Linux Host +* AP Matrix Configuration for a Linux Guest +* Starting a Linux Guest Configured with an AP Matrix +* Example: Configure AP Matrices for Two Linux Guests + +Introduction: +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +The IBM Adjunct Processor (AP) Cryptographic Facility is comprised +of three AP instructions and from 1 to 256 PCIe cryptographic adapter card= s. +These AP devices provide cryptographic functions to all CPUs assigned to a +linux system running in an IBM Z system LPAR. + +On s390x, AP adapter cards are exposed via the AP bus. This document +describes how those cards may be made available to KVM guests using the +VFIO mediated device framework. + +AP Architectural Overview: +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D +In order understand the terminology used in the rest of this document, let= 's +start with some definitions: + +* AP adapter + + An AP adapter is an IBM Z adapter card that can perform cryptographic + functions. There can be from 0 to 256 adapters assigned to an LPAR. Adap= ters + assigned to the LPAR in which a linux host is running will be available = to + the linux host. Each adapter is identified by a number from 0 to 255. Wh= en + installed, an AP adapter is accessed by AP instructions executed by any = CPU. + +* AP domain + + An adapter is partitioned into domains. Each domain can be thought of as + a set of hardware registers for processing AP instructions. An adapter c= an + hold up to 256 domains. Each domain is identified by a number from 0 to = 255. + Domains can be further classified into two types: + + * Usage domains are domains that can be accessed directly to process AP + commands + + * Control domains are domains that are accessed indirectly by AP + commands sent to a usage domain to control or change the domain, for + example; to set a secure private key for the domain. + +* AP Queue + + An AP queue is the means by which an AP command-request message is sent = to an + AP usage domain inside a specific AP. An AP queue is identified by a tup= le + comprised of an AP adapter ID (APID) and an AP queue index (APQI). The + APQI corresponds to a given usage domain number within the adapter. This= tuple + forms an AP Queue Number (APQN) uniquely identifying an AP queue. AP + instructions include a field containing the APQN to identify the AP queu= e to + which the AP command-request message is to be sent for processing. + +* AP Instructions: + + There are three AP instructions: + + * NQAP: to enqueue an AP command-request message to a queue + * DQAP: to dequeue an AP command-reply message from a queue + * PQAP: to administer the queues + +Start Interpretive Execution (SIE) Instruction +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +A KVM guest is started by executing the Start Interpretive Execution (SIE) +instruction. The SIE state description is a control block that contains the +state information for a KVM guest and is supplied as input to the SIE +instruction. The SIE state description contains a field that references +a Crypto Control Block (CRYCB). The CRYCB contains three fields to identif= y the +adapters, usage domains and control domains assigned to the KVM guest: + +* The AP Mask (APM) field is a bit mask that identifies the AP adapters as= signed + to the KVM guest. Each bit in the mask, from most significant to least + significant bit, corresponds to an APID from 0-255. If a bit is set, the + corresponding adapter is valid for use by the KVM guest. + +* The AP Queue Mask (AQM) field is a bit mask identifying the AP queues as= signed + to the KVM guest. Each bit in the mask, from most significant to least + significant bit, corresponds to an AP queue index (APQI) from 0-255. If = a bit + is set, the corresponding queue is valid for use by the KVM guest. + +* The AP Domain Mask field is a bit mask that identifies the AP control do= mains + assigned to the KVM guest. The ADM bit mask controls which domains can be + changed by an AP command-request message sent to a usage domain from the + guest. Each bit in the mask, from least significant to most significant = bit, + corresponds to a domain from 0-255. If a bit is set, the corresponding d= omain + can be modified by an AP command-request message sent to a usage domain + configured for the KVM guest. + +If you recall from the description of an AP Queue, AP instructions include +an APQN to identify the AP adapter and AP queue to which an AP command-req= uest +message is to be sent (NQAP and PQAP instructions), or from which a +command-reply message is to be received (DQAP instruction). The validity o= f an +APQN is defined by the matrix calculated from the APM and AQM; it is the +intersection of all assigned adapter numbers (APM) with all assigned queue +indexes (AQM). For example, if adapters 1 and 2 and usage domains 5 and 6 = are +assigned to a guest, the APQNs (1,5), (1,6), (2,5) and (2,6) will be valid= for +the guest. + +The APQNs provide secure key functionality - i.e., a private key is stored= on +the adapter card for each of its domains - so each APQN must be assigned t= o at +most one guest or the linux host. + + Example 1: Valid configuration: + ------------------------------ + Guest1: adapters 1,2 domains 5,6 + Guest2: adapter 1,2 domain 7 + + This is valid because both guests have a unique set of APQNs: Guest1 has + APQNs (1,5), (1,6), (2,5) and (2,6); Guest2 has APQNs (1,7) and (2,7). + + Example 2: Invalid configuration: + -------------------------------- + Guest1: adapters 1,2 domains 5,6 + Guest2: adapter 1 domains 6,7 + + This is an invalid configuration because both guests have access to + APQN (1,6). + +AP device Configuration on Linux Host: +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +A linux system is a guest of the LPAR in which it is running and has acces= s to +the AP resources configured for the LPAR. The LPAR's AP matrix is +configured using the 'Customize/Delete Activation Profiles' dialog from th= e HMC. +This dialog displays the activation profiles configured for the linux syst= em. +Selecting the specific activation profile to be edited and clicking the +'Customize Profile' button will open the 'Customize Image Profiles' dialog. +Selecting the 'Crypto' link in the tree view on the left hand side of the = dialog +will display the AP matrix configuration in the right hand panel. There, o= ne can +assign AP adapters - called Cryptos - and domains to the LPAR. When the li= nux +system is started using this activation profile, it will have access to the +matrix of AP adapters and domains configured via the activation profile. + +When the linux system is started, the AP adapter devices will be connected= to +the AP bus and the following AP matrix interfaces will be created in sysfs: + +/sys/bus/ap +... [devices] +...... xx.yyyy +...... ... +...... cardxx +...... ... + +Where: + cardxx is adapter number xx (in hex) + yyyy is a usage domain number yyyy (in hex) +....xx.yyyy is APQN (xx,yyyy) + +For example, if AP adapters 5 and 6 and domains 4 and 71 (0x47) are config= ured +for the LPAR, the sysfs representation on the linux system would look like= this: + +/sys/bus/ap +... [devices] +...... 05.0004 +...... 05.0047 +...... 06.0004 +...... 06.0047 +...... card05 +...... card06 + +There will also be AP device drivers created to control each type of AP ma= trix +interface available to the IBM Z system: + +/sys/bus/ap +... [drivers] +...... [cex2acard] for Crypto Express 2/3 accelerator cards +...... [cex2aqueue] for AP queues served by Crypto Express 2/3 + accelerator cards +...... [cex4card] for Crypto Express 4/5/6 accelerator and coproce= ssor + cards +...... [cex4queue] for AP queues served by Crypto Express 4/5/6 + accelerator and coprocessor cards +...... [pcixcccard] for Crypto Express 2/3 coprocessor cards +...... [pcixccqueue] for AP queues served by Crypto Express 2/3 + coprocessor cards + +Links to the AP interfaces controlled by each AP device driver will be cre= ated +in the device driver's sysfs directory. For example, if AP adapter 5 and d= omains +4 and 71 (0x47) are assigned to the LPAR and adapter 5 is a CEX5 card, the +following links will be created in the CEX5 drivers' sysfs directories: + +/sys/bus/ap +... [drivers] +...... [cex4card] +......... [card05] +...... [cex4queue] +......... [05.0004] +......... [05.0047] + +AP Matrix Configuration for a Linux Guest: +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +In order to configure the AP matrix for a guest, the adapters, usage domai= ns +and control domains to be used by the guest must be assigned to the guest.= This +section describes how to configure a guest's AP matrix. + +The kernel interfaces for configuring an AP matrix for a linux guest are b= uilt +on the VFIO mediated device framework and are provided by the vfio_ap +kernel module. By default, the vfio_ap module is a loadable module, The +dependency chain for the vfio_ap module is: +* vfio +* mdev +* vfio_mdev +* vfio_ap + +When installed, the vfio_ap module is initialized. During module initializ= ation, +a vfio_ap driver is created and registered with the AP bus creating the +following sysfs interfaces: + + /sys/bus/ap/drivers/ +...[vfio_ap] +...... bind +...... unbind + +The vfio_ap device driver will create a 'matrix' device to hold the APQNs +reserved for exclusive use by KVM guests: + +/sys/devices/ +... [vfio_ap] +......[matrix] symlink to the matrix device directory + +The vfio_ap device driver serves several purposes: +1. Provides an interface for securing APQNs preventing their use by the ho= st + linux system and reserving their use by one or more guests. +2. Creates the sysfs interfaces for configuring an AP matrix for a linux g= uest. + +Securing APQNs +-------------- + An APQN is reserved by unbinding an AP queue device AP bus device drive= r and + binding it to the vfio_ap device driver. For example, suppose we want to + secure APQN (05,0004). Assuming that the AP adapter card 5 is a CEX5 + coprocessor card: + + echo 05.0004 > /sys/bus/ap/drivers/cex4queue/unbind + echo 05.0004 > /sys/bus/ap/drivers/vfio_ap/bind + + This action will store the APQN in the /sys/devices/vfio_ap/matrix devi= ce + which makes it available for use by a linux guest. + +Configuring an AP matrix for a linux guest. +------------------------------------------ +These sysfs interfaces are built on the VFIO mediated device framework. To +configure an AP matrix for a guest, a mediated matrix device must first be +created for the /sys/devices/vfio_ap/matrix device. The sysfs interfaceAPQ= I corresponding to +for creating a mediated matrix device is in: + +/sys/devices +... [vfio_ap] +......[matrix] +......... [mdev_supported_types] +............ [vfio_ap-passthrough] +............... create +............... [devices] + +A mediated AP matrix device is created by writing a UUID to the attribute +file named 'create', for example: + + uuidgen > create + +When a mediated AP matrix device is created, a sysfs directory named after +the UUID: + +/sys/devices +... [vfio_ap] +......[matrix] +......... [mdev_supported_types] +............ [vfio_ap-passthrough] +............... create +............... [devices] +.................. [$uuid] + +There will also be three sets of attribute files created in the mediated +matrix device's sysfs directory to configure an AP matrix for the +KVM guest: + +/sys/devices +... [vfio_ap] +......[matrix] +......... [mdev_supported_types] +............ [vfio_ap-passthrough] +............... create +............... [devices] +.................. [$uuid] +..................... assign_adapter +..................... assign_control_domain +..................... assign_domain +..................... matrix +..................... unassign_adapter +..................... unassign_control_domain +..................... unassign_domain + +assign_adapter + To assign an AP adapter to the mediated matrix device, its APID is writ= ten + 'assign_adapter' file. This may be done multiple times to assign more t= han + one adapter. The APID may be specified using conventional semantics + as a decimal, hexidecimal, or octal number. For example, to assign adap= ters + 4, 5 and 16 to mediated matrix device $uuid in decimal, hexidecimal and= octal + respectively: + + echo 4 > assign_adapter + echo 0x5 > assign_adapter + echo 020 + +unassign_adapter + To unassign an AP adapter, its APID is written to the 'unassign_adapter' + file. This may also be done multiple times to unassign more than one ad= apter. + +assign_domain + To assign a usage domain, the APQI corresponding to the domain number is + written into the 'assign_domain' file. This may be done multiple times = to + assign more than one usage domain. The APQI may be specified using + conventional semantics as a decimal, hexidecimal, or octal number. For + example, to assign usage domains 4, 8, and 71 to mediated matrix device + $uuid in decimal, hexidecimal and octal respectively: + + echo 4 > assign_domain + echo 0x8 > assign_domain + echo 0107 > assign_domain + +unassign_domain + To unassign a usage domain, the APQI corresponding to the domain number= is + written into the 'unassign_domain' file. This may be done multiple time= s to + unassign more than one usage domain. + +assign_control_domain + To assign a control domain, the domain number is written into the + 'assign_control_domain' file. This may be done multiple times to + assign more than one control domain. The domain number may be specified= using + conventional semantics as a decimal, hexidecimal, or octal number. For + example, to assign control domains 4, 8, and 71 to mediated matrix dev= ice + $uuid in decimal, hexidecimal and octal respectively: + + echo 4 > assign_domain + echo 0x8 > assign_domain + echo 0107 > assign_domain + +unassign_control_domain + To unassign a control domain, the domain number is written into the + 'unassign_domain' file. This may be done multiple times to unassign mor= e than + one control domain. + +Notes: +* Hot plug/unplug is not currently supported for mediated AP matrix device= s, + so the AP matrix resulting from assignment and/or unassignment of AP + adapters, usage domains and control domains to a mediated AP matrix devi= ce + while the guest is running will not take affect until the linux guest is + rebooted. +* By architectural convention, all usage domains configured for a KVM guest + will also be implicitly assigned as control domains also, to there is no + need to assign control domains that are assigned as usage domains. + +Starting a Linux Guest Configured with an AP Matrix: +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D +In addition to providing the sysfs interfaces for configuring the AP matri= x for +a linux guest, a mediated matrix device also acts as a communication pathw= ay +between QEMU and the vfio_ap device driver. To gain access to the +device driver, the following option must be specified on the QEMU command = line: + + -device vfio_ap,sysfsdev=3D$path-to-mdev + +The sysfsdev parameter specifies the path to the mediated matrix device. +There are a number of ways to specify this path: + +/sys/devices/vfio_ap/matrix/$uuid +/sys/bus/mdev/devices/$uuid +/sys/bus/mdev/drivers/vfio_mdev/$uuid +/sys/devices/vfio_ap/matrix/mdev_supported_types/vfio_ap-passthrough/devic= es/$uuid + +When the linux guest is subsequently started, the guest will open the medi= ated +matrix device's file descriptor to get information about the mediated matr= ix +device. The vfio_ap device driver will update the APM, AQM, and ADM fields= in the +guest's CRYCB with the adapter, usage domain and control domains assigned = to +via the mediated matrix device's sysfs attribute files. Programs running o= n the +linux guest will then: + +1. Have direct access to the APQNs derived from the intersection of the AP + adapter and usage domain numbers specified in the APM and AQM respectiv= ely + +2. Have authorization to process AP commands to change - e.g., store a new + secure key - a control domain identified in an AP instruction sent to a= valid + APQN. + +CPU model features: + +Three CPU model features are available for controlling guest access to AP +facilities: + +1. AP facilities feature + + The AP facilities feature indicates that AP facilities are installed on= the + guest. This feature will be enabled by the kernel only if the AP facili= ties + are installed on the host system. It will turned on automatically for g= uests + started with CPU model zEC12 or newer. The feature is s390-specific and= is + represented as a parameter of the -cpu option on the QEMU command line: + + qemu-system-s390x -cpu $model,ap=3Don|off + + Where: + + $model is the CPU model defined for the guest (defaults to the mo= del of + the host system if not specified). + + ap=3Don|off indicates whether AP facilities are installed (on) or= not + installed (off). The default for CPU models zEC12 or ne= wer + is ap=3Don. AP facilities must be installed when this p= arameter + is used in conjunction with -device vfio-ap,sysfsdev=3D= $path or + the guest will not start. + +2. Query Configuration Information (QCI) facility + + The QCI facility is used by the AP bus running on the guest to query the + configuration of the AP facilities. This facility will be enabled by + the kernel only if the QCI facility is installed on the host system. It= will + be turned on automatically for guests started with CPU model zEC12 or n= ewer. + The feature is s390-specific and is represented as a parameter of the -= cpu + option on the QEMU command line: + + qemu-system-s390x -cpu $model,qci=3Don|off + + Where: + + $model is the CPU model defined for the guest + + qci=3Don|off indicates whether the QCI facility is installed (on)= or not + installed (off). The default for CPU models zEC12 or n= ewer + is qci=3Don. Turning the QCI facility on makes no sens= e if it + is not used in conjunction with the + '-device vfio-ap,sysfsdev=3D$path' option. A warning w= ill be + presented if QCI is turned on and the AP facilities ar= e not + installed. + + If the QCI facility is turned off, APQNs with an APQI + greater than 15 will not be accessible from the guest. + +3. Adjunct Process Facility Test (APFT) facility + + The APFT facility is used by the AP bus running on the guest to test the + AP facilities available for a given AP queue. This facility will be ena= bled + by the kernel only if the APFT facility is installed on the host system= . It + will be turned on automatically for guests started with CPU model zEC12= or + newer. The feature is s390-specific and is represented as a parameter o= f the + -cpu option on the QEMU command line: + + qemu-system-s390x -cpu $model,apft=3Don|off + + Where: + + $model is the CPU model defined for the guest (defaults to the mo= del of + the host system if not specified). + + apft=3Don|off indicates whether the APFT facility is installed (o= n) or + not installed (off). The default for CPU models zEC12= and + newer is apft=3Don. Turning the APFT facility on make= s no + sense if it is not used in conjunction with the + -device vfio-ap,sysfsdev=3D$path option. A warning wi= ll be + presented if APFT is turned on and the AP facilities = are + not installed. + + It also makes no sense to turn APFT off when used in + conjunction with the vfio-ap device because the APFT + facility is required; the AP bus running on the guest= will + not detect CEX4 and newer devices without it. Since o= nly + CEX4 and newer devices are supported for guest usage,= no AP + devices can be made accessible to a guest started wit= hout + APFT installed. + +Example: Configure AP Matrixes for Two Linux Guests: +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D +Let's now provide an example to illustrate how KVM guests may be given +access to AP facilities. For this example, we will show how to configure +two guests such that executing the lszcrypt command on the guests would +look like this: + +Guest1 +------ +CARD.DOMAIN TYPE MODE =20 +------------------------------ +05 CEX5C CCA-Coproc=20 +05.0004 CEX5C CCA-Coproc +05.00ab CEX5C CCA-Coproc=20 +06 CEX5A Accelerator +06.0004 CEX5A Accelerator +06.00ab CEX5C CCA-Coproc=20 + +Guest2 +------ +CARD.DOMAIN TYPE MODE =20 +------------------------------ +05 CEX5A Accelerator +05.0047 CEX5A Accelerator +05.00ff CEX5A Accelerator + +These are the steps for configuring the Guest1 and Guest2: + +1. The first thing that needs to be done is to secure the AP queues to be + used by the two guests so that the host can not access them. This is do= ne + by unbinding each AP Queue device from its respective AP driver. In our + example, these queues are bound to the cex4queue driver. This would be + the sysfs location of these devices: + + /sys/bus/ap + --- [drivers] + ------ [cex4queue] + --------- [05.0004] + --------- [05.0047] --------------------- control_domains + --------------------- domains + --------- [05.00ab] + --------- [05.00ff] + --------- [06.0004] + --------- [06.00ab] + --------- unbind + + To unbind AP queue 05.0004 from the cex4queue device driver: + + echo 05.0004 > unbind + + This must also be done for AP queues 05.00ab, 05.0047, 05.00ff, 06.0004, + and 06.00ab. + +2. The next step is to reserve the queues for use by the two KVM guests. + This is accomplished by binding them to the VFIO AP device driver. + This is the sysfs location of the VFIO AP device driver: + + /sys/bus/ap + ---[drivers] + ------ [vfio_ap] + ---------- bind + + To bind queue 05.0004 to the vfio_ap driver: + + echo 05.0004 > bind + + This must also be done for AP queues 05.00ab, 05.0047, 05.00ff, 06.0004, + and 06.00ab. + +3. Create the mediated devices needed to configure the AP matrixes for the + two guests and to provide an interface to the vfio_ap driver for + use by the guests: + + /sys/devices/ + --- [vfio_ap] + ------ [matrix] (this is the matrix device) + --------- [mdev_supported_types] + ------------ [vfio_ap-passthrough] (passthrough mediated matrix device = type) + --------------- create + --------------- [devices] + + To create the mediated devices for the two guests: + + uuidgen > create + uuidgen > create + + This will create two mediated devices in the [devices] subdirectory nam= ed + with the UUID written to the create attribute file. We call them $uuid1 + and $uuid2: + + /sys/devices/ + --- [vfio_ap] + ------ [matrix] + --------- [mdev_supported_types] + ------------ [vfio_ap-passthrough] + --------------- [devices] + ------------------ [$uuid1] + --------------------- assign_adapter + --------------------- assign_control_domain + --------------------- assign_domain + --------------------- matrix + --------------------- unassign_adapter + --------------------- unassign_control_domain + --------------------- unassign_domain + + ------------------ [$uuid2] + --------------------- assign_adapter + --------------------- assign_cTo assign an adapter, the APID of the ada= pter is written to the + file. ontrol_domain + --------------------- assign_domain + --------------------- matrix + --------------------- unassign_adapter + --------------------- unassign_control_domain + --------------------- unassign_domain + +4. The administrator now needs to configure the matrixes for mediated + devices $uuid1 (for Guest1) and $uuid2 (for Guest2). + + This is how the matrix is configured for Guest1: + + echo 5 > assign_adapter + echo 6 > assign_adapter + echo 4 > assign_domain + echo 0xab > assign_domain + + By architectural convention, all usage domains - i.e., domains assigned + via the assign_domain attribute file - will also be configured in the A= DM + field of the KVM guest's CRYCB, so there is no need to assign control + domains here unless you want to assign control domains that are not + assigned as usage domains. + + If a mistake is made configuring an adapter, domain or control domain, + you can use the unassign_xxx files to unassign the adapter, domain or + control domain. + + To display the matrix configuration for Guest1: + + cat matrix + + This is how the matrix is configured for Guest2: + + echo 5 > assign_adapter + echo 0x47 > assign_domain + echo 0xff > assign_domain + +5. Start Guest1 + + /usr/bin/qemu-system-s390x ... -device vfio-ap,sysfsdev=3D/sys/devices/= vfio_ap/matrix/$uuid1 ... + +6. Start Guest2 + + /usr/bin/qemu-system-s390x ... -device vfio-ap,sysfsdev=3D/sys/devices/= vfio_ap/matrix/$uuid2 ... --=20 1.7.1