From nobody Sun May 19 12:13:38 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1685142977; cv=none; d=zohomail.com; s=zohoarc; b=cqxQwyhcQF6GFmoPwWwgaeSKHzv9u7KjxmYpO+j3btZ2/txH30akaJBF03IThHKi6NEB36ECxdxh2GvAjG8pSB2v25moA2Qsj8pjGJGHWxxqEgVeoXkdlVYSHRZGEtZP4324RFbFM0UsIhIyRJfmOXnsvZAytXRUREvfUFvZjSw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1685142977; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=RYrP0o+CAqP3kVFMwvnEaTCaiiKfYNj8nTDnmx99pDo=; b=FWOgmp625RCWVlAAyXFe8kBbk4A8uj/2jENoHJRYlQSSsG/WLC7pXs+cUj6lGza2h2s8Z11Lo+biZzwoU8U5QkFcxvOrS1Oc5rd0Vlf+PfBy4u4lZgiAHj2vU/yku810zAfNWo5cZ5Vkba1ABwB/wEeqUmGD6Dt91ElAzZdklSg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 168514297701311.420920333786057; Fri, 26 May 2023 16:16:17 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q2gfL-0004di-7p; Fri, 26 May 2023 19:16:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q2gfJ-0004cR-C8 for qemu-devel@nongnu.org; Fri, 26 May 2023 19:16:09 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q2gfH-0007yr-Mo for qemu-devel@nongnu.org; Fri, 26 May 2023 19:16:09 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-227-658i01JjMOCa8covHj8yEQ-1; Fri, 26 May 2023 19:16:05 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3AB1C8002BF; Fri, 26 May 2023 23:16:05 +0000 (UTC) Received: from omen.home.shazbot.org (unknown [10.22.10.168]) by smtp.corp.redhat.com (Postfix) with ESMTP id B51F740C1438; Fri, 26 May 2023 23:16:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685142966; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RYrP0o+CAqP3kVFMwvnEaTCaiiKfYNj8nTDnmx99pDo=; b=LJ3tIRghVTGtGSbknXYW2dEDkARv4jI3HvPsfK9z7FL7jiZQzXIi+aIG8l0PldbslC0o1+ rb2pF/naqASTRn0o4LDN5lQIVP6rcjsbrhkncLCjwDnjQSgzQayvphQ5Jynd9izXlZpU1b mfAiIEgZA2oxH/gOVBoQQxFwBZLJlQI= X-MC-Unique: 658i01JjMOCa8covHj8yEQ-1 From: Alex Williamson To: robin@streamhpc.com, mst@redhat.com, marcel.apfelbaum@gmail.com Cc: Alex Williamson , qemu-devel@nongnu.org, clg@redhat.com Subject: [RFC PATCH v2 1/4] linux-headers: Update for vfio capability reporting AtomicOps Date: Fri, 26 May 2023 17:15:55 -0600 Message-Id: <20230526231558.1660396-2-alex.williamson@redhat.com> In-Reply-To: <20230526231558.1660396-1-alex.williamson@redhat.com> References: <20230526231558.1660396-1-alex.williamson@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=alex.williamson@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1685142978945100003 Content-Type: text/plain; charset="utf-8" This is a partial linux-headers update for illustrative and testing purposes only, NOT FOR COMMIT. Signed-off-by: Alex Williamson --- linux-headers/linux/vfio.h | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/linux-headers/linux/vfio.h b/linux-headers/linux/vfio.h index 4a534edbdcba..443a8851e156 100644 --- a/linux-headers/linux/vfio.h +++ b/linux-headers/linux/vfio.h @@ -240,6 +240,20 @@ struct vfio_device_info { #define VFIO_DEVICE_INFO_CAP_ZPCI_UTIL 3 #define VFIO_DEVICE_INFO_CAP_ZPCI_PFIP 4 =20 +/* + * The following VFIO_DEVICE_INFO capability reports support for PCIe Atom= icOp + * completion to the root bus with supported widths provided via flags. + */ +#define VFIO_DEVICE_INFO_CAP_PCI_ATOMIC_COMP 5 +struct vfio_device_info_cap_pci_atomic_comp { + struct vfio_info_cap_header header; + __u32 flags; +#define VFIO_PCI_ATOMIC_COMP32 (1 << 0) +#define VFIO_PCI_ATOMIC_COMP64 (1 << 1) +#define VFIO_PCI_ATOMIC_COMP128 (1 << 2) + __u32 reserved; +}; + /** * VFIO_DEVICE_GET_REGION_INFO - _IOWR(VFIO_TYPE, VFIO_BASE + 8, * struct vfio_region_info) --=20 2.39.2 From nobody Sun May 19 12:13:38 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1685143025; cv=none; d=zohomail.com; s=zohoarc; b=TtmR4/7jRKsx5o6g34f3VSQaSuBNf47ETfWmZnldqFQByljsJ8r5CC5yq2G9Ml3+0jpQQjTYK4q17E5TCyhej1TJmWPUUYgJIi7fO+QCCfwcFJl+NDpBLsoCyIBQf30+Iug6yqgYmDGvifC5nvSsv/NNyN9JHa6/G17TwaYxkTo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1685143025; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=eMaYG4q1Nk0spk2aBov6NS66YAeRyg9c57fbvvP0aws=; b=cdeeyV1Aq/ktIbvzNkNw2P2xDY7UGfJQSsTCwCeeNg5EVi9a9HruWIKSgO+WgNEZAbAyBKkOEPxo9PM6qRTBBCOu6vunY+1gCyeMG+o9GQYfH7ZVYoCPS1no9lG5+S9Eumuw5yWOfK2rP6uUe3pbRPc8TKAUEdNYDQBOWhWJMyM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 16851430254991005.6833281271095; Fri, 26 May 2023 16:17:05 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q2gfO-0004fL-1y; Fri, 26 May 2023 19:16:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q2gfM-0004ec-F3 for qemu-devel@nongnu.org; Fri, 26 May 2023 19:16:12 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q2gfK-00082B-NN for qemu-devel@nongnu.org; Fri, 26 May 2023 19:16:12 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-312-qQk1rReBPKmi1Z8b4mZxZA-1; Fri, 26 May 2023 19:16:06 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id DCB55811E7C; Fri, 26 May 2023 23:16:05 +0000 (UTC) Received: from omen.home.shazbot.org (unknown [10.22.10.168]) by smtp.corp.redhat.com (Postfix) with ESMTP id 54128400E118; Fri, 26 May 2023 23:16:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685142969; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eMaYG4q1Nk0spk2aBov6NS66YAeRyg9c57fbvvP0aws=; b=TKLiVpxyfBlQ79197HWO/hvl7hlfpOZS2LAYpSHnaCBqVF3lG4gEXe5JuvjFxgEVVxIRs+ eKiqQcjnOa3F8m8g3CYqJpyVzYHOpfMzUPvJ/bJ4Cbbi7X7ZIgLe3XxoJxOAMmrS7FOj9d Qr2FkdOiPvnrTe/i3ggtODEtH/SLebA= X-MC-Unique: qQk1rReBPKmi1Z8b4mZxZA-1 From: Alex Williamson To: robin@streamhpc.com, mst@redhat.com, marcel.apfelbaum@gmail.com Cc: Alex Williamson , qemu-devel@nongnu.org, clg@redhat.com, =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= Subject: [RFC PATCH v2 2/4] vfio: Implement a common device info helper Date: Fri, 26 May 2023 17:15:56 -0600 Message-Id: <20230526231558.1660396-3-alex.williamson@redhat.com> In-Reply-To: <20230526231558.1660396-1-alex.williamson@redhat.com> References: <20230526231558.1660396-1-alex.williamson@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=alex.williamson@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1685143027475100003 A common helper implementing the realloc algorithm for handling capabilities. Reviewed-by: Philippe Mathieu-Daud=C3=A9 Reviewed-by: C=C3=A9dric Le Goater Signed-off-by: Alex Williamson --- hw/s390x/s390-pci-vfio.c | 37 ++++------------------------ hw/vfio/common.c | 46 ++++++++++++++++++++++++++--------- include/hw/vfio/vfio-common.h | 1 + 3 files changed, 41 insertions(+), 43 deletions(-) diff --git a/hw/s390x/s390-pci-vfio.c b/hw/s390x/s390-pci-vfio.c index f51190d4662f..66e8ec3bdef9 100644 --- a/hw/s390x/s390-pci-vfio.c +++ b/hw/s390x/s390-pci-vfio.c @@ -289,38 +289,11 @@ static void s390_pci_read_pfip(S390PCIBusDevice *pbde= v, memcpy(pbdev->zpci_fn.pfip, cap->pfip, CLP_PFIP_NR_SEGMENTS); } =20 -static struct vfio_device_info *get_device_info(S390PCIBusDevice *pbdev, - uint32_t argsz) +static struct vfio_device_info *get_device_info(S390PCIBusDevice *pbdev); { - struct vfio_device_info *info =3D g_malloc0(argsz); - VFIOPCIDevice *vfio_pci; - int fd; + VFIOPCIDevice *vfio_pci =3D container_of(pbdev->pdev, VFIOPCIDevice, p= dev); =20 - vfio_pci =3D container_of(pbdev->pdev, VFIOPCIDevice, pdev); - fd =3D vfio_pci->vbasedev.fd; - - /* - * If the specified argsz is not large enough to contain all capabilit= ies - * it will be updated upon return from the ioctl. Retry until we have - * a big enough buffer to hold the entire capability chain. On error, - * just exit and rely on CLP defaults. - */ -retry: - info->argsz =3D argsz; - - if (ioctl(fd, VFIO_DEVICE_GET_INFO, info)) { - trace_s390_pci_clp_dev_info(vfio_pci->vbasedev.name); - g_free(info); - return NULL; - } - - if (info->argsz > argsz) { - argsz =3D info->argsz; - info =3D g_realloc(info, argsz); - goto retry; - } - - return info; + return vfio_get_device_info(vfio_pci->vbasedev.fd); } =20 /* @@ -335,7 +308,7 @@ bool s390_pci_get_host_fh(S390PCIBusDevice *pbdev, uint= 32_t *fh) =20 assert(fh); =20 - info =3D get_device_info(pbdev, sizeof(*info)); + info =3D get_device_info(pbdev); if (!info) { return false; } @@ -356,7 +329,7 @@ void s390_pci_get_clp_info(S390PCIBusDevice *pbdev) { g_autofree struct vfio_device_info *info =3D NULL; =20 - info =3D get_device_info(pbdev, sizeof(*info)); + info =3D get_device_info(pbdev); if (!info) { return; } diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 78358ede2764..ed142296e9fe 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -2843,11 +2843,35 @@ void vfio_put_group(VFIOGroup *group) } } =20 +struct vfio_device_info *vfio_get_device_info(int fd) +{ + struct vfio_device_info *info; + uint32_t argsz =3D sizeof(*info); + + info =3D g_malloc0(argsz); + +retry: + info->argsz =3D argsz; + + if (ioctl(fd, VFIO_DEVICE_GET_INFO, info)) { + g_free(info); + return NULL; + } + + if (info->argsz > argsz) { + argsz =3D info->argsz; + info =3D g_realloc(info, argsz); + goto retry; + } + + return info; +} + int vfio_get_device(VFIOGroup *group, const char *name, VFIODevice *vbasedev, Error **errp) { - struct vfio_device_info dev_info =3D { .argsz =3D sizeof(dev_info) }; - int ret, fd; + g_autofree struct vfio_device_info *info =3D NULL; + int fd; =20 fd =3D ioctl(group->fd, VFIO_GROUP_GET_DEVICE_FD, name); if (fd < 0) { @@ -2859,11 +2883,11 @@ int vfio_get_device(VFIOGroup *group, const char *n= ame, return fd; } =20 - ret =3D ioctl(fd, VFIO_DEVICE_GET_INFO, &dev_info); - if (ret) { + info =3D vfio_get_device_info(fd); + if (!info) { error_setg_errno(errp, errno, "error getting device info"); close(fd); - return ret; + return -1; } =20 /* @@ -2891,14 +2915,14 @@ int vfio_get_device(VFIOGroup *group, const char *n= ame, vbasedev->group =3D group; QLIST_INSERT_HEAD(&group->device_list, vbasedev, next); =20 - vbasedev->num_irqs =3D dev_info.num_irqs; - vbasedev->num_regions =3D dev_info.num_regions; - vbasedev->flags =3D dev_info.flags; + vbasedev->num_irqs =3D info->num_irqs; + vbasedev->num_regions =3D info->num_regions; + vbasedev->flags =3D info->flags; + + trace_vfio_get_device(name, info->flags, info->num_regions, info->num_= irqs); =20 - trace_vfio_get_device(name, dev_info.flags, dev_info.num_regions, - dev_info.num_irqs); + vbasedev->reset_works =3D !!(info->flags & VFIO_DEVICE_FLAGS_RESET); =20 - vbasedev->reset_works =3D !!(dev_info.flags & VFIO_DEVICE_FLAGS_RESET); return 0; } =20 diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index eed244f25f34..a8dcda592c08 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -212,6 +212,7 @@ void vfio_region_finalize(VFIORegion *region); void vfio_reset_handler(void *opaque); VFIOGroup *vfio_get_group(int groupid, AddressSpace *as, Error **errp); void vfio_put_group(VFIOGroup *group); +struct vfio_device_info *vfio_get_device_info(int fd); int vfio_get_device(VFIOGroup *group, const char *name, VFIODevice *vbasedev, Error **errp); =20 --=20 2.39.2 From nobody Sun May 19 12:13:38 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1685143018; cv=none; d=zohomail.com; s=zohoarc; b=YE55Sy7p6gHfLVZ9zkOn8bFWItccoX4gjK3t9VZ92QSA/2v2jJtgS0soITeKuSl7+4RLrMAo2UYiLoWg7YdDQczQ04V37NbwXEJxFw6zQBJ9Oh9lKnQn2ORGjZGbuEDld93pZkCjkfoI0AJQlgr6koXUfjo4drwsnr9m7wjiJHg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1685143018; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=ppsSqBFmd85MY/HWcySwkiiUV0ay/ybg6reH9NMFing=; b=EKC0R6Au/wS15FBnwfHI2M/6kFLxk9HZdb+3BqOzFnKW6MWsVTVeaIbCFIoJ45CVQ8BnVXhYtiNhGFvvk9ubLrdU2pZ97Tm/+S3KzR4f/NLg3yNE/swM0ytRnzAqBrjy/alLZyI4ggcLVwNecBI6PS/jcKyAq2wLpmos1InBqR8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1685143018666491.97259441609117; Fri, 26 May 2023 16:16:58 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q2gfO-0004fa-Lz; Fri, 26 May 2023 19:16:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q2gfM-0004ei-J9 for qemu-devel@nongnu.org; Fri, 26 May 2023 19:16:12 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q2gfK-00082L-UV for qemu-devel@nongnu.org; Fri, 26 May 2023 19:16:12 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-461-1svPI7B4MU-_RFXXAGRaeQ-1; Fri, 26 May 2023 19:16:06 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 704BB2A2AD44; Fri, 26 May 2023 23:16:06 +0000 (UTC) Received: from omen.home.shazbot.org (unknown [10.22.10.168]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0191140C1438; Fri, 26 May 2023 23:16:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685142970; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ppsSqBFmd85MY/HWcySwkiiUV0ay/ybg6reH9NMFing=; b=Hh+Hjq3RKgjel8nIvcTmpkq5At7MxNNRuML4QEit+eV8eML5A0dHAOUC+FV4P4jxYGWcVX 9g555qD/tkKVWu+UBMZpemP7+kUaDLQ+b7s1T/RIv/B8FU2X/GkE5Ch+BncS9kJB9+w1NI 3PREdGSysLFY4tst6vh9ZIUiYxmxgtQ= X-MC-Unique: 1svPI7B4MU-_RFXXAGRaeQ-1 From: Alex Williamson To: robin@streamhpc.com, mst@redhat.com, marcel.apfelbaum@gmail.com Cc: Alex Williamson , qemu-devel@nongnu.org, clg@redhat.com Subject: [RFC PATCH v2 3/4] pcie: Add a PCIe capability version helper Date: Fri, 26 May 2023 17:15:57 -0600 Message-Id: <20230526231558.1660396-4-alex.williamson@redhat.com> In-Reply-To: <20230526231558.1660396-1-alex.williamson@redhat.com> References: <20230526231558.1660396-1-alex.williamson@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=alex.williamson@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1685143019035100007 Content-Type: text/plain; charset="utf-8" Report the PCIe capability version for a device Signed-off-by: Alex Williamson Reviewed-by: C=C3=A9dric Le Goater Reviewed-by: Philippe Mathieu-Daud=C3=A9 Reviewed-by: Robin Voetter Tested-by: Robin Voetter --- hw/pci/pcie.c | 7 +++++++ include/hw/pci/pcie.h | 1 + 2 files changed, 8 insertions(+) diff --git a/hw/pci/pcie.c b/hw/pci/pcie.c index b8c24cf45f7e..b7f107ed8dd4 100644 --- a/hw/pci/pcie.c +++ b/hw/pci/pcie.c @@ -274,6 +274,13 @@ uint8_t pcie_cap_get_type(const PCIDevice *dev) PCI_EXP_FLAGS_TYPE) >> PCI_EXP_FLAGS_TYPE_SHIFT; } =20 +uint8_t pcie_cap_get_version(const PCIDevice *dev) +{ + uint32_t pos =3D dev->exp.exp_cap; + assert(pos > 0); + return pci_get_word(dev->config + pos + PCI_EXP_FLAGS) & PCI_EXP_FLAGS= _VERS; +} + /* MSI/MSI-X */ /* pci express interrupt message number */ /* 7.8.2 PCI Express Capabilities Register: Interrupt Message Number */ diff --git a/include/hw/pci/pcie.h b/include/hw/pci/pcie.h index 3cc2b159570f..51ab57bc3c50 100644 --- a/include/hw/pci/pcie.h +++ b/include/hw/pci/pcie.h @@ -93,6 +93,7 @@ void pcie_cap_exit(PCIDevice *dev); int pcie_endpoint_cap_v1_init(PCIDevice *dev, uint8_t offset); void pcie_cap_v1_exit(PCIDevice *dev); uint8_t pcie_cap_get_type(const PCIDevice *dev); +uint8_t pcie_cap_get_version(const PCIDevice *dev); void pcie_cap_flags_set_vector(PCIDevice *dev, uint8_t vector); uint8_t pcie_cap_flags_get_vector(PCIDevice *dev); =20 --=20 2.39.2 From nobody Sun May 19 12:13:38 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1685143017; cv=none; d=zohomail.com; s=zohoarc; b=Eugk/8M1TSikTzCBm2Dr24e6RNFqwPOqPrM5UjNWz80g2MzAN557EN49G8tUmPBNWjgk63Dx5wACbJByBVjnKdViszHCxV7pjOToi5XuhB9caHwzgpWTjji2xveb/kXEjcbSj+ExxUEMd7sWAElFTsEq7dzMKxlI68jZcy6KD8g= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1685143017; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=woZ49TSbxbpvJ//MBsCbLlx2R9jmGAibuJOnxx5U/Xg=; b=nX5L9V3OmktpEsh3SFgKlmeZ+RA9lxlpyPDXiXv5yxwm8SapaG6ZLhJE+xpuaWumwW1PvGpwV575OQBGLBfkokYcTxYNjNLD8IT11XD9PVY7998plHKceapNM3b0Y31wlDgajUlojb7QlsI4i54/K2rpuJiR7Aeavv8KlyOPq3Y= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1685143017327932.1103747508023; Fri, 26 May 2023 16:16:57 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q2gfP-0004fc-8R; Fri, 26 May 2023 19:16:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q2gfN-0004f3-9z for qemu-devel@nongnu.org; Fri, 26 May 2023 19:16:13 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q2gfL-00082z-Df for qemu-devel@nongnu.org; Fri, 26 May 2023 19:16:12 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-168-AlVYwEiGOdOygblnOOLRuw-1; Fri, 26 May 2023 19:16:07 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0FC7C1C04180; Fri, 26 May 2023 23:16:07 +0000 (UTC) Received: from omen.home.shazbot.org (unknown [10.22.10.168]) by smtp.corp.redhat.com (Postfix) with ESMTP id 892BB40D1B66; Fri, 26 May 2023 23:16:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685142970; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=woZ49TSbxbpvJ//MBsCbLlx2R9jmGAibuJOnxx5U/Xg=; b=MxvuwKpvjibAjSzVwbgzvFgo1EUO7A40Cz1l5CeN+o6fFdoTj+xwk3oBOdmskeHaFydIZ5 xRFxEILBo/u706/caEMiMb1toYXEc/C53HLpM41GpX6+Ynh9cX2S3BeFLXOrT9ewQjwgaE 6vYq4Q2VMQt8UibJwaN1v2n4b3W6JW0= X-MC-Unique: AlVYwEiGOdOygblnOOLRuw-1 From: Alex Williamson To: robin@streamhpc.com, mst@redhat.com, marcel.apfelbaum@gmail.com Cc: Alex Williamson , qemu-devel@nongnu.org, clg@redhat.com Subject: [RFC PATCH v2 4/4] vfio/pci: Enable AtomicOps completers on root ports Date: Fri, 26 May 2023 17:15:58 -0600 Message-Id: <20230526231558.1660396-5-alex.williamson@redhat.com> In-Reply-To: <20230526231558.1660396-1-alex.williamson@redhat.com> References: <20230526231558.1660396-1-alex.williamson@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=alex.williamson@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1685143018823100005 Content-Type: text/plain; charset="utf-8" Dynamically enable Atomic Ops completer support around realize/exit of vfio-pci devices reporting host support for these accesses and adhering to a minimal configuration standard. While the Atomic Ops completer bits in the root port device capabilities2 register are read-only, the PCIe spec does allow RO bits to change to reflect hardware state. We take advantage of that here around the realize and exit functions of the vfio-pci device. Signed-off-by: Alex Williamson Reviewed-by: Philippe Mathieu-Daud=C3=A9 Reviewed-by: Robin Voetter Tested-by: Robin Voetter --- hw/vfio/pci.c | 78 +++++++++++++++++++++++++++++++++++++++++++++++++++ hw/vfio/pci.h | 1 + 2 files changed, 79 insertions(+) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index bf27a3990564..d8a0fd595560 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -1826,6 +1826,81 @@ static void vfio_add_emulated_long(VFIOPCIDevice *vd= ev, int pos, vfio_set_long_bits(vdev->emulated_config_bits + pos, mask, mask); } =20 +static void vfio_pci_enable_rp_atomics(VFIOPCIDevice *vdev) +{ + struct vfio_device_info_cap_pci_atomic_comp *cap; + g_autofree struct vfio_device_info *info =3D NULL; + PCIBus *bus =3D pci_get_bus(&vdev->pdev); + PCIDevice *parent =3D bus->parent_dev; + struct vfio_info_cap_header *hdr; + uint32_t mask =3D 0; + uint8_t *pos; + + /* + * PCIe Atomic Ops completer support is only added automatically for s= ingle + * function devices downstream of a root port supporting DEVCAP2. Sup= port + * is added during realize and, if added, removed during device exit. = The + * single function requirement avoids conflicting requirements should a + * slot be composed of multiple devices with differing capabilities. + */ + if (pci_bus_is_root(bus) || !parent || !parent->exp.exp_cap || + pcie_cap_get_type(parent) !=3D PCI_EXP_TYPE_ROOT_PORT || + pcie_cap_get_version(parent) !=3D PCI_EXP_FLAGS_VER2 || + vdev->pdev.devfn || + vdev->pdev.cap_present & QEMU_PCI_CAP_MULTIFUNCTION) { + return; + } + + pos =3D parent->config + parent->exp.exp_cap + PCI_EXP_DEVCAP2; + + /* Abort if there'a already an Atomic Ops configuration on the root po= rt */ + if (pci_get_long(pos) & (PCI_EXP_DEVCAP2_ATOMIC_COMP32 | + PCI_EXP_DEVCAP2_ATOMIC_COMP64 | + PCI_EXP_DEVCAP2_ATOMIC_COMP128)) { + return; + } + + info =3D vfio_get_device_info(vdev->vbasedev.fd); + if (!info) { + return; + } + + hdr =3D vfio_get_device_info_cap(info, VFIO_DEVICE_INFO_CAP_PCI_ATOMIC= _COMP); + if (!hdr) { + return; + } + + cap =3D (void *)hdr; + if (cap->flags & VFIO_PCI_ATOMIC_COMP32) { + mask |=3D PCI_EXP_DEVCAP2_ATOMIC_COMP32; + } + if (cap->flags & VFIO_PCI_ATOMIC_COMP64) { + mask |=3D PCI_EXP_DEVCAP2_ATOMIC_COMP64; + } + if (cap->flags & VFIO_PCI_ATOMIC_COMP128) { + mask |=3D PCI_EXP_DEVCAP2_ATOMIC_COMP128; + } + + if (!mask) { + return; + } + + pci_long_test_and_set_mask(pos, mask); + vdev->clear_parent_atomics_on_exit =3D true; +} + +static void vfio_pci_disable_rp_atomics(VFIOPCIDevice *vdev) +{ + if (vdev->clear_parent_atomics_on_exit) { + PCIDevice *parent =3D pci_get_bus(&vdev->pdev)->parent_dev; + uint8_t *pos =3D parent->config + parent->exp.exp_cap + PCI_EXP_DE= VCAP2; + + pci_long_test_and_clear_mask(pos, PCI_EXP_DEVCAP2_ATOMIC_COMP32 | + PCI_EXP_DEVCAP2_ATOMIC_COMP64 | + PCI_EXP_DEVCAP2_ATOMIC_COMP128); + } +} + static int vfio_setup_pcie_cap(VFIOPCIDevice *vdev, int pos, uint8_t size, Error **errp) { @@ -1929,6 +2004,8 @@ static int vfio_setup_pcie_cap(VFIOPCIDevice *vdev, i= nt pos, uint8_t size, QEMU_PCI_EXP_LNKCAP_MLS(QEMU_PCI_EXP_LNK_2_5GT)= , ~0); vfio_add_emulated_word(vdev, pos + PCI_EXP_LNKCTL, 0, ~0); } + + vfio_pci_enable_rp_atomics(vdev); } =20 /* @@ -3265,6 +3342,7 @@ static void vfio_exitfn(PCIDevice *pdev) timer_free(vdev->intx.mmap_timer); } vfio_teardown_msi(vdev); + vfio_pci_disable_rp_atomics(vdev); vfio_bars_exit(vdev); vfio_migration_exit(&vdev->vbasedev); } diff --git a/hw/vfio/pci.h b/hw/vfio/pci.h index 2674476d6c77..a2771b9ff3cc 100644 --- a/hw/vfio/pci.h +++ b/hw/vfio/pci.h @@ -174,6 +174,7 @@ struct VFIOPCIDevice { bool no_vfio_ioeventfd; bool enable_ramfb; bool defer_kvm_irq_routing; + bool clear_parent_atomics_on_exit; VFIODisplay *dpy; Notifier irqchip_change_notifier; }; --=20 2.39.2