From nobody Sun May 19 13:08:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1684533503; cv=none; d=zohomail.com; s=zohoarc; b=PeAN8xviVmSSTf0jLy1l0l4JpYzFGbl48fqEwwsRwxA/ctJzbsq/1KNJscOynQi5gRp095LLNlElob/lpzeYxByQer0S/Msk9QlyRe/mVTVWdIWGZXfi1h4vFdLnxjIXrOgd+XtlH71W/pHaAtG+hD9bj+BPcSnj1Sb5rxTexws= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1684533503; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=RYrP0o+CAqP3kVFMwvnEaTCaiiKfYNj8nTDnmx99pDo=; b=ZIEx8SAoP+Km90JOYecJvnKreDaU2gf2XLy8N+Xl0IZke4mRd48Wq5FPbRU79SPsO3rWoh5XMdXwR9rvz+Bk2yIogA8aVjbWctMCER/CJEC4AtvREyBEfqO/wezHHeke1FwiufPeGAuvEYOj+oHikxqj4mEOCaRsbcysTHoFm8Q= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1684533503981189.77642971454202; Fri, 19 May 2023 14:58:23 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q086k-0005wU-Nm; Fri, 19 May 2023 17:57:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q086j-0005wE-Fc for qemu-devel@nongnu.org; Fri, 19 May 2023 17:57:53 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q086h-0004F4-Op for qemu-devel@nongnu.org; Fri, 19 May 2023 17:57:53 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-467-ZA92AA6lN1uOzK4upMqo0w-1; Fri, 19 May 2023 17:57:49 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id AAEB4101A53B; Fri, 19 May 2023 21:57:48 +0000 (UTC) Received: from omen.home.shazbot.org (unknown [10.22.10.168]) by smtp.corp.redhat.com (Postfix) with ESMTP id 43AA8C0004B; Fri, 19 May 2023 21:57:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684533470; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RYrP0o+CAqP3kVFMwvnEaTCaiiKfYNj8nTDnmx99pDo=; b=OMk6z/hlMzlrb/jKzCUH1YVS2lcP8o9XCu8+WVE9ZzdZiLpX2poybl2OXzNj7en564J3wC ERXMAq02Zv5mge6r0aWbIe9R5hYbaLhXy7yHZGbZILhKTi1/i+38cq3MoMfXEnCgPdUUFN vPAmqKIFwCZ5BOn1awaPiX+SqsAA9g8= X-MC-Unique: ZA92AA6lN1uOzK4upMqo0w-1 From: Alex Williamson To: robin@streamhpc.com, mst@redhat.com, marcel.apfelbaum@gmail.com Cc: Alex Williamson , qemu-devel@nongnu.org Subject: [RFC PATCH 1/4] linux-headers: Update for vfio capability reporting AtomicOps Date: Fri, 19 May 2023 15:57:36 -0600 Message-Id: <20230519215739.402729-2-alex.williamson@redhat.com> In-Reply-To: <20230519215739.402729-1-alex.williamson@redhat.com> References: <20230519215739.402729-1-alex.williamson@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=alex.williamson@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1684533504565100001 Content-Type: text/plain; charset="utf-8" This is a partial linux-headers update for illustrative and testing purposes only, NOT FOR COMMIT. Signed-off-by: Alex Williamson --- linux-headers/linux/vfio.h | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/linux-headers/linux/vfio.h b/linux-headers/linux/vfio.h index 4a534edbdcba..443a8851e156 100644 --- a/linux-headers/linux/vfio.h +++ b/linux-headers/linux/vfio.h @@ -240,6 +240,20 @@ struct vfio_device_info { #define VFIO_DEVICE_INFO_CAP_ZPCI_UTIL 3 #define VFIO_DEVICE_INFO_CAP_ZPCI_PFIP 4 =20 +/* + * The following VFIO_DEVICE_INFO capability reports support for PCIe Atom= icOp + * completion to the root bus with supported widths provided via flags. + */ +#define VFIO_DEVICE_INFO_CAP_PCI_ATOMIC_COMP 5 +struct vfio_device_info_cap_pci_atomic_comp { + struct vfio_info_cap_header header; + __u32 flags; +#define VFIO_PCI_ATOMIC_COMP32 (1 << 0) +#define VFIO_PCI_ATOMIC_COMP64 (1 << 1) +#define VFIO_PCI_ATOMIC_COMP128 (1 << 2) + __u32 reserved; +}; + /** * VFIO_DEVICE_GET_REGION_INFO - _IOWR(VFIO_TYPE, VFIO_BASE + 8, * struct vfio_region_info) --=20 2.39.2 From nobody Sun May 19 13:08:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1684533539; cv=none; d=zohomail.com; s=zohoarc; b=NEa7VzxBe6OhARxzm9rzV34H6kSNX6OWy+1NqqpFfmNV11+dOuP/cPMQ62XbEGUBWskr9iTx3MjLqqyUZDRIrb7dtli6m2+kH76tHbKTNUgW3Vm+q9DKNv1nGisDXFR0ynUCKPBUiNzRxbImEW4lzpnkoqy+bp4XLdqgO7lIqHM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1684533539; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=wZvw6QjxR2MUxzxGaroxNq9sovUgvJA+2KZO2vFMJUE=; b=DJ2UtHKlzAQzfpReFfSdg/cb/Pm6vYDlRaOlHkxT3ytjln8L067Lx3uY1QM/6BvsqUkRY2zl2Ku4iiZGhnQ9SH7VzY8IxzbOMoMeswrLJGLlvOCXb150+RB2wyePgJMlrMWHcVI1wGy8T3l0ikmIGaVzBhNFMsnPbFSksJ/jn8U= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1684533539587926.5800980715416; Fri, 19 May 2023 14:58:59 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q086n-0005yu-II; Fri, 19 May 2023 17:57:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q086l-0005wf-59 for qemu-devel@nongnu.org; Fri, 19 May 2023 17:57:55 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q086j-0004FD-Gr for qemu-devel@nongnu.org; Fri, 19 May 2023 17:57:54 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-507-2kliOwucOyWLTq6lTVPw1g-1; Fri, 19 May 2023 17:57:49 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 39680101A53A; Fri, 19 May 2023 21:57:49 +0000 (UTC) Received: from omen.home.shazbot.org (unknown [10.22.10.168]) by smtp.corp.redhat.com (Postfix) with ESMTP id C3EE5C0004B; Fri, 19 May 2023 21:57:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684533472; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wZvw6QjxR2MUxzxGaroxNq9sovUgvJA+2KZO2vFMJUE=; b=ZRyOYFNhZDtdJMu6Yqs1WTVWn6POasV4mL2rB14x2SDEVoluNwm9dow8pdqs1r4Jb4+pU8 lichmv2Ye1tiTSkYC6CAYg6O7M/hpopR0BDr5jUNUvqhibMrzWb2Iz272PCMnp8wBdQz8i cfFLBWcADbukFWZy1A5ogA1aGkYyQec= X-MC-Unique: 2kliOwucOyWLTq6lTVPw1g-1 From: Alex Williamson To: robin@streamhpc.com, mst@redhat.com, marcel.apfelbaum@gmail.com Cc: Alex Williamson , qemu-devel@nongnu.org Subject: [RFC PATCH 2/4] vfio: Implement a common device info helper Date: Fri, 19 May 2023 15:57:37 -0600 Message-Id: <20230519215739.402729-3-alex.williamson@redhat.com> In-Reply-To: <20230519215739.402729-1-alex.williamson@redhat.com> References: <20230519215739.402729-1-alex.williamson@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=alex.williamson@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1684533540771100003 Content-Type: text/plain; charset="utf-8" A common helper implementing the realloc algorithm for handling capabilities. Signed-off-by: Alex Williamson Reviewed-by: C=C3=A9dric Le Goater Reviewed-by: Philippe Mathieu-Daud=C3=A9 --- hw/s390x/s390-pci-vfio.c | 37 ++++------------------------ hw/vfio/common.c | 46 ++++++++++++++++++++++++++--------- include/hw/vfio/vfio-common.h | 1 + 3 files changed, 41 insertions(+), 43 deletions(-) diff --git a/hw/s390x/s390-pci-vfio.c b/hw/s390x/s390-pci-vfio.c index f51190d4662f..66e8ec3bdef9 100644 --- a/hw/s390x/s390-pci-vfio.c +++ b/hw/s390x/s390-pci-vfio.c @@ -289,38 +289,11 @@ static void s390_pci_read_pfip(S390PCIBusDevice *pbde= v, memcpy(pbdev->zpci_fn.pfip, cap->pfip, CLP_PFIP_NR_SEGMENTS); } =20 -static struct vfio_device_info *get_device_info(S390PCIBusDevice *pbdev, - uint32_t argsz) +static struct vfio_device_info *get_device_info(S390PCIBusDevice *pbdev); { - struct vfio_device_info *info =3D g_malloc0(argsz); - VFIOPCIDevice *vfio_pci; - int fd; + VFIOPCIDevice *vfio_pci =3D container_of(pbdev->pdev, VFIOPCIDevice, p= dev); =20 - vfio_pci =3D container_of(pbdev->pdev, VFIOPCIDevice, pdev); - fd =3D vfio_pci->vbasedev.fd; - - /* - * If the specified argsz is not large enough to contain all capabilit= ies - * it will be updated upon return from the ioctl. Retry until we have - * a big enough buffer to hold the entire capability chain. On error, - * just exit and rely on CLP defaults. - */ -retry: - info->argsz =3D argsz; - - if (ioctl(fd, VFIO_DEVICE_GET_INFO, info)) { - trace_s390_pci_clp_dev_info(vfio_pci->vbasedev.name); - g_free(info); - return NULL; - } - - if (info->argsz > argsz) { - argsz =3D info->argsz; - info =3D g_realloc(info, argsz); - goto retry; - } - - return info; + return vfio_get_device_info(vfio_pci->vbasedev.fd); } =20 /* @@ -335,7 +308,7 @@ bool s390_pci_get_host_fh(S390PCIBusDevice *pbdev, uint= 32_t *fh) =20 assert(fh); =20 - info =3D get_device_info(pbdev, sizeof(*info)); + info =3D get_device_info(pbdev); if (!info) { return false; } @@ -356,7 +329,7 @@ void s390_pci_get_clp_info(S390PCIBusDevice *pbdev) { g_autofree struct vfio_device_info *info =3D NULL; =20 - info =3D get_device_info(pbdev, sizeof(*info)); + info =3D get_device_info(pbdev); if (!info) { return; } diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 78358ede2764..ed142296e9fe 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -2843,11 +2843,35 @@ void vfio_put_group(VFIOGroup *group) } } =20 +struct vfio_device_info *vfio_get_device_info(int fd) +{ + struct vfio_device_info *info; + uint32_t argsz =3D sizeof(*info); + + info =3D g_malloc0(argsz); + +retry: + info->argsz =3D argsz; + + if (ioctl(fd, VFIO_DEVICE_GET_INFO, info)) { + g_free(info); + return NULL; + } + + if (info->argsz > argsz) { + argsz =3D info->argsz; + info =3D g_realloc(info, argsz); + goto retry; + } + + return info; +} + int vfio_get_device(VFIOGroup *group, const char *name, VFIODevice *vbasedev, Error **errp) { - struct vfio_device_info dev_info =3D { .argsz =3D sizeof(dev_info) }; - int ret, fd; + g_autofree struct vfio_device_info *info =3D NULL; + int fd; =20 fd =3D ioctl(group->fd, VFIO_GROUP_GET_DEVICE_FD, name); if (fd < 0) { @@ -2859,11 +2883,11 @@ int vfio_get_device(VFIOGroup *group, const char *n= ame, return fd; } =20 - ret =3D ioctl(fd, VFIO_DEVICE_GET_INFO, &dev_info); - if (ret) { + info =3D vfio_get_device_info(fd); + if (!info) { error_setg_errno(errp, errno, "error getting device info"); close(fd); - return ret; + return -1; } =20 /* @@ -2891,14 +2915,14 @@ int vfio_get_device(VFIOGroup *group, const char *n= ame, vbasedev->group =3D group; QLIST_INSERT_HEAD(&group->device_list, vbasedev, next); =20 - vbasedev->num_irqs =3D dev_info.num_irqs; - vbasedev->num_regions =3D dev_info.num_regions; - vbasedev->flags =3D dev_info.flags; + vbasedev->num_irqs =3D info->num_irqs; + vbasedev->num_regions =3D info->num_regions; + vbasedev->flags =3D info->flags; + + trace_vfio_get_device(name, info->flags, info->num_regions, info->num_= irqs); =20 - trace_vfio_get_device(name, dev_info.flags, dev_info.num_regions, - dev_info.num_irqs); + vbasedev->reset_works =3D !!(info->flags & VFIO_DEVICE_FLAGS_RESET); =20 - vbasedev->reset_works =3D !!(dev_info.flags & VFIO_DEVICE_FLAGS_RESET); return 0; } =20 diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index eed244f25f34..a8dcda592c08 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -212,6 +212,7 @@ void vfio_region_finalize(VFIORegion *region); void vfio_reset_handler(void *opaque); VFIOGroup *vfio_get_group(int groupid, AddressSpace *as, Error **errp); void vfio_put_group(VFIOGroup *group); +struct vfio_device_info *vfio_get_device_info(int fd); int vfio_get_device(VFIOGroup *group, const char *name, VFIODevice *vbasedev, Error **errp); =20 --=20 2.39.2 From nobody Sun May 19 13:08:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1684533540; cv=none; d=zohomail.com; s=zohoarc; b=CVxcXi7fuhoJES8WJkd3ZWlgfSi3p8Y7q5gU4zbCqskpxYRT5oVqZ96IHxC5423OQrimGU9bGaUELCNdw4DrvHCH2gI4ZQ2QAjP7+IER/go9+QzYGse5e/HDzfrqmOWxFqxhuv6Gwt4iVEGyR5CHnrsdYYyJXBjjMNQBM2Gr6h4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1684533540; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=ppsSqBFmd85MY/HWcySwkiiUV0ay/ybg6reH9NMFing=; b=BHMZkOEyJoDlqiY9Y6m/M0XfvIm+ByhMPfh91pbf0npGTHKXlMX9uO9VAgWFplHot9wjJjq8STVXpQvhYh3lPSk2uiR652YAjk9SZPe8LEMjAatrBrf27DsM+PMyJmGYuGOwJclfKcdH+/M5a6OnjS7/9Z4WooSzogyhYZr6hOM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1684533540475435.5994002946329; Fri, 19 May 2023 14:59:00 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q086n-0005yv-K7; Fri, 19 May 2023 17:57:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q086l-0005x7-Lr for qemu-devel@nongnu.org; Fri, 19 May 2023 17:57:55 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q086k-0004FL-4Q for qemu-devel@nongnu.org; Fri, 19 May 2023 17:57:55 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-135-QJ1nuUh6M9Cj3u7yf_XYhg-1; Fri, 19 May 2023 17:57:50 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id AE58A101A531; Fri, 19 May 2023 21:57:49 +0000 (UTC) Received: from omen.home.shazbot.org (unknown [10.22.10.168]) by smtp.corp.redhat.com (Postfix) with ESMTP id 52CA6C0004B; Fri, 19 May 2023 21:57:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684533473; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ppsSqBFmd85MY/HWcySwkiiUV0ay/ybg6reH9NMFing=; b=a0wjkSifUsoj7Cntn9saJkZrDlmzM3BmKyKjnQeO1+zE9WpdvXuC1R0/1rNVLHIU8VXkv6 Mn5gbHS1t8+QnWPfq2pFXtRWOwgT8eTER3ucPmvrZOOocV+YAZWK87q4Tm2Y5XPwtQN6ea hLpnuFrMtWr3WpRem9Fy+xG08DqdEv4= X-MC-Unique: QJ1nuUh6M9Cj3u7yf_XYhg-1 From: Alex Williamson To: robin@streamhpc.com, mst@redhat.com, marcel.apfelbaum@gmail.com Cc: Alex Williamson , qemu-devel@nongnu.org Subject: [RFC PATCH 3/4] pcie: Add a PCIe capability version helper Date: Fri, 19 May 2023 15:57:38 -0600 Message-Id: <20230519215739.402729-4-alex.williamson@redhat.com> In-Reply-To: <20230519215739.402729-1-alex.williamson@redhat.com> References: <20230519215739.402729-1-alex.williamson@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=alex.williamson@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1684533542546100005 Content-Type: text/plain; charset="utf-8" Report the PCIe capability version for a device Signed-off-by: Alex Williamson --- hw/pci/pcie.c | 7 +++++++ include/hw/pci/pcie.h | 1 + 2 files changed, 8 insertions(+) diff --git a/hw/pci/pcie.c b/hw/pci/pcie.c index b8c24cf45f7e..b7f107ed8dd4 100644 --- a/hw/pci/pcie.c +++ b/hw/pci/pcie.c @@ -274,6 +274,13 @@ uint8_t pcie_cap_get_type(const PCIDevice *dev) PCI_EXP_FLAGS_TYPE) >> PCI_EXP_FLAGS_TYPE_SHIFT; } =20 +uint8_t pcie_cap_get_version(const PCIDevice *dev) +{ + uint32_t pos =3D dev->exp.exp_cap; + assert(pos > 0); + return pci_get_word(dev->config + pos + PCI_EXP_FLAGS) & PCI_EXP_FLAGS= _VERS; +} + /* MSI/MSI-X */ /* pci express interrupt message number */ /* 7.8.2 PCI Express Capabilities Register: Interrupt Message Number */ diff --git a/include/hw/pci/pcie.h b/include/hw/pci/pcie.h index 3cc2b159570f..51ab57bc3c50 100644 --- a/include/hw/pci/pcie.h +++ b/include/hw/pci/pcie.h @@ -93,6 +93,7 @@ void pcie_cap_exit(PCIDevice *dev); int pcie_endpoint_cap_v1_init(PCIDevice *dev, uint8_t offset); void pcie_cap_v1_exit(PCIDevice *dev); uint8_t pcie_cap_get_type(const PCIDevice *dev); +uint8_t pcie_cap_get_version(const PCIDevice *dev); void pcie_cap_flags_set_vector(PCIDevice *dev, uint8_t vector); uint8_t pcie_cap_flags_get_vector(PCIDevice *dev); =20 --=20 2.39.2 From nobody Sun May 19 13:08:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1684533519; cv=none; d=zohomail.com; s=zohoarc; b=K09YBZCVXneB2FC464o0eliNJtnrqjgmkpLR9gwB0Bc8/OFscLSKSR6JXlb5KWvXmY8UIVgH0DTjUuZJEJo9XNnKfanQh9qHgifdrk7c1PzL3lql4wdZk2VikcT0QZdMzEJ0LDeghHNMGF/GOLUneM/MjEb0Uqb/HQZM+g3BUWw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1684533519; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=fgLJMJytCtabs3p65dU+5yU7aU1Lzei6dYgf+/EPmqw=; b=QtDrLaEAkQbgJDAX3bEodsw/0k362gh9Fw46pB8zY01sYFxeakVK8e2e6lDWRjnV6zrhdHBXLLC351nbnCPxho86Gg2ZWTmP19Evt23E1krY3Ok6cYe1SjUK174SiAINanGXnyZ26xDjKtvEe8Wic+zw2SYy30HINvpTMDHNIHA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 168453351914888.09754563962929; Fri, 19 May 2023 14:58:39 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q086o-0005zD-6p; Fri, 19 May 2023 17:57:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q086m-0005yK-GP for qemu-devel@nongnu.org; Fri, 19 May 2023 17:57:56 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q086l-0004FT-1u for qemu-devel@nongnu.org; Fri, 19 May 2023 17:57:56 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-205-yoCbIoAGMYi19gyO_PTBUQ-1; Fri, 19 May 2023 17:57:50 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3042A2A5954B; Fri, 19 May 2023 21:57:50 +0000 (UTC) Received: from omen.home.shazbot.org (unknown [10.22.10.168]) by smtp.corp.redhat.com (Postfix) with ESMTP id C854AC0004B; Fri, 19 May 2023 21:57:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684533474; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fgLJMJytCtabs3p65dU+5yU7aU1Lzei6dYgf+/EPmqw=; b=SKNHp9vrxIKmdZuNPM8S8tKrHRa1yakv/M9E5BM6oTZVzskPR271d13yDI9MrsRPi/St8l GGBi/Cz48gV14ch7TRIPTOPAnyaWQEChyJNydVnrRDIWbBVi4aUOHW/YCMnu5A2Q8gwfAm X9RdGoNtaCb0E2EWuIAnK01MCSueRfo= X-MC-Unique: yoCbIoAGMYi19gyO_PTBUQ-1 From: Alex Williamson To: robin@streamhpc.com, mst@redhat.com, marcel.apfelbaum@gmail.com Cc: Alex Williamson , qemu-devel@nongnu.org Subject: [RFC PATCH 4/4] vfio/pci: Enable AtomicOps completers on root ports Date: Fri, 19 May 2023 15:57:39 -0600 Message-Id: <20230519215739.402729-5-alex.williamson@redhat.com> In-Reply-To: <20230519215739.402729-1-alex.williamson@redhat.com> References: <20230519215739.402729-1-alex.williamson@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=alex.williamson@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1684533520858100001 Content-Type: text/plain; charset="utf-8" For cold-plug configurations where the vfio-pci device reports enabled support for PCIe AtomicOps completers and we have a compatible VM configuration, mirror the host AtomicOps completer support in the virtual root port. Signed-off-by: Alex Williamson --- hw/vfio/pci.c | 66 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 66 insertions(+) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index bf27a3990564..61d045785bf8 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -1826,6 +1826,70 @@ static void vfio_add_emulated_long(VFIOPCIDevice *vd= ev, int pos, vfio_set_long_bits(vdev->emulated_config_bits + pos, mask, mask); } =20 +/* + * For a cold-plugged device connected directly to a root port where + * vfio-pci reports AtomicOp completer support on the host, twiddle + * bits in the emulated root port to reflect this capability. + */ +static void vfio_pci_enable_rp_atomics(VFIOPCIDevice *vdev) +{ + struct vfio_device_info_cap_pci_atomic_comp *cap; + g_autofree struct vfio_device_info *info =3D NULL; + struct vfio_info_cap_header *hdr; + PCIDevice *parent; + uint32_t mask =3D 0; + PCIBus *bus; + + /* + * XXX + * - Should this also be restricted to TYPE_VFIO_PCI_NOHOTPLUG device= s? + * - Should we clear the capability on exit? + */ + if (vdev->pdev.qdev.hotplugged) { + return; + } + + info =3D vfio_get_device_info(vdev->vbasedev.fd); + if (!info) { + return; + } + + hdr =3D vfio_get_device_info_cap(info, VFIO_DEVICE_INFO_CAP_PCI_ATOMIC= _COMP); + if (!hdr) { + return; + } + + cap =3D (void *)hdr; + if (cap->flags & VFIO_PCI_ATOMIC_COMP32) { + mask |=3D PCI_EXP_DEVCAP2_ATOMIC_COMP32; + } + if (cap->flags & VFIO_PCI_ATOMIC_COMP64) { + mask |=3D PCI_EXP_DEVCAP2_ATOMIC_COMP64; + } + if (cap->flags & VFIO_PCI_ATOMIC_COMP128) { + mask |=3D PCI_EXP_DEVCAP2_ATOMIC_COMP128; + } + + if (!mask) { + return; + } + + bus =3D pci_get_bus(&vdev->pdev); + parent =3D bus->parent_dev; + + if (!parent || !pci_is_express(parent) || !parent->exp.exp_cap) { + return; + } + + if (pcie_cap_get_type(parent) !=3D PCI_EXP_TYPE_ROOT_PORT || + pcie_cap_get_version(parent) !=3D PCI_EXP_FLAGS_VER2) { + return; + } + + pci_long_test_and_set_mask(parent->config + parent->exp.exp_cap + + PCI_EXP_DEVCAP2, mask); +} + static int vfio_setup_pcie_cap(VFIOPCIDevice *vdev, int pos, uint8_t size, Error **errp) { @@ -1929,6 +1993,8 @@ static int vfio_setup_pcie_cap(VFIOPCIDevice *vdev, i= nt pos, uint8_t size, QEMU_PCI_EXP_LNKCAP_MLS(QEMU_PCI_EXP_LNK_2_5GT)= , ~0); vfio_add_emulated_word(vdev, pos + PCI_EXP_LNKCTL, 0, ~0); } + + vfio_pci_enable_rp_atomics(vdev); } =20 /* --=20 2.39.2