From nobody Thu Apr 9 17:04:51 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; arc=pass (i=1 dmarc=pass fromdomain=epam.com); dmarc=pass(p=quarantine dis=none) header.from=epam.com ARC-Seal: i=2; a=rsa-sha256; t=1773054544; cv=pass; d=zohomail.com; s=zohoarc; b=TZ5WHSHhoqRnl/o0N4qolyytqtWq+nkcf599TqAJKLOCEPevi2G6pvm/dfsgP2PDSRg5rVxKGB4V6e7oOkiiEVYLtD9ykDdUQEv7apRLFBTnTNbJ2GSUMRZl9kt+iwHc9O/VsTCHEeNxFP+jqybpZe0ZWcR/mEbmowCQDeIGCz0= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1773054544; h=Content-ID:Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=jPd7eg99vjV5lo4tppv9T3tRyoUFlgTQOLxJi+eBzBY=; b=Dr/b1ZziJscYYx+zxFjtF0VQfDRsyznrHBkuzobxe8sUSpWtx+mpGmnUtS/igKwR2xDF7BD+jdZbl0TwCmJH+4M0DRVESoMMYD/uhIs8oP+zr2JKHTklKZmqAVdCFb3gEGoKuKXYgSuRBij6KH1XUd8Xq4nbrt7F+CHK9QGGY84= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; arc=pass (i=1 dmarc=pass fromdomain=epam.com); dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1773054544019863.4570955459309; Mon, 9 Mar 2026 04:09:04 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1249278.1546779 (Exim 4.92) (envelope-from ) id 1vzYTZ-0004iM-SZ; Mon, 09 Mar 2026 11:08:41 +0000 Received: by outflank-mailman (output) from mailman id 1249278.1546779; Mon, 09 Mar 2026 11:08:41 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1vzYTZ-0004iD-NR; Mon, 09 Mar 2026 11:08:41 +0000 Received: by outflank-mailman (input) for mailman id 1249278; Mon, 09 Mar 2026 11:08:40 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1vzYTX-0003iR-Tl for xen-devel@lists.xenproject.org; Mon, 09 Mar 2026 11:08:40 +0000 Received: from MRWPR03CU001.outbound.protection.outlook.com (mail-francesouthazlp170110003.outbound.protection.outlook.com [2a01:111:f403:c207::3]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 52cadce9-1ba8-11f1-b164-2bf370ae4941; Mon, 09 Mar 2026 12:08:38 +0100 (CET) Received: from PAVPR03MB10102.eurprd03.prod.outlook.com (2603:10a6:102:30d::12) by AS2PR03MB9695.eurprd03.prod.outlook.com (2603:10a6:20b:60d::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9678.25; Mon, 9 Mar 2026 11:08:33 +0000 Received: from PAVPR03MB10102.eurprd03.prod.outlook.com ([fe80::b8c6:f37a:987a:beb]) by PAVPR03MB10102.eurprd03.prod.outlook.com ([fe80::b8c6:f37a:987a:beb%5]) with mapi id 15.20.9678.020; Mon, 9 Mar 2026 11:08:33 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 52cadce9-1ba8-11f1-b164-2bf370ae4941 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=drDPO9QznwPw0acjwL60yGPkwYzkpG+GVxUg97cIyKdC8Alk1uuH9lIjqjP3fCVRdLsYzf+YHNXhkSPGfvF8xzE7qc1So/gbYXSvYkWTrmLgUAl13UNrYYjfSZSqeMqy1nz08kZCruUqUFTOlt8QZWyde+NLu6OsPtmEbTnwj1TE1Tw2ZRQQgf31DqFFOKGHSlzF4ayo8/J1cyMdMkzFoZhpmdefd0B3IC0UzTHIMpEX1iNBxdxIWUtdz13jD/S6Lqg8fItYBahvzjoNZqTtLatmFZ+iagPnsreqBkK6aQzlatYz4zn/yUzaqFd74FfWRIwba/8y8nCvZAG9eo6GWQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=jPd7eg99vjV5lo4tppv9T3tRyoUFlgTQOLxJi+eBzBY=; b=EKYqFMaPdpQTh/zZ2YFaEOLR70yWeWBwNJnhgjAUgLktNpTc/94GJVYvhfqjp+Ps2z+OzciGn99Mu1qI17/uymfDKIvC6uyNWwQ9scyEZOyDsJhY++bLI1B2iJKHcB+4UJu19bVfgU9HOFNGHWPJnABi5XBSfv4K9kTy01jcDn63ilY28lDKxErLcxOk9UpA7suJCLgjnu0BsCIhuCjh2WXbZrCWQMRAFo6OZ5k4xDP5u3flf7wLtZL4WeUkhmPQ4+kazVj6wX7fjtULLQtcT2Pb84Y7zFXhp8+UwXlCpoaawhdDJpiiCjGhFtzdZC1U9Flpq9PGgm8X1Ckqk7egbA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jPd7eg99vjV5lo4tppv9T3tRyoUFlgTQOLxJi+eBzBY=; b=Tskh8JBL0HRr4v1du1eU5NDPUlYOzC6HBpi2nkCyI32VTBNkas3enBg/A2Xd83Bi5ZtEdloPJEAVWOEGu0z2O9PEVI1A4/suqvuHHUvB9FTYACkUIjkNlYrkTyiQgNs3MgibVQ6/yoyASTBEeAZsvF2O2GI69LamtSaTFya2dVbUsp/Y6Gf/JHkncMU44H6G+3BIBVI9o3RftDp/1yYansXhg30528W/8obQsbxXU9F7vtol4LBffdto9MQ6CEEhE5eNMemM9woLQm6jAxCpHqiukhTvo5sOqpa3Vw/rYsNdkubz033t1gQCNtwAfJiNv/ORplaMwy1uNFkM5Z+VZQ== From: Mykyta Poturai To: "xen-devel@lists.xenproject.org" CC: Stewart Hildebrand , =?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= , "Daniel P. Smith" , Mykyta Poturai Subject: [PATCH v2 6/8] vpci: add SR-IOV support for PVH Dom0 Thread-Topic: [PATCH v2 6/8] vpci: add SR-IOV support for PVH Dom0 Thread-Index: AQHcr7UOJDdt9SAQzUinWXRod1IIzA== Date: Mon, 9 Mar 2026 11:08:27 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=epam.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: PAVPR03MB10102:EE_|AS2PR03MB9695:EE_ x-ms-office365-filtering-correlation-id: a4840fa3-50ac-4db8-4211-08de7dcc33f4 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0;ARA:13230040|366016|376014|1800799024|38070700021; x-microsoft-antispam-message-info: XhqXnexDxH+jMPCZSI2iTGH4fAKsHP/TaJ5qdrYQEG9NxZTUt+ZizIX7OqlIGrmmQNhPcT2Q6I+zmQmDTVYB6IjrQM+hv0eUcAqAmhlRe4IFtROYQH65TBnIQSkGT7971kgTFzZKQ+haQyDjrMifzzTlkUSOY6xP0w99q6pE4gmAyka0fSkPVSNfefH2wZ2aASsJoU6R4C6mbYWl/C3B0TuCZzIXEvlIS00W/mlwgj25cY1bUOcsuBroiouS2mTHlU/jJ0DiDi9WBh+4ObCRGJJFL8HOZkjlb5nXKnZUy5k4BnODpZNHY5QaQKfpQHlR9cgPYTDZVFNUgBkc7AG5ER2aVoX+HCG/Ylr3IwhB6DBaobgg6cVIu8qm0V5vDn2ePp4GFCiD7XXDJE/PSZTT2dLwJigXELskmeZM+M48+RUxvX1YcNZtnSIJ3h7eu0M2t6Tp9tydNpHJtinaVnf9w1K3uHzFhM6jUb3R5JsoJ6yq8kjcWk6zhBioa2l/AdUlym0zP1PJCpyLKSxuWMAC+9LmDtGpxU8YWdEjCdKfAljNpPnt6g/UtU2mpDZNAWUd4Wp7FEgLdjq0Q4pBROhDsc36OCJ8hB3KZHCYaWEjN/0An8UGXpxtE9nxN2XMmFavMLSMtB78DBv4kKmcbQdcrlq8yIzhnQZSxfVYKGfviSPSLR9tsS9GfX1s30AbQsylKDp7fcyJuzxnmE69s9SKjahM4gs2U+jhTAGRo46evMA2hH9JFMIxNtaW+yj/hUmHdg2BXTOQ/jeVeq2DdQv5wAUdwu/GX+EKzuUp/ecDyAo= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAVPR03MB10102.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(376014)(1800799024)(38070700021);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?B?QnlBUU55aEJXU1hXVlpkdlFzamtvcGVYdFFtSk02S2RVNWRPTzgwbFlkSkZI?= =?utf-8?B?M25tMVZGZVVITEtzbGE5dnY3RDk5K3g1VmdpN0M0ZzQ2eEZLU0dKLzJ6aHhh?= =?utf-8?B?eHEzcUJUYzcrQXZQa3lQSk93bDd0VG9Xa2s2V1hDcDNxMzI0SkVMekhOVEE1?= =?utf-8?B?M2ZHWjRuR1NkcXRXSFhmeXRTZmFlTjU4cTlvM2tzMVhhMkZZQ1ZDQjNDNzRN?= =?utf-8?B?TXdmS0JiMGlDazdsaTZ4alYyQytyNnF4OEFNeXNyZXNlWllWQ1ZjTUlVUmhp?= =?utf-8?B?R1lIYU1UaWdlZHhMV29RVjhoMk8zQjF2dTdwMzVDYXY3Wkl3V3pTTHFHOXJT?= =?utf-8?B?RXRMYTJZcXo0YmxKNFVxdjBzSHBqVTVzalVEcXBhWkhBOXVxd2JHY2pwMU9o?= =?utf-8?B?Um1VV2dHR1pUOXNCUTJnc0syTU5IR25lUjVULzhDVXkyY1cxVVd4WGhsOEEw?= =?utf-8?B?NkVtYjJzOWhJTkt1Yi9CR0c0ZEdDYzVINXJaWFFodjdnVlh4RkNlNkNmNlpa?= =?utf-8?B?YnBxeUVGcGtmbngzcFNSU2tiTmE5R09qVVlWVXFYZWY1U0pnZEtrYmRBQ0xU?= =?utf-8?B?QWwrL09RVnRjTDNtNjJ2L0lxb0pTZ1hkOGk3ZXF4WHFFc3NhbWY0amp4Z0dM?= =?utf-8?B?bklFSjBWOXRLbnhwRFFCV0ZVdUNKYXl0K1FlbHJPUFNURnlVd3FqVmtNdUl6?= =?utf-8?B?U0lQQks0NmRFT1pQL3dtWFZOVVYyOE5HWkdGVVV6Qmg2K1NSS0wrazdGTnhT?= =?utf-8?B?eVdKbkorYWpqT3lUZUxPb3dkZkJlaGFIMklndWdhSWx2NFZJZU96MlVTQXN6?= =?utf-8?B?eVByVDZaUmxHd1RPM2ttV1JTdExRa2NhVDc2OUl6ZGwvQTdZdWdhMm1NaWk0?= =?utf-8?B?Qm9zbTQwa21JTWg1clZqMEZIK00wZThoNzRxNW1qNU9Zb1U1M1g0UFMrcUYy?= =?utf-8?B?aDBSZDU4aUVmS3VSUEJNcGZyK1V2TWVYTVV6SWQvL3lTTksvZksxaGExMHZ0?= =?utf-8?B?OEU5SGdNYWVXSmJyL1Z6ZzhpUS9Wb2pvUTc1SThhSW4zNmNDR09IZitwQTc4?= =?utf-8?B?N3FaSzNwNjZhWnpTVGk0cWtFWEt6bllBNzllY3grMyt3SDA2eG5SUlVrdFpv?= =?utf-8?B?ak4vQndaODdEOFZWaHplMUNGVWgwWlRLRWwyQTFabTJzWWczcFk2QjZ6S3ky?= =?utf-8?B?WHNXcys4UWRiMTd3VGdhNWxaUTNYM1p4VDZnU3Fmc2lFZEJhUmY2eWQrVGVX?= =?utf-8?B?N0Y4OThYTlZuWUdTdHhGNHNKVTl6NUNYaGZLYkdxeHNXYStBdjRyeEMrU2lT?= =?utf-8?B?T3pkV1BKcUFkV0tMWm1ZRXJJNndrVG10STZ2SzArS0E2K2UyUmNtTk5sSzNz?= =?utf-8?B?UVc3cUp4amZUQ09tOHB4V3F2Z0hhbFUxRndMM0FuT2RkSmZ0bmVnVExEY3RP?= =?utf-8?B?MG5RaVJqblFzTUtxRkE5TEl3RW9KdS84Ylp0YjdVZkt6T3A1S0pKbm1WazYz?= =?utf-8?B?WHo1MUJXRXZ5QkRVYnk0YThJL0c0emQ2VU5ZUGVQL1EvN0dKSW1mM1hLcmd1?= =?utf-8?B?SnM1UkhnOXpnSlhyOFQ2ZEs2cDJjbEI3SkV5R3BTV1JFZFBlMTZpK2JNdkFo?= =?utf-8?B?S2VHbjRJNEpNdTNVMndFQmtFT2Q0QmtPanl5d3BpZWxWRHpwc0x0OTkxT3FS?= =?utf-8?B?b2RWY2h3STVTdlpFZTM2TjhuNmZVS1grSEt4RkcwTmh6RjR1RXFPMWswcUJG?= =?utf-8?B?RU0rQWNISXBPNWJHRVd0R2x3K2MxT3BpbkIwSER4Mi90UE5XQkRGNkVSYzFO?= =?utf-8?B?d09wVCthbndiR3duLzI2U2NPRkNaWjJDNHJmY2tmL0VOVS93QzRKUWlNUHB5?= =?utf-8?B?K3JpUU5jemJvcTNTVUlkNjNScTQ5OHo5SmNUR2lZcGJDRXZnREw5cml0TEpt?= =?utf-8?B?VjBQcnBrNUZ6VkZMQW41aUdJRDhWYnFZMkxVZXJ5eHRPUnpTVXlNM0R2a2Ew?= =?utf-8?B?TlZMRGxROWZQSGJWaXZUdlJhS1NoVlBVdmRBblVUaExHZGkwektFbDdxTExu?= =?utf-8?B?S1I4MXUvc0lncmVYUXZ2WEFDUWRmRzZXdndXYUI4SUZSTlJBSlVQM05TSXQy?= =?utf-8?B?dkw0NHppREtHeXNPVTJFSDZmQ0x1RGFwRWY1K0FYbjlObGlmQTBOWEVXb2ly?= =?utf-8?B?SFcrZlJuTGRRZXpoUkNYR2Q0NUV5YWd0NncweHpBTHoyb2M2YWNPT0FVMDZJ?= =?utf-8?B?WkpscmY5WEs4K1FEd2s0ZUh5NGFUWjBSSUVLU2Z4NTZRR3pwMHArZlZFT0hI?= =?utf-8?B?NUVLbG5VSVNQbUdDeEJjM3ZjeGU4Q3h5MGE2eWFhRVQveHpBOVdJUXprVUI1?= =?utf-8?Q?MeeManXb2lAi6PCk=3D?= Content-Type: text/plain; charset="utf-8" Content-ID: <69B3D15764319041A131EA3108B1172A@eurprd03.prod.outlook.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: PAVPR03MB10102.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: a4840fa3-50ac-4db8-4211-08de7dcc33f4 X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Mar 2026 11:08:27.8941 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 9BASa2I4w+dmNwlYiFCzgAnqInA92UBy8aXw+UHqa35h6k0NMcuf62Ir0aP7NWyel7uBhdMAsqyHVQ7JcnFgJQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS2PR03MB9695 X-ZohoMail-DKIM: pass (identity @epam.com) X-ZM-MESSAGEID: 1773054545512154100 From: Stewart Hildebrand This code is expected to only be used by privileged domains, unprivileged domains should not get access to the SR-IOV capability. Implement RW handlers for PCI_SRIOV_CTRL register to dynamically map/unmap VF BARS. Recalculate BAR sizes before mapping VFs to account for possible changes in the system page size register. Allow forcing vpci_modify_bars to not defer the actual mapping changes, which is needed to fix the sequential calls to vpci_modify_bars when enabling VFs from Dom0. Signed-off-by: Roger Pau Monn=C3=A9 Signed-off-by: Stewart Hildebrand Signed-off-by: Mykyta Poturai --- v1->v2: * switch to VF discovery by Xen * fix sequential vpci_modify_bars calls * move documentation changes to a separate commit --- xen/drivers/vpci/Makefile | 2 +- xen/drivers/vpci/header.c | 17 +- xen/drivers/vpci/sriov.c | 363 ++++++++++++++++++++++++++++++++++++++ xen/include/xen/vpci.h | 12 +- 4 files changed, 385 insertions(+), 9 deletions(-) create mode 100644 xen/drivers/vpci/sriov.c diff --git a/xen/drivers/vpci/Makefile b/xen/drivers/vpci/Makefile index a7c8a30a89..fe1e57b64d 100644 --- a/xen/drivers/vpci/Makefile +++ b/xen/drivers/vpci/Makefile @@ -1,2 +1,2 @@ -obj-y +=3D vpci.o header.o rebar.o +obj-y +=3D vpci.o header.o rebar.o sriov.o obj-$(CONFIG_HAS_PCI_MSI) +=3D msi.o msix.o diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c index 284964f0d4..c55c3380d4 100644 --- a/xen/drivers/vpci/header.c +++ b/xen/drivers/vpci/header.c @@ -264,7 +264,7 @@ bool vpci_process_pending(struct vcpu *v) return false; } =20 -static int __init apply_map(struct domain *d, const struct pci_dev *pdev, +static int apply_map(struct domain *d, const struct pci_dev *pdev, uint16_t cmd) { struct vpci_header *header =3D &pdev->vpci->header; @@ -323,7 +323,8 @@ static void defer_map(const struct pci_dev *pdev, uint1= 6_t cmd, bool rom_only) raise_softirq(SCHEDULE_SOFTIRQ); } =20 -int vpci_modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_on= ly) +int vpci_modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_on= ly, + bool no_defer) { struct vpci_header *header =3D &pdev->vpci->header; struct pci_dev *tmp; @@ -519,7 +520,7 @@ int vpci_modify_bars(const struct pci_dev *pdev, uint16= _t cmd, bool rom_only) d =3D dom_xen; } =20 - if ( system_state < SYS_STATE_active ) + if ( system_state < SYS_STATE_active || no_defer ) { /* * Mappings might be created when building Dom0 if the memory deco= ding @@ -566,7 +567,7 @@ static void cf_check cmd_write( * memory decoding bit has not been changed, so leave everything a= s-is, * hoping the guest will realize and try again. */ - vpci_modify_bars(pdev, cmd, false); + vpci_modify_bars(pdev, cmd, false, false); else pci_conf_write16(pdev->sbdf, reg, cmd); } @@ -736,7 +737,7 @@ static void cf_check rom_write( */ else if ( vpci_modify_bars(pdev, new_enabled ? PCI_COMMAND_MEMORY : 0, - true) ) + true, false) ) /* * No memory has been added or removed from the p2m (because the a= ctual * p2m changes are deferred in defer_map) and the ROM enable bit h= as @@ -954,6 +955,9 @@ int vpci_init_header(struct pci_dev *pdev) =20 header->guest_cmd =3D cmd; =20 + if ( pdev->info.is_virtfn ) + return vf_init_header(pdev); + /* Disable memory decoding before sizing. */ if ( !is_hwdom || (cmd & PCI_COMMAND_MEMORY) ) pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd & ~PCI_COMMAND_MEMOR= Y); @@ -1062,7 +1066,8 @@ int vpci_init_header(struct pci_dev *pdev) goto fail; } =20 - return (cmd & PCI_COMMAND_MEMORY) ? vpci_modify_bars(pdev, cmd, false)= : 0; + return (cmd & PCI_COMMAND_MEMORY) + ? vpci_modify_bars(pdev, cmd, false, false) : 0; =20 fail: pci_conf_write16(pdev->sbdf, PCI_COMMAND, cmd); diff --git a/xen/drivers/vpci/sriov.c b/xen/drivers/vpci/sriov.c new file mode 100644 index 0000000000..6f691149e9 --- /dev/null +++ b/xen/drivers/vpci/sriov.c @@ -0,0 +1,363 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Handlers for accesses to the SR-IOV capability structure. + * + * Copyright (C) 2026 Citrix Systems R&D + */ + +#include +#include +#include + +static int vf_init_bars(const struct pci_dev *vf_pdev) +{ + int vf_idx; + unsigned int i; + const struct pci_dev *pf_pdev =3D vf_pdev->pf_pdev; + struct vpci_bar *bars =3D vf_pdev->vpci->header.bars; + struct vpci_bar *physfn_vf_bars =3D pf_pdev->vpci->sriov->vf_bars; + unsigned int sriov_pos =3D pci_find_ext_capability(pf_pdev, + PCI_EXT_CAP_ID_SRIOV); + uint16_t offset =3D pci_conf_read16(pf_pdev->sbdf, + sriov_pos + PCI_SRIOV_VF_OFFSET); + uint16_t stride =3D pci_conf_read16(pf_pdev->sbdf, + sriov_pos + PCI_SRIOV_VF_STRIDE); + + vf_idx =3D vf_pdev->sbdf.sbdf - (pf_pdev->sbdf.sbdf + offset); + if ( vf_idx < 0 ) + return -EINVAL; + + if ( stride ) + { + if ( vf_idx % stride ) + return -EINVAL; + vf_idx /=3D stride; + } + + /* + * Set up BARs for this VF out of PF's VF BARs taking into account + * the index of the VF. + */ + for ( i =3D 0; i < PCI_SRIOV_NUM_BARS; i++ ) + { + bars[i].addr =3D physfn_vf_bars[i].addr + vf_idx * physfn_vf_bars[= i].size; + bars[i].guest_addr =3D bars[i].addr; + bars[i].size =3D physfn_vf_bars[i].size; + bars[i].type =3D physfn_vf_bars[i].type; + bars[i].prefetchable =3D physfn_vf_bars[i].prefetchable; + } + + return 0; +} + +/* Must be called form vpci_process_pending context */ +static int map_vfs(const struct pci_dev *pf_pdev, uint16_t cmd) +{ + struct pci_dev *vf_pdev; + int rc; + + ASSERT(rw_is_write_locked(&pf_pdev->domain->pci_lock)); + + list_for_each_entry(vf_pdev, &pf_pdev->vf_list, vf_list) { + rc =3D vpci_modify_bars(vf_pdev, cmd, false, true); + if ( rc ) + { + gprintk(XENLOG_ERR, "failed to %s VF %pp: %d\n", + (cmd & PCI_COMMAND_MEMORY) ? "map" : "unmap", + &vf_pdev->sbdf, rc); + return rc; + } + } + + return 0; +} + + +static int size_vf_bars(struct pci_dev *pf_pdev, unsigned int sriov_pos) +{ + /* + * NB: a non-const pci_dev of the PF is needed in order to update + * vf_rlen. + */ + struct vpci_bar *bars; + unsigned int i; + int rc =3D 0; + + ASSERT(rw_is_write_locked(&pf_pdev->domain->pci_lock)); + ASSERT(!pf_pdev->info.is_virtfn); + ASSERT(pf_pdev->vpci->sriov); + + /* Read BARs for VFs out of PF's SR-IOV extended capability. */ + bars =3D pf_pdev->vpci->sriov->vf_bars; + /* Set the BARs addresses and size. */ + for ( i =3D 0; i < PCI_SRIOV_NUM_BARS; i +=3D rc ) + { + unsigned int idx =3D sriov_pos + PCI_SRIOV_BAR + i * 4; + uint32_t bar; + uint64_t addr, size; + + bar =3D pci_conf_read32(pf_pdev->sbdf, idx); + + rc =3D pci_size_mem_bar(pf_pdev->sbdf, idx, &addr, &size, + PCI_BAR_VF | + ((i =3D=3D PCI_SRIOV_NUM_BARS - 1) ? PCI_BAR= _LAST + : 0)); + + /* + * Update vf_rlen on the PF. According to the spec the size of + * the BARs can change if the system page size register is + * modified, so always update rlen when enabling VFs. + */ + pf_pdev->physfn.vf_rlen[i] =3D size; + + if ( !size ) + { + bars[i].type =3D VPCI_BAR_EMPTY; + continue; + } + + bars[i].addr =3D addr; + bars[i].guest_addr =3D addr; + bars[i].size =3D size; + bars[i].prefetchable =3D bar & PCI_BASE_ADDRESS_MEM_PREFETCH; + + switch ( rc ) + { + case 1: + bars[i].type =3D VPCI_BAR_MEM32; + break; + + case 2: + bars[i].type =3D VPCI_BAR_MEM64_LO; + bars[i + 1].type =3D VPCI_BAR_MEM64_HI; + break; + + default: + ASSERT_UNREACHABLE(); + } + } + + rc =3D rc > 0 ? 0 : rc; + + return rc; +} + +struct callback_data { + const struct pci_dev *pdev; + unsigned int pos; + uint32_t value; + bool enable : 1; + bool disable : 1; + bool map : 1; + bool unmap : 1; +}; + +static void cf_check control_write_cb(void *data) +{ + struct callback_data *cb =3D data; + const struct pci_dev *pdev =3D cb->pdev; + uint16_t offset =3D pci_conf_read16(pdev->sbdf, cb->pos + PCI_SRIOV_VF= _OFFSET); + uint16_t stride =3D pci_conf_read16(pdev->sbdf, cb->pos + PCI_SRIOV_VF= _STRIDE); + struct vpci_sriov *sriov =3D pdev->vpci->sriov; + int rc =3D 0; + unsigned int i; + + if ( cb->unmap ) + { + write_lock(&pdev->domain->pci_lock); + map_vfs(pdev, 0); + write_unlock(&pdev->domain->pci_lock); + } + + if ( cb->enable || cb->disable ) + { + for ( i =3D 0; i < sriov->num_vfs; i++ ) + { + const pci_sbdf_t vf_sbdf =3D { + .sbdf =3D pdev->sbdf.sbdf + offset + stride * i, + }; + + if ( cb->enable ) + { + const struct pci_dev_info info =3D { + .is_virtfn =3D true, + .is_extfn =3D false, + .physfn.bus =3D pdev->sbdf.bus, + .physfn.devfn =3D pdev->sbdf.devfn, + }; + rc =3D pci_add_device(vf_sbdf.seg, vf_sbdf.bus, vf_sbdf.de= vfn, + &info, pdev->node); + } + if ( cb->disable ) + rc =3D pci_remove_device(vf_sbdf.seg, vf_sbdf.bus, vf_sbdf= .devfn); + + if ( rc && rc !=3D -ENODEV) + gprintk(XENLOG_ERR, "failed to %s VF %pp: %d\n", + cb->enable ? "add" : "remove", &vf_sbdf, rc); + } + } + + if ( cb->map ) + { + write_lock(&pdev->domain->pci_lock); + rc =3D map_vfs(pdev, PCI_COMMAND_MEMORY); + + if ( rc ) + map_vfs(pdev, 0); + write_unlock(&pdev->domain->pci_lock); + } + + pci_conf_write16(pdev->sbdf, cb->pos + PCI_SRIOV_CTRL, cb->value); + xfree(cb); +} + +static void cf_check control_write(const struct pci_dev *pdev, unsigned in= t reg, + uint32_t val, void *data) +{ + unsigned int sriov_pos =3D reg - PCI_SRIOV_CTRL; + struct vpci_sriov *sriov =3D pdev->vpci->sriov; + struct callback_data *cb =3D NULL; + uint16_t control =3D pci_conf_read16(pdev->sbdf, reg); + bool mem_enabled =3D control & PCI_SRIOV_CTRL_MSE; + bool new_mem_enabled =3D val & PCI_SRIOV_CTRL_MSE; + bool enabled =3D control & PCI_SRIOV_CTRL_VFE; + bool new_enabled =3D val & PCI_SRIOV_CTRL_VFE; + + ASSERT(!pdev->info.is_virtfn); + + if ( new_enabled =3D=3D enabled && new_mem_enabled =3D=3D mem_enabled ) + { + pci_conf_write16(pdev->sbdf, reg, val); + return; + } + + cb =3D xzalloc(struct callback_data); + + if ( !cb ) + { + gprintk(XENLOG_ERR, + "%pp: Unable to allocate memory for SR-IOV enable\n", + pdev); + return; + } + + cb->pdev =3D pdev; + cb->pos =3D sriov_pos; + cb->value =3D val; + cb->map =3D new_mem_enabled && !mem_enabled; + cb->unmap =3D !new_mem_enabled && mem_enabled; + cb->enable =3D new_enabled && !enabled; + cb->disable =3D !new_enabled && enabled; + + current->vpci.task =3D WAIT; + current->vpci.wait.callback =3D control_write_cb; + current->vpci.wait.data =3D cb; + current->vpci.wait.end =3D NOW(); + + if ( cb->enable ) + { + size_vf_bars((struct pci_dev *)pdev, sriov_pos); + + /* + * Only update the number of active VFs when enabling, when + * disabling use the cached value in order to always remove the sa= me + * number of VFs that were active. + */ + sriov->num_vfs =3D pci_conf_read16(pdev->sbdf, + sriov_pos + PCI_SRIOV_NUM_VF); + /* + * NB: VFE needs to be enabled before calling pci_add_device so Xen + * can access the config space of VFs. FIXME casting away const-ne= ss + * to modify vf_rlen + */ + pci_conf_write16(pdev->sbdf, reg, control | PCI_SRIOV_CTRL_VFE); + /* + * The spec states that the software must wait at least 100ms befo= re + * attempting to access VF registers when enabling virtual functio= ns + * on the PF. + */ + + current->vpci.wait.end =3D NOW() + MILLISECS(100); + } +} + +int vf_init_header(struct pci_dev *vf_pdev) +{ + const struct pci_dev *pf_pdev; + unsigned int sriov_pos; + int rc =3D 0; + uint16_t ctrl; + + ASSERT(rw_is_write_locked(&vf_pdev->domain->pci_lock)); + + if ( !vf_pdev->info.is_virtfn ) + return 0; + + pf_pdev =3D vf_pdev->pf_pdev; + ASSERT(pf_pdev); + + rc =3D vf_init_bars(vf_pdev); + if ( rc ) + return rc; + + sriov_pos =3D pci_find_ext_capability(pf_pdev, PCI_EXT_CAP_ID_SRIOV); + ctrl =3D pci_conf_read16(pf_pdev->sbdf, sriov_pos + PCI_SRIOV_CTRL); + + if ( (pf_pdev->domain =3D=3D vf_pdev->domain) && (ctrl & PCI_SRIOV_CTR= L_MSE) ) + { + rc =3D vpci_modify_bars(vf_pdev, PCI_COMMAND_MEMORY, false, false); + if ( rc ) + return rc; + } + + return rc; +} + +static int cf_check init_sriov(struct pci_dev *pdev) +{ + unsigned int pos; + + ASSERT(!pdev->info.is_virtfn); + + pos =3D pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_SRIOV); + + if ( !pos ) + return 0; + + if ( xsm_resource_setup_pci(XSM_PRIV, pdev->sbdf.bdf) ) + { + printk(XENLOG_ERR + "%pp: SR-IOV configuration unsupported for unpriv %pd\n", + &pdev->sbdf, pdev->domain); + return 0; + } + + pdev->vpci->sriov =3D xzalloc(struct vpci_sriov); + if ( !pdev->vpci->sriov ) + return -ENOMEM; + + return vpci_add_register(pdev->vpci, vpci_hw_read16, control_write, + pos + PCI_SRIOV_CTRL, 2, NULL); +} + +static int cf_check cleanup_sriov(const struct pci_dev *pdev, bool hide) +{ + if ( hide ) + return 0; + + XFREE(pdev->vpci->sriov); + + return 0; +} + +REGISTER_VPCI_EXTCAP(SRIOV, init_sriov, cleanup_sriov); + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * tab-width: 4 + * indent-tabs-mode: nil + * End: + */ diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h index 47cdb54d42..ae5f3b7274 100644 --- a/xen/include/xen/vpci.h +++ b/xen/include/xen/vpci.h @@ -45,6 +45,7 @@ typedef struct { REGISTER_VPCI_CAPABILITY(PCI_EXT_CAP_ID_##name, name, finit, fclean, t= rue) =20 int __must_check vpci_init_header(struct pci_dev *pdev); +int __must_check vf_init_header(struct pci_dev *pdev); =20 /* Assign vPCI to device by adding handlers. */ int __must_check vpci_assign_device(struct pci_dev *pdev); @@ -146,7 +147,6 @@ struct vpci { * upon to know whether BARs are mapped into the guest p2m. */ bool bars_mapped : 1; - /* FIXME: currently there's no support for SR-IOV. */ } header; =20 /* MSI data. */ @@ -200,6 +200,13 @@ struct vpci { struct vpci_arch_msix_entry arch; } entries[]; } *msix; + + struct vpci_sriov { + /* PF only */ + struct vpci_bar vf_bars[PCI_SRIOV_NUM_BARS]; + uint16_t num_vfs; + } *sriov; + #ifdef CONFIG_HAS_VPCI_GUEST_SUPPORT /* Guest SBDF of the device. */ #define INVALID_GUEST_SBDF ((pci_sbdf_t){ .sbdf =3D ~0U }) @@ -323,7 +330,8 @@ bool vpci_ecam_read(pci_sbdf_t sbdf, unsigned int reg, = unsigned int len, unsigned long *data); =20 /* Map/unmap the BARs of a vPCI device. */ -int vpci_modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_on= ly); +int vpci_modify_bars(const struct pci_dev *pdev, uint16_t cmd, bool rom_on= ly, + bool no_defer); =20 #endif /* __XEN__ */ =20 --=20 2.51.2