From nobody Mon Apr 13 11:53:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2B12FC4332F for ; Fri, 11 Nov 2022 14:27:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234505AbiKKO1i (ORCPT ); Fri, 11 Nov 2022 09:27:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40882 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231261AbiKKO1a (ORCPT ); Fri, 11 Nov 2022 09:27:30 -0500 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 257E11AD; Fri, 11 Nov 2022 06:27:29 -0800 (PST) Received: from kwepemi100025.china.huawei.com (unknown [172.30.72.54]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4N81Hg1GfHzmVpr; Fri, 11 Nov 2022 22:27:11 +0800 (CST) Received: from DESKTOP-27KDQMV.china.huawei.com (10.174.148.223) by kwepemi100025.china.huawei.com (7.221.188.158) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Fri, 11 Nov 2022 22:27:26 +0800 From: "Longpeng(Mike)" To: CC: , , , , , , , , Longpeng Subject: [RFC 1/4] pci/sriov: extract sriov_numvfs common helper Date: Fri, 11 Nov 2022 22:27:19 +0800 Message-ID: <20221111142722.1172-2-longpeng2@huawei.com> X-Mailer: git-send-email 2.25.0.windows.1 In-Reply-To: <20221111142722.1172-1-longpeng2@huawei.com> References: <20221111142722.1172-1-longpeng2@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.174.148.223] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To kwepemi100025.china.huawei.com (7.221.188.158) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Longpeng Extract a common helper to set sriov nums. Signed-off-by: Longpeng --- drivers/pci/iov.c | 38 +++++++++++++++++++++++--------------- 1 file changed, 23 insertions(+), 15 deletions(-) diff --git a/drivers/pci/iov.c b/drivers/pci/iov.c index 952217572113..3b441cd92960 100644 --- a/drivers/pci/iov.c +++ b/drivers/pci/iov.c @@ -406,23 +406,9 @@ static ssize_t sriov_numvfs_show(struct device *dev, return sysfs_emit(buf, "%u\n", num_vfs); } =20 -/* - * num_vfs > 0; number of VFs to enable - * num_vfs =3D 0; disable all VFs - * - * Note: SRIOV spec does not allow partial VF - * disable, so it's all or none. - */ -static ssize_t sriov_numvfs_store(struct device *dev, - struct device_attribute *attr, - const char *buf, size_t count) +static int sriov_numvfs_common(struct pci_dev *pdev, u16 num_vfs) { - struct pci_dev *pdev =3D to_pci_dev(dev); int ret =3D 0; - u16 num_vfs; - - if (kstrtou16(buf, 0, &num_vfs) < 0) - return -EINVAL; =20 if (num_vfs > pci_sriov_get_totalvfs(pdev)) return -ERANGE; @@ -471,6 +457,28 @@ static ssize_t sriov_numvfs_store(struct device *dev, exit: device_unlock(&pdev->dev); =20 + return ret; +} + +/* + * num_vfs > 0; number of VFs to enable + * num_vfs =3D 0; disable all VFs + * + * Note: SRIOV spec does not allow partial VF + * disable, so it's all or none. + */ +static ssize_t sriov_numvfs_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + struct pci_dev *pdev =3D to_pci_dev(dev); + int ret; + u16 num_vfs; + + if (kstrtou16(buf, 0, &num_vfs) < 0) + return -EINVAL; + + ret =3D sriov_numvfs_common(pdev, num_vfs); if (ret < 0) return ret; =20 --=20 2.23.0 From nobody Mon Apr 13 11:53:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9443C433FE for ; Fri, 11 Nov 2022 14:27:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234539AbiKKO1q (ORCPT ); Fri, 11 Nov 2022 09:27:46 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40884 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231911AbiKKO1b (ORCPT ); Fri, 11 Nov 2022 09:27:31 -0500 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8D96B1BA; Fri, 11 Nov 2022 06:27:29 -0800 (PST) Received: from kwepemi100025.china.huawei.com (unknown [172.30.72.53]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4N81Cj0yfKzbng6; Fri, 11 Nov 2022 22:23:45 +0800 (CST) Received: from DESKTOP-27KDQMV.china.huawei.com (10.174.148.223) by kwepemi100025.china.huawei.com (7.221.188.158) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Fri, 11 Nov 2022 22:27:27 +0800 From: "Longpeng(Mike)" To: CC: , , , , , , , , Longpeng Subject: [RFC 2/4] pci/sriov: add vf_bitmap to mark the vf id allocation Date: Fri, 11 Nov 2022 22:27:20 +0800 Message-ID: <20221111142722.1172-3-longpeng2@huawei.com> X-Mailer: git-send-email 2.25.0.windows.1 In-Reply-To: <20221111142722.1172-1-longpeng2@huawei.com> References: <20221111142722.1172-1-longpeng2@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.174.148.223] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To kwepemi100025.china.huawei.com (7.221.188.158) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Longpeng Add a bitmap to mark the vf id allocation, the can be use to avoid the repeated allocations. Signed-off-by: Longpeng --- drivers/pci/iov.c | 13 +++++++++++++ drivers/pci/pci.h | 1 + 2 files changed, 14 insertions(+) diff --git a/drivers/pci/iov.c b/drivers/pci/iov.c index 3b441cd92960..886489aae0de 100644 --- a/drivers/pci/iov.c +++ b/drivers/pci/iov.c @@ -341,6 +341,8 @@ int pci_iov_add_virtfn(struct pci_dev *dev, int id) =20 pci_bus_add_device(virtfn); =20 + __set_bit(id, iov->vf_bitmap); + return 0; =20 failed1: @@ -357,6 +359,7 @@ void pci_iov_remove_virtfn(struct pci_dev *dev, int id) { char buf[VIRTFN_ID_LEN]; struct pci_dev *virtfn; + struct pci_sriov *iov; =20 virtfn =3D pci_get_domain_bus_and_slot(pci_domain_nr(dev->bus), pci_iov_virtfn_bus(dev, id), @@ -364,6 +367,9 @@ void pci_iov_remove_virtfn(struct pci_dev *dev, int id) if (!virtfn) return; =20 + iov =3D pci_physfn(dev)->sriov; + __clear_bit(id, iov->vf_bitmap); + sprintf(buf, "virtfn%u", id); sysfs_remove_link(&dev->dev.kobj, buf); /* @@ -791,6 +797,11 @@ static int sriov_init(struct pci_dev *dev, int pos) if (!iov) return -ENOMEM; =20 + rc =3D -ENOMEM; + iov->vf_bitmap =3D bitmap_zalloc(total, GFP_KERNEL); + if (!iov->vf_bitmap) + goto free_iov; + nres =3D 0; for (i =3D 0; i < PCI_SRIOV_NUM_BARS; i++) { res =3D &dev->resource[i + PCI_IOV_RESOURCES]; @@ -853,6 +864,7 @@ static int sriov_init(struct pci_dev *dev, int pos) res->flags =3D 0; } =20 +free_iov: kfree(iov); return rc; } @@ -864,6 +876,7 @@ static void sriov_release(struct pci_dev *dev) if (dev !=3D dev->sriov->dev) pci_dev_put(dev->sriov->dev); =20 + bitmap_free(dev->sriov->vf_bitmap); kfree(dev->sriov); dev->sriov =3D NULL; } diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index b1ebb7ab8805..a76d4ff83886 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -301,6 +301,7 @@ struct pci_sriov { u16 subsystem_device; /* VF subsystem device */ resource_size_t barsz[PCI_SRIOV_NUM_BARS]; /* VF BAR size */ bool drivers_autoprobe; /* Auto probing of VFs by driver */ + unsigned long *vf_bitmap; /* Allocated VFs */ }; =20 /** --=20 2.23.0 From nobody Mon Apr 13 11:53:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B18CAC433FE for ; Fri, 11 Nov 2022 14:28:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234583AbiKKO2B (ORCPT ); Fri, 11 Nov 2022 09:28:01 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40896 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232851AbiKKO1c (ORCPT ); Fri, 11 Nov 2022 09:27:32 -0500 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 79A47C12; Fri, 11 Nov 2022 06:27:31 -0800 (PST) Received: from kwepemi100025.china.huawei.com (unknown [172.30.72.55]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4N81HV1c9rzHvp9; Fri, 11 Nov 2022 22:27:02 +0800 (CST) Received: from DESKTOP-27KDQMV.china.huawei.com (10.174.148.223) by kwepemi100025.china.huawei.com (7.221.188.158) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Fri, 11 Nov 2022 22:27:27 +0800 From: "Longpeng(Mike)" To: CC: , , , , , , , , Longpeng Subject: [RFC 3/4] pci/sriov: add sriov_numfs_no_scan interface Date: Fri, 11 Nov 2022 22:27:21 +0800 Message-ID: <20221111142722.1172-4-longpeng2@huawei.com> X-Mailer: git-send-email 2.25.0.windows.1 In-Reply-To: <20221111142722.1172-1-longpeng2@huawei.com> References: <20221111142722.1172-1-longpeng2@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.174.148.223] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To kwepemi100025.china.huawei.com (7.221.188.158) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Longpeng Add sriov_numfs_no_scan interface, we can enable the sriov without adding all VFs immediately. For example: echo 16 > /sys/bus/pci/devices/0000\:65\:00.0/sriov_numvfs_no_scan ( Enable 16 VFs, but not add it to the system. ) Signed-off-by: Longpeng --- drivers/pci/iov.c | 64 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) diff --git a/drivers/pci/iov.c b/drivers/pci/iov.c index 886489aae0de..cb24d4b60e0d 100644 --- a/drivers/pci/iov.c +++ b/drivers/pci/iov.c @@ -491,6 +491,68 @@ static ssize_t sriov_numvfs_store(struct device *dev, return count; } =20 +static ssize_t sriov_numvfs_no_scan_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + struct pci_dev *pdev =3D to_pci_dev(dev); + int ret; + u16 num_vfs; + + if (kstrtou16(buf, 0, &num_vfs) < 0) + return -EINVAL; + + device_lock(&pdev->dev); + /* Clear the flag when disabling VFs */ + pdev->no_vf_scan =3D num_vfs ? 1 : 0; + device_unlock(&pdev->dev); + + ret =3D sriov_numvfs_common(pdev, num_vfs); + if (ret < 0) + goto fail; + + if (!num_vfs) + return count; + + /* + * Special case: set the same num_vfs again. + */ + if (test_bit(0, pdev->sriov->vf_bitmap)) + return count; + + /* + * Default to enabling VF0, we need to read some config registers that ar= e the + * same across all associated VFs. See pci_read_vf_config_common(). + */ + ret =3D pci_iov_add_virtfn(pdev, 0); + if (ret < 0) { + int rc; + + rc =3D sriov_numvfs_common(pdev, 0); + if (rc < 0) + pci_warn(pdev, "Roll back to SR-IOV disabled state failed, %d\n", rc); + + goto fail; + } + + return count; + +fail: + device_lock(&pdev->dev); + if (pdev->no_vf_scan) + pdev->no_vf_scan =3D 0; + device_unlock(&pdev->dev); + + return ret; +} + +static ssize_t sriov_numvfs_no_scan_show(struct device *dev, + struct device_attribute *attr, + char *buf) +{ + return sriov_numvfs_show(dev, attr, buf); +} + static ssize_t sriov_offset_show(struct device *dev, struct device_attribute *attr, char *buf) @@ -544,6 +606,7 @@ static ssize_t sriov_drivers_autoprobe_store(struct dev= ice *dev, =20 static DEVICE_ATTR_RO(sriov_totalvfs); static DEVICE_ATTR_RW(sriov_numvfs); +static DEVICE_ATTR_RW(sriov_numvfs_no_scan); static DEVICE_ATTR_RO(sriov_offset); static DEVICE_ATTR_RO(sriov_stride); static DEVICE_ATTR_RO(sriov_vf_device); @@ -552,6 +615,7 @@ static DEVICE_ATTR_RW(sriov_drivers_autoprobe); static struct attribute *sriov_pf_dev_attrs[] =3D { &dev_attr_sriov_totalvfs.attr, &dev_attr_sriov_numvfs.attr, + &dev_attr_sriov_numvfs_no_scan.attr, &dev_attr_sriov_offset.attr, &dev_attr_sriov_stride.attr, &dev_attr_sriov_vf_device.attr, --=20 2.23.0 From nobody Mon Apr 13 11:53:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9F71C433FE for ; Fri, 11 Nov 2022 14:27:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234566AbiKKO1u (ORCPT ); Fri, 11 Nov 2022 09:27:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40894 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232372AbiKKO1b (ORCPT ); Fri, 11 Nov 2022 09:27:31 -0500 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 418F1BB4; Fri, 11 Nov 2022 06:27:31 -0800 (PST) Received: from kwepemi100025.china.huawei.com (unknown [172.30.72.55]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4N81Ck5sMrzbngJ; Fri, 11 Nov 2022 22:23:46 +0800 (CST) Received: from DESKTOP-27KDQMV.china.huawei.com (10.174.148.223) by kwepemi100025.china.huawei.com (7.221.188.158) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Fri, 11 Nov 2022 22:27:28 +0800 From: "Longpeng(Mike)" To: CC: , , , , , , , , Longpeng Subject: [RFC 4/4] pci/sriov: add sriov_scan_vf_id interface Date: Fri, 11 Nov 2022 22:27:22 +0800 Message-ID: <20221111142722.1172-5-longpeng2@huawei.com> X-Mailer: git-send-email 2.25.0.windows.1 In-Reply-To: <20221111142722.1172-1-longpeng2@huawei.com> References: <20221111142722.1172-1-longpeng2@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.174.148.223] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To kwepemi100025.china.huawei.com (7.221.188.158) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Longpeng The users can add a specific VF through the sriov_scan_vf_id interface. For example: echo 4 > /sys/bus/pci/devices/0000\:65\:00.0/sriov_scan_vf_id ( Add VF4 into the system. ) Signed-off-by: Longpeng --- drivers/pci/iov.c | 47 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+) diff --git a/drivers/pci/iov.c b/drivers/pci/iov.c index cb24d4b60e0d..2b585b3a9ad8 100644 --- a/drivers/pci/iov.c +++ b/drivers/pci/iov.c @@ -553,6 +553,51 @@ static ssize_t sriov_numvfs_no_scan_show(struct device= *dev, return sriov_numvfs_show(dev, attr, buf); } =20 +static ssize_t sriov_scan_vf_id_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + struct pci_dev *pdev =3D to_pci_dev(dev); + u16 vf_id; + ssize_t ret; + + if (kstrtou16(buf, 0, &vf_id) < 0) + return -EINVAL; + + device_lock(&pdev->dev); + + if (!pdev->sriov->num_VFs) { + ret =3D -ENOENT; + goto exit; + } + + if (vf_id >=3D pdev->sriov->num_VFs) { + ret =3D -ENODEV; + goto exit; + } + + if (test_bit(vf_id, pdev->sriov->vf_bitmap)) { + ret =3D -EEXIST; + goto exit; + } + + device_unlock(&pdev->dev); + + /* Already scanned VF0 when enabling VFs */ + if (vf_id =3D=3D 0) + return count; + + ret =3D pci_iov_add_virtfn(pdev, vf_id); + if (ret < 0) + return ret; + + return count; + +exit: + device_unlock(&pdev->dev); + return ret; +} + static ssize_t sriov_offset_show(struct device *dev, struct device_attribute *attr, char *buf) @@ -607,6 +652,7 @@ static ssize_t sriov_drivers_autoprobe_store(struct dev= ice *dev, static DEVICE_ATTR_RO(sriov_totalvfs); static DEVICE_ATTR_RW(sriov_numvfs); static DEVICE_ATTR_RW(sriov_numvfs_no_scan); +static DEVICE_ATTR_WO(sriov_scan_vf_id); static DEVICE_ATTR_RO(sriov_offset); static DEVICE_ATTR_RO(sriov_stride); static DEVICE_ATTR_RO(sriov_vf_device); @@ -616,6 +662,7 @@ static struct attribute *sriov_pf_dev_attrs[] =3D { &dev_attr_sriov_totalvfs.attr, &dev_attr_sriov_numvfs.attr, &dev_attr_sriov_numvfs_no_scan.attr, + &dev_attr_sriov_scan_vf_id.attr, &dev_attr_sriov_offset.attr, &dev_attr_sriov_stride.attr, &dev_attr_sriov_vf_device.attr, --=20 2.23.0