From nobody Sat Apr 11 02:18:39 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E64ECC25B0E for ; Tue, 16 Aug 2022 19:40:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237246AbiHPTkO (ORCPT ); Tue, 16 Aug 2022 15:40:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50826 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237239AbiHPTkL (ORCPT ); Tue, 16 Aug 2022 15:40:11 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 85AC88671F for ; Tue, 16 Aug 2022 12:40:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1660678809; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YJVt87I+RkFO3jUAhHrJxKBqiCZ7HeO0w2k1yZLam5o=; b=OCEKUEo5ip1C5eA+AkKjcJtL3dvsaa2Oi+dhRHRQtlp2DnJXeCa0LCef3RkiAYZZTRh3DX EZLkyZ5DrQmgyU1TamfVBvo5lOMSaZKV5M+5+f+qtKvySJYVuEmcPeZovZMSmYNzhaJ1sh cKt5yV2icvfgio5+ffrqwPS9Ps1+NzY= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-275-uFEVRtW1MFW4FV7_cbTngg-1; Tue, 16 Aug 2022 15:40:05 -0400 X-MC-Unique: uFEVRtW1MFW4FV7_cbTngg-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id CDFBD18A6524; Tue, 16 Aug 2022 19:40:04 +0000 (UTC) Received: from [172.30.41.16] (unknown [10.22.18.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 84E761415128; Tue, 16 Aug 2022 19:40:04 +0000 (UTC) Subject: [PATCH 1/3] PCI: Fix double-free in resource attribute error path From: Alex Williamson To: linux-pci@vger.kernel.org, bhelgaas@google.com Cc: linux-kernel@vger.kernel.org Date: Tue, 16 Aug 2022 13:40:04 -0600 Message-ID: <166067879638.1885802.1955389738707186135.stgit@omen> In-Reply-To: <166067824399.1885802.12557332818208187324.stgit@omen> References: <166067824399.1885802.12557332818208187324.stgit@omen> User-Agent: StGit/1.5.dev2+g9ce680a52bd9 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.85 on 10.11.54.7 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org If pci_create_attr() fails the remaining resource bin attributes are removed and freed based on the pointer entries which are not cleared in the case that creation of the bin attribute fails. Signed-off-by: Alex Williamson --- drivers/pci/pci-sysfs.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/drivers/pci/pci-sysfs.c b/drivers/pci/pci-sysfs.c index fc804e08e3cb..9ac92e6a2397 100644 --- a/drivers/pci/pci-sysfs.c +++ b/drivers/pci/pci-sysfs.c @@ -1196,8 +1196,14 @@ static int pci_create_attr(struct pci_dev *pdev, int= num, int write_combine) res_attr->size =3D pci_resource_len(pdev, num); res_attr->private =3D (void *)(unsigned long)num; retval =3D sysfs_create_bin_file(&pdev->dev.kobj, res_attr); - if (retval) + if (retval) { + if (write_combine) + pdev->res_attr_wc[num] =3D NULL; + else + pdev->res_attr[num] =3D NULL; + kfree(res_attr); + } =20 return retval; } From nobody Sat Apr 11 02:18:39 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E835C25B0E for ; Tue, 16 Aug 2022 19:40:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237301AbiHPTkj (ORCPT ); Tue, 16 Aug 2022 15:40:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51096 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236010AbiHPTkh (ORCPT ); Tue, 16 Aug 2022 15:40:37 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 97A9786889 for ; Tue, 16 Aug 2022 12:40:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1660678835; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ejPyL03exSUyv1sSmQS6RpycXfqdQ91zc3yteXbW3HE=; b=hspEqSQTyteVxK5Q3z9K4sQnNtEi3pAao/DGO44tRJx1aS1KsPAhQZM9i0nhmof/8ckVch Hi7CywLm1iRxDmW2z9+d9l7G7GL9kBM7+Cid/6NP2CiVVDC3QNU0hO27wHHm0ayfLJ66jS fCACZK4phiPZ4A7Ej7Jduw8VJ7YunkA= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-540-h7lRGLixPaON3uAu49SmIw-1; Tue, 16 Aug 2022 15:40:31 -0400 X-MC-Unique: h7lRGLixPaON3uAu49SmIw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id DDD301C04B63; Tue, 16 Aug 2022 19:40:30 +0000 (UTC) Received: from [172.30.41.16] (unknown [10.22.18.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 86B4B1121314; Tue, 16 Aug 2022 19:40:30 +0000 (UTC) Subject: [RFC PATCH 2/3] PCI: Skip reassigning bridge resources if reducing BAR size From: Alex Williamson To: linux-pci@vger.kernel.org, bhelgaas@google.com Cc: =?utf-8?q?Christian_K=C3=B6nig?= , linux-kernel@vger.kernel.org Date: Tue, 16 Aug 2022 13:40:30 -0600 Message-ID: <166067880991.1885802.3339740933334886723.stgit@omen> In-Reply-To: <166067824399.1885802.12557332818208187324.stgit@omen> References: <166067824399.1885802.12557332818208187324.stgit@omen> User-Agent: StGit/1.5.dev2+g9ce680a52bd9 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org More work is probably necessary here, pci_reassign_bridge_resources() has heavy weight requirements that the aperture windows are unused, without even testing if the requested resize is possible in the existing aperture. One case we can clearly skip is when reducing the size of a BAR. Cc: Christian K=C3=B6nig Signed-off-by: Alex Williamson --- drivers/pci/setup-res.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/pci/setup-res.c b/drivers/pci/setup-res.c index 439ac5f5907a..7a8e1e4f4d33 100644 --- a/drivers/pci/setup-res.c +++ b/drivers/pci/setup-res.c @@ -450,7 +450,7 @@ int pci_resize_resource(struct pci_dev *dev, int resno,= int size) res->end =3D res->start + pci_rebar_size_to_bytes(size) - 1; =20 /* Check if the new config works by trying to assign everything. */ - if (dev->bus->self) { + if (size > old && dev->bus->self) { ret =3D pci_reassign_bridge_resources(dev->bus->self, res->flags); if (ret) goto error_resize; From nobody Sat Apr 11 02:18:39 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 761F6C32772 for ; Tue, 16 Aug 2022 19:41:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237262AbiHPTlO (ORCPT ); Tue, 16 Aug 2022 15:41:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51966 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236885AbiHPTlL (ORCPT ); Tue, 16 Aug 2022 15:41:11 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8D04F883EE for ; Tue, 16 Aug 2022 12:41:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1660678868; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iy9C67BR9Y8mzLSmrgLV3EkX5N4BTVdAX1TDUnMFU5U=; b=JS2MRAOHJywQWwCt9TOi4bovthqVRuuPDjqk+F8FBjRjD+pOtkY99m+A4kpoNhlkpoo9y/ wX1pgFlUv/hveYnLj6MyN32Qcig0DDdpjmygjJzdBENXzK0SkN8PHO/jo8HhUkcqop7shL 0S6MtIF08P4psjWFw01g85Jk1Z+MtTA= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-213-uHBhXavFMo26heGh2Sk8OQ-1; Tue, 16 Aug 2022 15:41:05 -0400 X-MC-Unique: uHBhXavFMo26heGh2Sk8OQ-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 1FA0D3C0D19D; Tue, 16 Aug 2022 19:41:05 +0000 (UTC) Received: from [172.30.41.16] (unknown [10.22.18.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7DD612026D4C; Tue, 16 Aug 2022 19:41:04 +0000 (UTC) Subject: [RFC PATCH 3/3] PCI: Expose PCIe Resizable BAR support via sysfs From: Alex Williamson To: linux-pci@vger.kernel.org, bhelgaas@google.com Cc: =?utf-8?q?Christian_K=C3=B6nig?= , linux-kernel@vger.kernel.org Date: Tue, 16 Aug 2022 13:41:04 -0600 Message-ID: <166067883598.1885802.7663904087127986133.stgit@omen> In-Reply-To: <166067824399.1885802.12557332818208187324.stgit@omen> References: <166067824399.1885802.12557332818208187324.stgit@omen> User-Agent: StGit/1.5.dev2+g9ce680a52bd9 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This proposes a simple sysfs interface to Resizable BAR support, largely for the purposes of assigning such devices to a VM through VFIO. Resizable BARs present a difficult feature to expose to a VM through emulation, as resizing a BAR is done on the host. It can fail, and often does, but we have no means via emulation of a PCIe REBAR capability to handle the error cases. A vfio-pci specific ioctl interface is also cumbersome as there are often multiple devices within the same bridge aperture and handling them is a challenge. In the interface proposed here, expanding a BAR potentially requires such devices to be soft-removed during the resize operation and rescanned after, in order for all the necessary resources to be released. A pci-sysfs interface is also more universal than a vfio specific interface. Please see the ABI documentation update for usage. Cc: Christian K=C3=B6nig Signed-off-by: Alex Williamson --- NB, I realize the read value of the syfs attribute provides two values, the bitmap of possible sizes and the current size. There are a number of ways to determine the current size, including stat(1) on the resourceN file, but I found this output to be useful while developing the interface and provides consistency with the store value to the attribute. Suggestions welcome for better semantics. Documentation/ABI/testing/sysfs-bus-pci | 27 +++++++ drivers/pci/pci-sysfs.c | 118 +++++++++++++++++++++++++++= ++++ include/linux/pci.h | 1=20 3 files changed, 146 insertions(+) diff --git a/Documentation/ABI/testing/sysfs-bus-pci b/Documentation/ABI/te= sting/sysfs-bus-pci index 6fc2c2efe8ab..5eea5d89c9f2 100644 --- a/Documentation/ABI/testing/sysfs-bus-pci +++ b/Documentation/ABI/testing/sysfs-bus-pci @@ -457,3 +457,30 @@ Description: =20 The file is writable if the PF is bound to a driver that implements ->sriov_set_msix_vec_count(). + +What: /sys/bus/pci/devices/.../resourceN_resize +Date: August 2022 +Contact: Alex Williamson +Description: + These files provide an interface to PCIe Resizable BAR support. + A file is created for each BAR resource (N) supported by the + PCIe Resizable BAR extended capability of the device. Reading + each file exposes the capability and current setting for the + device, ex. + + # cat resource1_resize + 00000000000001c0:6 + + The first field provides the supported sizes bitmap, where + bit0 =3D 1MB, bit1 =3D 2MB, bit2 =3D 4MB, etc. In the above example + the devices supports 64MB, 128MB, and 256MB BAR sizes. The + second field provides the current setting, the value 6 + indicates bit6 is set, which corresponds to 64MB. + + When writing the file, only the latter is used, ex. + + # echo 7 > resource1_resize + + This indicates to set the size value corresponding to bit 7, + which is 128MB. The resulting size is 2 ^ (bit# + 20). This + definition matches the PCIe specification of this capability. diff --git a/drivers/pci/pci-sysfs.c b/drivers/pci/pci-sysfs.c index 9ac92e6a2397..aa59a2de508f 100644 --- a/drivers/pci/pci-sysfs.c +++ b/drivers/pci/pci-sysfs.c @@ -1143,6 +1143,7 @@ static void pci_remove_resource_files(struct pci_dev = *pdev) =20 for (i =3D 0; i < PCI_STD_NUM_BARS; i++) { struct bin_attribute *res_attr; + struct dev_ext_attribute *resize_attr; =20 res_attr =3D pdev->res_attr[i]; if (res_attr) { @@ -1155,6 +1156,13 @@ static void pci_remove_resource_files(struct pci_dev= *pdev) sysfs_remove_bin_file(&pdev->dev.kobj, res_attr); kfree(res_attr); } + + resize_attr =3D pdev->res_attr_resize[i]; + if (resize_attr) { + sysfs_remove_file(&pdev->dev.kobj, + &resize_attr->attr.attr); + kfree(resize_attr); + } } } =20 @@ -1208,6 +1216,108 @@ static int pci_create_attr(struct pci_dev *pdev, in= t num, int write_combine) return retval; } =20 +static ssize_t pci_bar_resize_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct pci_dev *pdev =3D to_pci_dev(dev); + struct dev_ext_attribute *resize_attr =3D + container_of(attr, struct dev_ext_attribute, attr); + int bar =3D (int)(long)resize_attr->var; + ssize_t ret; + + pci_config_pm_runtime_get(pdev); + + /* + * pci_rebar_get_possible_sizes() only currently reads supported sizes + * from the capability register and therefore returns a u32. The spec + * allows additional supported bits in the control register, which + * then exceeds 32bit. Expose a u64 to userspace for future compat. + */ + ret =3D sysfs_emit(buf, "%016llx:%d\n", + (u64)pci_rebar_get_possible_sizes(pdev, bar), + pci_rebar_get_current_size(pdev, bar)); + + pci_config_pm_runtime_put(pdev); + + return ret; +} + +static ssize_t pci_bar_resize_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + struct pci_dev *pdev =3D to_pci_dev(dev); + struct dev_ext_attribute *resize_attr =3D + container_of(attr, struct dev_ext_attribute, attr); + int ret, i, bar =3D (int)(long)resize_attr->var; + unsigned long size, flags; + u16 cmd; + + if (kstrtoul(buf, 0, &size) < 0) + return -EINVAL; + + device_lock(dev); + if (dev->driver) { + ret =3D -EBUSY; + goto unlock; + } + + pci_config_pm_runtime_get(pdev); + + pci_read_config_word(pdev, PCI_COMMAND, &cmd); + pci_write_config_word(pdev, PCI_COMMAND, cmd & ~PCI_COMMAND_MEMORY); + + flags =3D pci_resource_flags(pdev, bar); + + for (i =3D 0; i < PCI_STD_NUM_BARS; i++) { + if (pci_resource_len(pdev, i) && + pci_resource_flags(pdev, i) =3D=3D flags) + pci_release_resource(pdev, i); + } + + ret =3D pci_resize_resource(pdev, bar, size); + + pci_assign_unassigned_bus_resources(pdev->bus); + + pci_write_config_word(pdev, PCI_COMMAND, cmd); + + pci_config_pm_runtime_put(pdev); +unlock: + device_unlock(dev); + + return ret ? ret : count; +} + +static int pci_create_resize_attr(struct pci_dev *pdev, int num) +{ + struct dev_ext_attribute *resize_attr; + char *resize_attr_name; + int retval; + + resize_attr =3D kzalloc(sizeof(*resize_attr) + 17, GFP_ATOMIC); + if (!resize_attr) + return -ENOMEM; + + resize_attr_name =3D (char *)(resize_attr + 1); + + sysfs_attr_init(&resize_attr->attr.attr); + sprintf(resize_attr_name, "resource%d_resize", num); + resize_attr->attr.attr.name =3D resize_attr_name; + resize_attr->attr.attr.mode =3D 0600; + resize_attr->attr.show =3D pci_bar_resize_show; + resize_attr->attr.store =3D pci_bar_resize_store; + resize_attr->var =3D (void *)(long)num; + + retval =3D sysfs_create_file(&pdev->dev.kobj, &resize_attr->attr.attr); + if (retval) { + kfree(resize_attr); + return retval; + } + + pdev->res_attr_resize[num] =3D resize_attr; + return 0; +} + /** * pci_create_resource_files - create resource files in sysfs for @dev * @pdev: dev in question @@ -1235,6 +1345,14 @@ static int pci_create_resource_files(struct pci_dev = *pdev) pci_remove_resource_files(pdev); return retval; } + + if (pci_rebar_get_current_size(pdev, i) >=3D 0) { + retval =3D pci_create_resize_attr(pdev, i); + if (retval) { + pci_remove_resource_files(pdev); + return retval; + } + } } return 0; } diff --git a/include/linux/pci.h b/include/linux/pci.h index 060af91bafcd..9c4db0c5f215 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -470,6 +470,7 @@ struct pci_dev { int rom_attr_enabled; /* Display of ROM attribute enabled? */ struct bin_attribute *res_attr[DEVICE_COUNT_RESOURCE]; /* sysfs file for = resources */ struct bin_attribute *res_attr_wc[DEVICE_COUNT_RESOURCE]; /* sysfs file f= or WC mapping of resources */ + struct dev_ext_attribute *res_attr_resize[DEVICE_COUNT_RESOURCE]; /* sysf= s file for resizing BAR resources */ =20 #ifdef CONFIG_HOTPLUG_PCI_PCIE unsigned int broken_cmd_compl:1; /* No compl for some cmds */