From nobody Tue Sep 30 19:37:34 2025 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BF584101C8; Thu, 28 Dec 2023 16:57:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="AlQ3hn6V" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1703782654; x=1735318654; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=LCe+OPVf6ypAOTjFWig95nNeJVsgngMAJh/hD3RJ254=; b=AlQ3hn6VQyZT+Yt+YqQ76HGZTBfE3/Qcf9XZViby66JOy81i7qPJ6L4S Ghe4qMMNl+z+9pV0xAxrLSFNvzu+MjOqah77U2u0cdARDWOu841CX/3DB 4uJPxBDi1b+wfBeYKO5ysY6ToW/CnWg9aoAr1Xvdgh+UgPanBB2KsNXin 7Ceqnr+w+5Plt43Y3AemSaDXiQCkUyQibZrOkkOfREbtul7TitYEtvz3q oWDsi3PMPtzzBr3Gkzcqbk0+WCdjJOb6hwL1OCEd1bfazxqODxFLdEVnZ HWuuhytGXPvNNCXq1uC2dIWmrHefTYFIzGfzf30eA+cl6ljSnNMNiFjlo Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10937"; a="399349946" X-IronPort-AV: E=Sophos;i="6.04,312,1695711600"; d="scan'208";a="399349946" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Dec 2023 08:57:34 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10937"; a="728356748" X-IronPort-AV: E=Sophos;i="6.04,312,1695711600"; d="scan'208";a="728356748" Received: from ijarvine-desk1.ger.corp.intel.com (HELO localhost) ([10.94.250.171]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Dec 2023 08:57:28 -0800 From: =?UTF-8?q?Ilpo=20J=C3=A4rvinen?= To: linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Rob Herring , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Igor Mammedov , Lukas Wunner , Mika Westerberg , Andy Shevchenko , "Rafael J . Wysocki" , Jesse Barnes , Yinghai Lu , linux-kernel@vger.kernel.org Cc: =?UTF-8?q?Ilpo=20J=C3=A4rvinen?= Subject: [PATCH v2 1/7] PCI: Fix resource double counting on remove & rescan Date: Thu, 28 Dec 2023 18:57:01 +0200 Message-Id: <20231228165707.3447-2-ilpo.jarvinen@linux.intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20231228165707.3447-1-ilpo.jarvinen@linux.intel.com> References: <20231228165707.3447-1-ilpo.jarvinen@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable pbus_size_mem() keeps the size of the optional resources in children_add_size. When calculating the PCI bridge window size, calculate_memsize() lower bounds size by old_size before adding children_add_size and performing the window size alignment. This results in double counting for the resources in children_add_size because old_size may be based on the previous size of the bridge window after it has already included children_add_size (that is, size1 in pbus_size_mem() from an earlier invocation of that function). As a result, on repeated remove of the bus & rescan cycles the resource size keeps increasing when children_add_size is non-zero as can be seen from this extract: iomem0: 23fffd00000-23fffdfffff : PCI Bus 0000:03 iomem1: 20000000000-200001fffff : PCI Bus 0000:03 iomem2: 20000000000-200002fffff : PCI Bus 0000:03 iomem3: 20000000000-200003fffff : PCI Bus 0000:03 iomem4: 20000000000-200004fffff : PCI Bus 0000:03 Solve the double counting by moving old_size check later in calculate_memsize() so that children_add_size is already accounted for. After the patch, the bridge window retains its size as expected: iomem0: 23fffd00000-23fffdfffff : PCI Bus 0000:03 iomem1: 20000000000-200000fffff : PCI Bus 0000:03 iomem2: 20000000000-200000fffff : PCI Bus 0000:03 Fixes: a4ac9fea016f ("PCI : Calculate right add_size") Signed-off-by: Ilpo J=C3=A4rvinen Reviewed-by: Mika Westerberg Tested-by: Lidong Wang =20 --- drivers/pci/setup-bus.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index fd74f1c99dba..e3e6ff8854a7 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -820,11 +820,9 @@ static resource_size_t calculate_memsize(resource_size= _t size, size =3D min_size; if (old_size =3D=3D 1) old_size =3D 0; - if (size < old_size) - size =3D old_size; =20 - size =3D ALIGN(max(size, add_size) + children_add_size, align); - return size; + size =3D max(size, add_size) + children_add_size; + return ALIGN(max(size, old_size), align); } =20 resource_size_t __weak pcibios_window_alignment(struct pci_bus *bus, --=20 2.30.2 From nobody Tue Sep 30 19:37:34 2025 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.126]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4A03C101C6; Thu, 28 Dec 2023 16:57:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="cmlibASN" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1703782664; x=1735318664; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=7uBBW2FH+XAQ1x3ggNOvzbi3Xo6RF4TvkDUO9/8j5q4=; b=cmlibASNgSruc0ZrVsS+c8wXITzrUabQq3V91lmySIFPUdOSs7liZsZU cfjX+3x7WlNVZ9v0P3YxBaFpVZtmsi9TZNRx+RBr5PscQJskvMx3rOypX NN8oNimpLTS2rmwGdDze2dmlg3zaoTucl5EABFdEijlUULFVP9h/DN2ir wZvDsrEqhe1MY6C2Q/wOwTIsyXb4/b9ooF799/zfhtlooFln7nI0Bz/E4 WuVzripYdM8oscoYP+zcihT1ZxlBThIAX6/+eTL71GFMSpv8uFd9c0zi3 I8vQ7nbqiFg5Vboa29i9mYM1MKCIm24eYUXxQ4WwJhDTx8rXn54jXkj5z g==; X-IronPort-AV: E=McAfee;i="6600,9927,10937"; a="381536596" X-IronPort-AV: E=Sophos;i="6.04,312,1695711600"; d="scan'208";a="381536596" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Dec 2023 08:57:43 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10937"; a="844488473" X-IronPort-AV: E=Sophos;i="6.04,312,1695711600"; d="scan'208";a="844488473" Received: from ijarvine-desk1.ger.corp.intel.com (HELO localhost) ([10.94.250.171]) by fmsmga008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Dec 2023 08:57:39 -0800 From: =?UTF-8?q?Ilpo=20J=C3=A4rvinen?= To: linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Rob Herring , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Igor Mammedov , Lukas Wunner , Mika Westerberg , Andy Shevchenko , "Rafael J . Wysocki" , linux-kernel@vger.kernel.org Cc: =?UTF-8?q?Ilpo=20J=C3=A4rvinen?= , Andy Shevchenko Subject: [PATCH v2 2/7] resource: Rename find_resource() to find_empty_resource_slot() Date: Thu, 28 Dec 2023 18:57:02 +0200 Message-Id: <20231228165707.3447-3-ilpo.jarvinen@linux.intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20231228165707.3447-1-ilpo.jarvinen@linux.intel.com> References: <20231228165707.3447-1-ilpo.jarvinen@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Rename find_resource() to find_empty_resource_slot() to better describe what the functions does. This is a preparation for exposing it beyond resource.c which is needed by PCI core. Also rename the __ variant to match the names. Signed-off-by: Ilpo J=C3=A4rvinen Reviewed-by: Andy Shevchenko Reviewed-by: Mika Westerberg Tested-by: Lidong Wang =20 --- kernel/resource.c | 19 +++++++++---------- 1 file changed, 9 insertions(+), 10 deletions(-) diff --git a/kernel/resource.c b/kernel/resource.c index 866ef3663a0b..94f67005e1e2 100644 --- a/kernel/resource.c +++ b/kernel/resource.c @@ -574,10 +574,9 @@ static void resource_clip(struct resource *res, resour= ce_size_t min, * Find empty slot in the resource tree with the given range and * alignment constraints */ -static int __find_resource(struct resource *root, struct resource *old, - struct resource *new, - resource_size_t size, - struct resource_constraint *constraint) +static int __find_empty_resource_slot(struct resource *root, struct resour= ce *old, + struct resource *new, resource_size_t size, + struct resource_constraint *constraint) { struct resource *this =3D root->child; struct resource tmp =3D *new, avail, alloc; @@ -633,11 +632,11 @@ next: if (!this || this->end =3D=3D root->end) /* * Find empty slot in the resource tree given range and alignment. */ -static int find_resource(struct resource *root, struct resource *new, - resource_size_t size, - struct resource_constraint *constraint) +static int find_empty_resource_slot(struct resource *root, struct resource= *new, + resource_size_t size, + struct resource_constraint *constraint) { - return __find_resource(root, NULL, new, size, constraint); + return __find_empty_resource_slot(root, NULL, new, size, constraint); } =20 /** @@ -660,7 +659,7 @@ static int reallocate_resource(struct resource *root, s= truct resource *old, =20 write_lock(&resource_lock); =20 - if ((err =3D __find_resource(root, old, &new, newsize, constraint))) + if ((err =3D __find_empty_resource_slot(root, old, &new, newsize, constra= int))) goto out; =20 if (resource_contains(&new, old)) { @@ -729,7 +728,7 @@ int allocate_resource(struct resource *root, struct res= ource *new, } =20 write_lock(&resource_lock); - err =3D find_resource(root, new, size, &constraint); + err =3D find_empty_resource_slot(root, new, size, &constraint); if (err >=3D 0 && __request_resource(root, new)) err =3D -EBUSY; write_unlock(&resource_lock); --=20 2.30.2 From nobody Tue Sep 30 19:37:34 2025 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A45B110795; Thu, 28 Dec 2023 16:57:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="I87tIBWH" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1703782675; x=1735318675; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=qcS3p0Xkas3hntmtuKxSyunc5fXp49lTbIYSeCzY6N4=; b=I87tIBWH+kEDXyI4+/g8cUfACDjcuuF7gtCXTbTFsSRpcRnv6MbCU+JO uI+ICJjuKYwUpdjuFaWYldi8oQiU4O79eon2G+68PI4kKpeQ/P6sxdf7v EAmR44/PVhxql2/Tgvm7xlUtwetAZayjGNEVZ7XzjtH9pswHmp1wg4Lz3 9wAVtfQD9HIdF8TnDENKWDc9mrrgkwcEqZ4u9sse55S5a2Mj2WSaBA0dA vFLEVnOnKaLoRMfMKzD0B5vgPpjPW4hi+F22cjxeV701yUqUqYBqobLBG moB8CK71ucovKE+Zq0WxC4IEVZkz4V4UD7i6AOpdcBZzDFWFT/8eO/YLU w==; X-IronPort-AV: E=McAfee;i="6600,9927,10937"; a="3409918" X-IronPort-AV: E=Sophos;i="6.04,312,1695711600"; d="scan'208";a="3409918" Received: from orviesa002.jf.intel.com ([10.64.159.142]) by fmvoesa105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Dec 2023 08:57:54 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.04,312,1695711600"; d="scan'208";a="20302116" Received: from ijarvine-desk1.ger.corp.intel.com (HELO localhost) ([10.94.250.171]) by orviesa002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Dec 2023 08:57:48 -0800 From: =?UTF-8?q?Ilpo=20J=C3=A4rvinen?= To: linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Rob Herring , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Igor Mammedov , Lukas Wunner , Mika Westerberg , Andy Shevchenko , "Rafael J . Wysocki" , linux-kernel@vger.kernel.org Cc: =?UTF-8?q?Ilpo=20J=C3=A4rvinen?= , Andy Shevchenko Subject: [PATCH v2 3/7] resource: Document find_empty_resource_slot() and resource_constraint Date: Thu, 28 Dec 2023 18:57:03 +0200 Message-Id: <20231228165707.3447-4-ilpo.jarvinen@linux.intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20231228165707.3447-1-ilpo.jarvinen@linux.intel.com> References: <20231228165707.3447-1-ilpo.jarvinen@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Document find_empty_resource_slot() and the struct resource_constraint as they are going to be exposed outside of resource.c. Signed-off-by: Ilpo J=C3=A4rvinen Reviewed-by: Andy Shevchenko Reviewed-by: Mika Westerberg Tested-by: Lidong Wang =20 --- kernel/resource.c | 28 +++++++++++++++++++++++++--- 1 file changed, 25 insertions(+), 3 deletions(-) diff --git a/kernel/resource.c b/kernel/resource.c index 94f67005e1e2..ed4bb8ad701a 100644 --- a/kernel/resource.c +++ b/kernel/resource.c @@ -46,7 +46,18 @@ struct resource iomem_resource =3D { }; EXPORT_SYMBOL(iomem_resource); =20 -/* constraints to be met while allocating resources */ +/** + * resource_constraint - constraints to be met while searching empty resou= rce slots + * @min: The minimum address for the memory range + * @max: The maximum address for the memory range + * @align: Alignment for the start address of the empty slot + * @alignf: Additional alignment constraints callback + * @alignf_data: Data provided for @alignf callback + * + * Contains the range and alignment constraints that have to be met during + * find_empty_resource_slot(). @alignf can be NULL indicating no alignment + * beyond @align is necessary. + */ struct resource_constraint { resource_size_t min, max, align; resource_size_t (*alignf)(void *, const struct resource *, @@ -629,8 +640,19 @@ next: if (!this || this->end =3D=3D root->end) return -EBUSY; } =20 -/* - * Find empty slot in the resource tree given range and alignment. +/** + * find_empty_resource_slot - Find empty slot in the resource tree + * @root: Root resource descriptor + * @new: Resource descriptor awaiting an empty resource slot + * @size: The minimum size of the empty slot + * @constraint: The range and alignment constraints to be met + * + * Finds an empty slot under @root in the resource tree satisfying range a= nd + * alignment @constraints. + * + * Return: + * * %0 - if successful, @new members start, end, and flags are altered. + * * %-EBUSY - if no empty slot was found. */ static int find_empty_resource_slot(struct resource *root, struct resource= *new, resource_size_t size, --=20 2.30.2 From nobody Tue Sep 30 19:37:34 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E9B841079A; Thu, 28 Dec 2023 16:58:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="oJLDlwXD" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1703782685; x=1735318685; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=G6NvyQgAcHvy8o7E18KM+q0zCkzqHm3N1UhsVgBDCvA=; b=oJLDlwXDfPB53ovTc5mM+GWzwVSSQhlQtcIBaxqvASXF1YZNHx8MIhlH 0SLXAbbPTNE4RPD41f5QGxSBBibgJ4ehIYeeWPW9PGwkC6eotg9hpsgZx HrDAGI2HN6mP/f9rQ24c6QnqENkuejo/Q7+w+mXBEGOB9EWoIYGbJ3DK3 n/gvBD2cpjisyrNquPqnWZo3Yg3AnfT816Tdmhtk1MVhswO2AM4dkU8qF renG0CU3Gdg1SSo2VNgqG8obASR2GDt294VwauBexWaEGVAmeuI7zmCxO dtwXrVL+FSvnTQh58pXkS6FnDXKNI175CcbQbk1o7/D7FWcrQgx7ryIhd w==; X-IronPort-AV: E=McAfee;i="6600,9927,10937"; a="9958035" X-IronPort-AV: E=Sophos;i="6.04,312,1695711600"; d="scan'208";a="9958035" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Dec 2023 08:58:04 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10937"; a="951847165" X-IronPort-AV: E=Sophos;i="6.04,312,1695711600"; d="scan'208";a="951847165" Received: from ijarvine-desk1.ger.corp.intel.com (HELO localhost) ([10.94.250.171]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Dec 2023 08:57:59 -0800 From: =?UTF-8?q?Ilpo=20J=C3=A4rvinen?= To: linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Rob Herring , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Igor Mammedov , Lukas Wunner , Mika Westerberg , Andy Shevchenko , "Rafael J . Wysocki" , linux-kernel@vger.kernel.org Cc: =?UTF-8?q?Ilpo=20J=C3=A4rvinen?= , Andy Shevchenko Subject: [PATCH v2 4/7] resource: Use typedef for alignf callback Date: Thu, 28 Dec 2023 18:57:04 +0200 Message-Id: <20231228165707.3447-5-ilpo.jarvinen@linux.intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20231228165707.3447-1-ilpo.jarvinen@linux.intel.com> References: <20231228165707.3447-1-ilpo.jarvinen@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable To make it simpler to declare resource constraint alignf callbacks, add typedef for it and document it. Suggested-by: Andy Shevchenko Signed-off-by: Ilpo J=C3=A4rvinen Reviewed-by: Andy Shevchenko Reviewed-by: Mika Westerberg Tested-by: Lidong Wang =20 --- drivers/pci/bus.c | 10 ++-------- include/linux/ioport.h | 22 ++++++++++++++++++---- include/linux/pci.h | 5 +---- kernel/resource.c | 8 ++------ 4 files changed, 23 insertions(+), 22 deletions(-) diff --git a/drivers/pci/bus.c b/drivers/pci/bus.c index 9c2137dae429..be5a4bb4d2fb 100644 --- a/drivers/pci/bus.c +++ b/drivers/pci/bus.c @@ -176,10 +176,7 @@ static void pci_clip_resource_to_region(struct pci_bus= *bus, static int pci_bus_alloc_from_region(struct pci_bus *bus, struct resource = *res, resource_size_t size, resource_size_t align, resource_size_t min, unsigned long type_mask, - resource_size_t (*alignf)(void *, - const struct resource *, - resource_size_t, - resource_size_t), + resource_alignf alignf, void *alignf_data, struct pci_bus_region *region) { @@ -250,10 +247,7 @@ static int pci_bus_alloc_from_region(struct pci_bus *b= us, struct resource *res, int pci_bus_alloc_resource(struct pci_bus *bus, struct resource *res, resource_size_t size, resource_size_t align, resource_size_t min, unsigned long type_mask, - resource_size_t (*alignf)(void *, - const struct resource *, - resource_size_t, - resource_size_t), + resource_alignf alignf, void *alignf_data) { #ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT diff --git a/include/linux/ioport.h b/include/linux/ioport.h index 14f5cfabbbc8..26bfd9515653 100644 --- a/include/linux/ioport.h +++ b/include/linux/ioport.h @@ -188,6 +188,23 @@ enum { #define DEFINE_RES_DMA(_dma) \ DEFINE_RES_DMA_NAMED((_dma), NULL) =20 +/** + * typedef resource_alignf - Resource alignment callback + * @data: Private data used by the callback + * @res: Resource candidate range (an empty resource slot) + * @size: The minimum size of the empty slot + * @align: Alignment from the constraints + * + * Callback allows calculating resource placement and alignment beyond min, + * max, and align fields in the struct resource_constraint. + * + * Return: Start address for the resource. + */ +typedef resource_size_t (*resource_alignf)(void *data, + const struct resource *res, + resource_size_t size, + resource_size_t align); + /* PC/ISA/whatever - the normal PC address spaces: IO and memory */ extern struct resource ioport_resource; extern struct resource iomem_resource; @@ -207,10 +224,7 @@ extern void arch_remove_reservations(struct resource *= avail); extern int allocate_resource(struct resource *root, struct resource *new, resource_size_t size, resource_size_t min, resource_size_t max, resource_size_t align, - resource_size_t (*alignf)(void *, - const struct resource *, - resource_size_t, - resource_size_t), + resource_alignf alignf, void *alignf_data); struct resource *lookup_resource(struct resource *root, resource_size_t st= art); int adjust_resource(struct resource *res, resource_size_t start, diff --git a/include/linux/pci.h b/include/linux/pci.h index 60ca768bc867..d635e64debd9 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -1546,10 +1546,7 @@ int __must_check pci_bus_alloc_resource(struct pci_b= us *bus, struct resource *res, resource_size_t size, resource_size_t align, resource_size_t min, unsigned long type_mask, - resource_size_t (*alignf)(void *, - const struct resource *, - resource_size_t, - resource_size_t), + resource_alignf alignf, void *alignf_data); =20 =20 diff --git a/kernel/resource.c b/kernel/resource.c index ed4bb8ad701a..9d7920104120 100644 --- a/kernel/resource.c +++ b/kernel/resource.c @@ -60,8 +60,7 @@ EXPORT_SYMBOL(iomem_resource); */ struct resource_constraint { resource_size_t min, max, align; - resource_size_t (*alignf)(void *, const struct resource *, - resource_size_t, resource_size_t); + resource_alignf alignf; void *alignf_data; }; =20 @@ -725,10 +724,7 @@ static int reallocate_resource(struct resource *root, = struct resource *old, int allocate_resource(struct resource *root, struct resource *new, resource_size_t size, resource_size_t min, resource_size_t max, resource_size_t align, - resource_size_t (*alignf)(void *, - const struct resource *, - resource_size_t, - resource_size_t), + resource_alignf alignf, void *alignf_data) { int err; --=20 2.30.2 From nobody Tue Sep 30 19:37:34 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CDA2C10780; Thu, 28 Dec 2023 16:58:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="HcWG7dvP" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1703782695; x=1735318695; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=3UeWqTc17b7iPzk1HVe/oYZDCXCyjBwLkhpNmmvXEjw=; b=HcWG7dvP1MycaW0AkFmXKH+k7N2UigDsmv+fQ94kJYfjEbpO4t87XBwb ISQ0FkV2KnttKjJQ6VNtDPlbezWND87Jk7OewrRVhfME3j31us2YZG5bX xSQkBpu8pxR2UqVZUjtEtltrPegKtqOuHH/zWlbRYldK/XRjJOxYyD87d pNlp5N27nrqWdj2miPFrbOwtFGn5ht1ZsmKRfifbH+LcQEWdnBi35sPdX pl+euJzuK0DuUqm/Lp/159mnXTHCXD1jcEuwZSEHGf7YnP2JaDxX+9DdH T9wNUWQpEtfL3MqOR6YMCqPmTXGdzJmVAx1wYB+Do7wQqa5dae3pCZTdU Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10937"; a="9958053" X-IronPort-AV: E=Sophos;i="6.04,312,1695711600"; d="scan'208";a="9958053" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Dec 2023 08:58:15 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10937"; a="951847238" X-IronPort-AV: E=Sophos;i="6.04,312,1695711600"; d="scan'208";a="951847238" Received: from ijarvine-desk1.ger.corp.intel.com (HELO localhost) ([10.94.250.171]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Dec 2023 08:58:09 -0800 From: =?UTF-8?q?Ilpo=20J=C3=A4rvinen?= To: linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Rob Herring , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Igor Mammedov , Lukas Wunner , Mika Westerberg , Andy Shevchenko , "Rafael J . Wysocki" , linux-kernel@vger.kernel.org Cc: =?UTF-8?q?Ilpo=20J=C3=A4rvinen?= , Andy Shevchenko Subject: [PATCH v2 5/7] resource: Handle simple alignment inside __find_empty_resource_slot() Date: Thu, 28 Dec 2023 18:57:05 +0200 Message-Id: <20231228165707.3447-6-ilpo.jarvinen@linux.intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20231228165707.3447-1-ilpo.jarvinen@linux.intel.com> References: <20231228165707.3447-1-ilpo.jarvinen@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable allocate_resource() accepts ->alignf() callback to perform custom alignment beyond constraint->align. If alignf is NULL, simple_align_resource() is used which only returns avail->start (no change). Using avail->start directly is natural and can be done with a conditional in __find_empty_resource_slot() instead which avoids unnecessarily using callback. It makes the code inside __find_empty_resource_slot() more obvious and removes the need for the caller to provide constraint->alignf unnecessarily. This is preparation for exporting find_empty_resource_slot(). Signed-off-by: Ilpo J=C3=A4rvinen Reviewed-by: Andy Shevchenko Reviewed-by: Mika Westerberg Tested-by: Lidong Wang =20 --- kernel/resource.c | 20 +++++++------------- 1 file changed, 7 insertions(+), 13 deletions(-) diff --git a/kernel/resource.c b/kernel/resource.c index 9d7920104120..f0e4d57d0c35 100644 --- a/kernel/resource.c +++ b/kernel/resource.c @@ -563,14 +563,6 @@ void __weak arch_remove_reservations(struct resource *= avail) { } =20 -static resource_size_t simple_align_resource(void *data, - const struct resource *avail, - resource_size_t size, - resource_size_t align) -{ - return avail->start; -} - static void resource_clip(struct resource *res, resource_size_t min, resource_size_t max) { @@ -590,6 +582,7 @@ static int __find_empty_resource_slot(struct resource *= root, struct resource *ol { struct resource *this =3D root->child; struct resource tmp =3D *new, avail, alloc; + resource_alignf alignf =3D constraint->alignf; =20 tmp.start =3D root->start; /* @@ -618,8 +611,12 @@ static int __find_empty_resource_slot(struct resource = *root, struct resource *ol avail.flags =3D new->flags & ~IORESOURCE_UNSET; if (avail.start >=3D tmp.start) { alloc.flags =3D avail.flags; - alloc.start =3D constraint->alignf(constraint->alignf_data, &avail, - size, constraint->align); + if (alignf) { + alloc.start =3D alignf(constraint->alignf_data, + &avail, size, constraint->align); + } else { + alloc.start =3D avail.start; + } alloc.end =3D alloc.start + size - 1; if (alloc.start <=3D alloc.end && resource_contains(&avail, &alloc)) { @@ -730,9 +727,6 @@ int allocate_resource(struct resource *root, struct res= ource *new, int err; struct resource_constraint constraint; =20 - if (!alignf) - alignf =3D simple_align_resource; - constraint.min =3D min; constraint.max =3D max; constraint.align =3D align; --=20 2.30.2 From nobody Tue Sep 30 19:37:34 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 497D11094A; Thu, 28 Dec 2023 16:58:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="RzDCblR/" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1703782706; x=1735318706; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=WYw+ej8p75zSV2PmRVDEXfp4ji6lrtgJzEwDsDP49x0=; b=RzDCblR/5VtUnlVh9tyGo2ztgfg7fST3j6IOU18rCVQU3gZzyKdQNqh+ O69b9TtU02E7F1ojIHVyyUFh4Xpl0NBV5dDLpP4ep7yCDWrDkGYLCMVaS Jsq2bQnQpcHg6axtu45IwzLNNFCPG+7jVR1C+6EjsGowZbmlPh9/t8lOD sODrxH/Qc4ByrYBgPoQiW7YcjjG81OqaNs72HEO9RRgv0bnBB2vJj+VTh 2GA9Pwfowef8wobV4Zkzu/gcBDN3ar2rITlvD3lJSlevcN5BS5Cvr18eF ZMCIeNXqY4eoZ+HeEOjCq2E9nv7+QwFsO7kk94iSQqvIu0JCBdLPaNEsS A==; X-IronPort-AV: E=McAfee;i="6600,9927,10937"; a="9958072" X-IronPort-AV: E=Sophos;i="6.04,312,1695711600"; d="scan'208";a="9958072" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Dec 2023 08:58:25 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10937"; a="951847269" X-IronPort-AV: E=Sophos;i="6.04,312,1695711600"; d="scan'208";a="951847269" Received: from ijarvine-desk1.ger.corp.intel.com (HELO localhost) ([10.94.250.171]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Dec 2023 08:58:19 -0800 From: =?UTF-8?q?Ilpo=20J=C3=A4rvinen?= To: linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Rob Herring , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Igor Mammedov , Lukas Wunner , Mika Westerberg , Andy Shevchenko , "Rafael J . Wysocki" , linux-kernel@vger.kernel.org Cc: =?UTF-8?q?Ilpo=20J=C3=A4rvinen?= , Andy Shevchenko Subject: [PATCH v2 6/7] resource: Export find_empty_resource_slot() Date: Thu, 28 Dec 2023 18:57:06 +0200 Message-Id: <20231228165707.3447-7-ilpo.jarvinen@linux.intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20231228165707.3447-1-ilpo.jarvinen@linux.intel.com> References: <20231228165707.3447-1-ilpo.jarvinen@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable PCI bridge window logic needs to find out in advance to the actual allocation if there is an empty slot big enough to fit the window. Export find_empty_resource_slot() for the purpose. Also move the struct resource_constraint into generic header to be able to use the new interface. Signed-off-by: Ilpo J=C3=A4rvinen Reviewed-by: Andy Shevchenko Reviewed-by: Mika Westerberg Tested-by: Lidong Wang =20 --- include/linux/ioport.h | 22 ++++++++++++++++++++++ kernel/resource.c | 25 ++++--------------------- 2 files changed, 26 insertions(+), 21 deletions(-) diff --git a/include/linux/ioport.h b/include/linux/ioport.h index 26bfd9515653..b6ed244bc4b9 100644 --- a/include/linux/ioport.h +++ b/include/linux/ioport.h @@ -205,6 +205,24 @@ typedef resource_size_t (*resource_alignf)(void *data, resource_size_t size, resource_size_t align); =20 +/** + * resource_constraint - constraints to be met while searching empty resou= rce slots + * @min: The minimum address for the memory range + * @max: The maximum address for the memory range + * @align: Alignment for the start address of the empty slot + * @alignf: Additional alignment constraints callback + * @alignf_data: Data provided for @alignf callback + * + * Contains the range and alignment constraints that have to be met during + * find_empty_resource_slot(). @alignf can be NULL indicating no alignment + * beyond @align is necessary. + */ +struct resource_constraint { + resource_size_t min, max, align; + resource_alignf alignf; + void *alignf_data; +}; + /* PC/ISA/whatever - the normal PC address spaces: IO and memory */ extern struct resource ioport_resource; extern struct resource iomem_resource; @@ -278,6 +296,10 @@ static inline bool resource_union(const struct resourc= e *r1, const struct resour return true; } =20 +int find_empty_resource_slot(struct resource *root, struct resource *new, + resource_size_t size, + struct resource_constraint *constraint); + /* Convenience shorthand with allocation */ #define request_region(start,n,name) __request_region(&ioport_resource, (= start), (n), (name), 0) #define request_muxed_region(start,n,name) __request_region(&ioport_resour= ce, (start), (n), (name), IORESOURCE_MUXED) diff --git a/kernel/resource.c b/kernel/resource.c index f0e4d57d0c35..c1bbb0943d9c 100644 --- a/kernel/resource.c +++ b/kernel/resource.c @@ -46,24 +46,6 @@ struct resource iomem_resource =3D { }; EXPORT_SYMBOL(iomem_resource); =20 -/** - * resource_constraint - constraints to be met while searching empty resou= rce slots - * @min: The minimum address for the memory range - * @max: The maximum address for the memory range - * @align: Alignment for the start address of the empty slot - * @alignf: Additional alignment constraints callback - * @alignf_data: Data provided for @alignf callback - * - * Contains the range and alignment constraints that have to be met during - * find_empty_resource_slot(). @alignf can be NULL indicating no alignment - * beyond @align is necessary. - */ -struct resource_constraint { - resource_size_t min, max, align; - resource_alignf alignf; - void *alignf_data; -}; - static DEFINE_RWLOCK(resource_lock); =20 static struct resource *next_resource(struct resource *p, bool skip_childr= en) @@ -650,12 +632,13 @@ next: if (!this || this->end =3D=3D root->end) * * %0 - if successful, @new members start, end, and flags are altered. * * %-EBUSY - if no empty slot was found. */ -static int find_empty_resource_slot(struct resource *root, struct resource= *new, - resource_size_t size, - struct resource_constraint *constraint) +int find_empty_resource_slot(struct resource *root, struct resource *new, + resource_size_t size, + struct resource_constraint *constraint) { return __find_empty_resource_slot(root, NULL, new, size, constraint); } +EXPORT_SYMBOL_GPL(find_empty_resource_slot); =20 /** * reallocate_resource - allocate a slot in the resource tree given range = & alignment. --=20 2.30.2 From nobody Tue Sep 30 19:37:34 2025 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.126]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5B12410A06; Thu, 28 Dec 2023 16:58:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="LARfE/EZ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1703782715; x=1735318715; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=9cmBqpNWxOkx/qRL+GcnO2FP0hog/X0WaEdjSaE40iI=; b=LARfE/EZYUnTKh5Nv3Zu2DP9KF+4Zu39FXYjJBv7Ly6cvHhaYoYSvY56 YQJDvm9mJdjH8KlZ8M//Esik1j0WM0MH13dbe+ALAIqivMwpnjL6QrMkF 1alqq41ASjR1AD2xxRfgNN/Q2MnI9o1Io7NNmYCI1Nfn89Lxprq2jbbA+ slDfHD3kMdTDI7hSyaXFr5O60VG+7sdT0gAez5uD8Lh4kI8qu9/1e2Piy b4NyF3Nt6aVTec88f4Hl/6sYf75j7HwWng6PN5uwrqnmZNGpiF+weBCHg LQE6qvmWERvjeM6BSc+2Q/qfPrBK1cCDbOzx5GU2r6mS1TvJeQAylvNun g==; X-IronPort-AV: E=McAfee;i="6600,9927,10937"; a="381536710" X-IronPort-AV: E=Sophos;i="6.04,312,1695711600"; d="scan'208";a="381536710" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Dec 2023 08:58:34 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10937"; a="844488561" X-IronPort-AV: E=Sophos;i="6.04,312,1695711600"; d="scan'208";a="844488561" Received: from ijarvine-desk1.ger.corp.intel.com (HELO localhost) ([10.94.250.171]) by fmsmga008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Dec 2023 08:58:30 -0800 From: =?UTF-8?q?Ilpo=20J=C3=A4rvinen?= To: linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Rob Herring , =?UTF-8?q?Krzysztof=20Wilczy=C5=84ski?= , Igor Mammedov , Lukas Wunner , Mika Westerberg , Andy Shevchenko , "Rafael J . Wysocki" , linux-kernel@vger.kernel.org Cc: =?UTF-8?q?Ilpo=20J=C3=A4rvinen?= Subject: [PATCH v2 7/7] PCI: Relax bridge window tail sizing rules Date: Thu, 28 Dec 2023 18:57:07 +0200 Message-Id: <20231228165707.3447-8-ilpo.jarvinen@linux.intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20231228165707.3447-1-ilpo.jarvinen@linux.intel.com> References: <20231228165707.3447-1-ilpo.jarvinen@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable During remove & rescan cycle, PCI subsystem will recalculate and adjust the bridge window sizing that was initially done by "BIOS". The size calculation is based on the required alignment of the largest resource among the downstream resources as per pbus_size_mem() (unimportant or zero parameters marked with "..."): min_align =3D calculate_mem_align(aligns, max_order); size0 =3D calculate_memsize(size, ..., min_align); and then in calculate_memsize(): size =3D ALIGN(max(size, ...) + ..., align); If the original bridge window sizing tried to conserve space, this will lead to massive increase of the required bridge window size when the downstream has a large disparity in BAR sizes. E.g., with 16MiB and 16GiB BARs this results in 32GiB bridge window size even if 16MiB BAR does not require gigabytes of space to fit. When doing remove & rescan for a bus that contains such a PCI device, a larger bridge window is suddenly required on rescan but when there is a bridge window upstream that is already assigned based on the original size, it cannot be enlarged to the new requirement. This causes the allocation of the bridge window to fail (0x600000000 > 0x400ffffff): pci 0000:02:01.0: PCI bridge to [bus 03] pci 0000:02:01.0: bridge window [mem 0x40400000-0x405fffff] pci 0000:02:01.0: bridge window [mem 0x6000000000-0x6400ffffff 64bit pref] pci 0000:01:00.0: PCI bridge to [bus 02-04] pci 0000:01:00.0: bridge window [mem 0x40400000-0x406fffff] pci 0000:01:00.0: bridge window [mem 0x6000000000-0x6400ffffff 64bit pref] ... pci_bus 0000:03: busn_res: [bus 03] is released pci 0000:03:00.0: reg 0x10: [mem 0x6400000000-0x6400ffffff 64bit pref] pci 0000:03:00.0: reg 0x18: [mem 0x6000000000-0x63ffffffff 64bit pref] pci 0000:03:00.0: reg 0x30: [mem 0x40400000-0x405fffff pref] pci 0000:02:01.0: PCI bridge to [bus 03] pci 0000:02:01.0: bridge window [mem 0x40400000-0x405fffff] pci 0000:02:01.0: bridge window [mem 0x6000000000-0x6400ffffff 64bit pref] pci 0000:02:01.0: BAR 9: no space for [mem size 0x600000000 64bit pref] pci 0000:02:01.0: BAR 9: failed to assign [mem size 0x600000000 64bit pref] pci 0000:02:01.0: BAR 8: assigned [mem 0x40400000-0x405fffff] pci 0000:03:00.0: BAR 2: no space for [mem size 0x400000000 64bit pref] pci 0000:03:00.0: BAR 2: failed to assign [mem size 0x400000000 64bit pref] pci 0000:03:00.0: BAR 0: no space for [mem size 0x01000000 64bit pref] pci 0000:03:00.0: BAR 0: failed to assign [mem size 0x01000000 64bit pref] pci 0000:03:00.0: BAR 6: assigned [mem 0x40400000-0x405fffff pref] pci 0000:02:01.0: PCI bridge to [bus 03] pci 0000:02:01.0: bridge window [mem 0x40400000-0x405fffff] This is a major surprise for users who are suddenly left with a PCIe device that was working fine with the original bridge window sizing. Even if the already assigned bridge window could be enlarged by reallocation in some cases (something the current code does not attempt to do), it is not possible in general case and the large amount of wasted space at the tail of the bridge window may lead to other resource exhaustion problems on Root Complex level (think of multiple PCIe cards with VFs and BAR size disparity in a single system). PCI specifications only expect natural alignment for BARs (PCI Express Base Specification, rev. 6.1 sect. 7.5.1.2.1) and minimum of 1MiB alignment for the bridge window (PCI Express Base Specification, rev 6.1 sect. 7.5.1.3). The current bridge window tail alignment rule was introduced in the commit 5d0a8965aea9 ("[PATCH] 2.5.14: New PCI allocation code (alpha, arm, parisc) [2/2]") that only states: "pbus_size_mem: core stuff; tested with randomly generated sets of resources". It does not explain the motivation for the extra tail space allocated that is not truly needed by the downstream resources. As such, it is far from clear if it ever has been required by any HW. To prevent PCIe cards with BAR size disparity from becoming unusable after remove & rescan cycle, attempt to do a truly minimal allocation for memory resources if needed. First check if the normally calculated bridge window will not fit into an already assigned upstream resource. In such case, try with relaxed bridge window tail sizing rules instead where no extra tail space is requested beyond what the downstream resources require. Only enforce the alignment requirement of the bridge window itself (normally 1MiB). With this patch, the resources are successfully allocated: pci 0000:02:01.0: PCI bridge to [bus 03] pci 0000:02:01.0: bridge window [mem 0x40400000-0x405fffff] pci 0000:02:01.0: bridge window [mem 0x6000000000-0x6400ffffff 64bit pref] pci 0000:02:01.0: bridge window [mem 0x6000000000-0x6400ffffff 64bit pref] = to [bus 03] requires relaxed alignment rules pci 0000:02:01.0: BAR 9: assigned [mem 0x6000000000-0x6400ffffff 64bit pref] pci 0000:02:01.0: BAR 8: assigned [mem 0x40400000-0x405fffff] pci 0000:03:00.0: BAR 2: assigned [mem 0x6000000000-0x63ffffffff 64bit pref] pci 0000:03:00.0: BAR 0: assigned [mem 0x6400000000-0x6400ffffff 64bit pref] pci 0000:03:00.0: BAR 6: assigned [mem 0x40400000-0x405fffff pref] pci 0000:02:01.0: PCI bridge to [bus 03] pci 0000:02:01.0: bridge window [mem 0x40400000-0x405fffff] pci 0000:02:01.0: bridge window [mem 0x6000000000-0x6400ffffff 64bit pref] This patch draws inspiration from the initial investigations and work by Mika Westerberg. Closes: https://bugzilla.kernel.org/show_bug.cgi?id=3D216795 Link: https://lore.kernel.org/linux-pci/20190812144144.2646-1-mika.westerbe= rg@linux.intel.com/ Fixes: 5d0a8965aea9 ("[PATCH] 2.5.14: New PCI allocation code (alpha, arm, = parisc) [2/2]") Signed-off-by: Ilpo J=C3=A4rvinen Cc: Mika Westerberg Reviewed-by: Mika Westerberg Tested-by: Lidong Wang =20 --- drivers/pci/setup-bus.c | 74 +++++++++++++++++++++++++++++++++++++++-- 1 file changed, 72 insertions(+), 2 deletions(-) diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index e3e6ff8854a7..7a32283262c2 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -21,6 +21,7 @@ #include #include #include +#include #include #include #include "pci.h" @@ -960,6 +961,62 @@ static inline resource_size_t calculate_mem_align(reso= urce_size_t *aligns, return min_align; } =20 +/** + * pbus_upstream_assigned_limit - Check no upstream resource limits alloca= tion + * @bus: The bus + * @mask: Mask the resource flag, then compare it with type + * @type: The type of resource from bridge + * @size: The size required from the bridge window + * @align: Required alignment for the resource + * + * Checks that @size can fit inside the upstream bridge resources that are + * already assigned. + * + * Return: -ENOSPC if @size cannot fit into an already assigned resource + * upstream resource. + */ +static int pbus_upstream_assigned_limit(struct pci_bus *bus, unsigned long= mask, + unsigned long type, resource_size_t size, + resource_size_t align) +{ + struct resource_constraint constraint =3D { + .max =3D RESOURCE_SIZE_MAX, + .align =3D align, + }; + struct pci_bus *downstream =3D bus; + struct resource *r; + + while ((bus =3D bus->parent)) { + if (pci_is_root_bus(bus)) + break; + + pci_bus_for_each_resource(bus, r) { + if (!r || !r->parent || (r->flags & mask) !=3D type) + continue; + + if (resource_size(r) >=3D size) { + struct resource gap =3D {}; + + if (!find_empty_resource_slot(r, &gap, size, &constraint)) + return 0; + } + + if (bus->self) { + pci_dbg(bus->self, + "Assigned bridge window %pR to %pR cannot fit 0x%llx required for %s = bridging to %pR\n", + r, &bus->busn_res, + (unsigned long long)size, + pci_name(downstream->self), + &downstream->busn_res); + } + + return -ENOSPC; + } + } + + return 0; +} + /** * pbus_size_mem() - Size the memory window of a given bus * @@ -986,7 +1043,7 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned= long mask, struct list_head *realloc_head) { struct pci_dev *dev; - resource_size_t min_align, align, size, size0, size1; + resource_size_t min_align, win_align, align, size, size0, size1; resource_size_t aligns[24]; /* Alignments from 1MB to 8TB */ int order, max_order; struct resource *b_res =3D find_bus_resource_of_type(bus, @@ -1064,10 +1121,23 @@ static int pbus_size_mem(struct pci_bus *bus, unsig= ned long mask, } } =20 + win_align =3D window_alignment(bus, b_res->flags); min_align =3D calculate_mem_align(aligns, max_order); - min_align =3D max(min_align, window_alignment(bus, b_res->flags)); + min_align =3D max(min_align, win_align); size0 =3D calculate_memsize(size, min_size, 0, 0, resource_size(b_res), m= in_align); add_align =3D max(min_align, add_align); + + if (bus->self && size0 && + pbus_upstream_assigned_limit(bus, mask | IORESOURCE_PREFETCH, type, + size0, add_align)) { + min_align =3D 1ULL << (max_order + 20); + min_align =3D max(min_align, win_align); + size0 =3D calculate_memsize(size, min_size, 0, 0, resource_size(b_res), = win_align); + add_align =3D win_align; + pci_info(bus->self, "bridge window %pR to %pR requires relaxed alignment= rules\n", + b_res, &bus->busn_res); + } + size1 =3D (!realloc_head || (realloc_head && !add_size && !children_add_s= ize)) ? size0 : calculate_memsize(size, min_size, add_size, children_add_size, resource_size(b_res), add_align); --=20 2.30.2