From nobody Tue Dec 16 16:38:54 2025 Received: from mail-oi1-f182.google.com (mail-oi1-f182.google.com [209.85.167.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A78142D12E7 for ; Thu, 11 Dec 2025 14:03:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765461824; cv=none; b=PGpuyF6R6GqMJABwhmMTNcsaiZAijodQ/WdgEA8oHCDU9F/X1tJW5vEyb/TLRSG4njdRIfNH9fcxHk2IdqS87bd1EnToLFp9UK2riXMy41zoP+1SO+IWn94FGV8GZlClOJEJ2sd624b0GJjRiQ3VVcJuDXH55hDa+wSEyuY+gvw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765461824; c=relaxed/simple; bh=dANt1IxVBpejBPX1GyE+hxJwTnKsLrEyXQVIM/d+pnQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=LCePz+Zsv5uFT9vghUV6ZFLZ3DOsw090mc4lF+L5FZmWc6z/LI+9ZcfgiTU9BVIgS8YwUpJg3ohvXRzqkvCDhe6WlDN44gJ3dgSiWqq0yZQZyFNnYcjnee1mxl+LBPnWFL/qul8x49P+erlyoGAFEgLCZ3c1sH6wPyRJgzCDiVA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=GEW9Vq9g; arc=none smtp.client-ip=209.85.167.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="GEW9Vq9g" Received: by mail-oi1-f182.google.com with SMTP id 5614622812f47-44fe903c1d6so31992b6e.0 for ; Thu, 11 Dec 2025 06:03:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1765461822; x=1766066622; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CLsfEOgWDExJ3WhLf4sg7dIsOWS7vuQUYkgDX6My13A=; b=GEW9Vq9gY4K2UcYxbwo2JfLFy/bCYafMvxJVmj9vuIsSIDaI99oD/wt12XJ+nH5SOK Cnmo/SzyEzYEmbhLH5A6sY+qeb3Y4VVtFXBrLh0VviS2siUlxfuejaBtyxv9Qv2YWmZT RftBdh/Kh269aJUY8ppWzcK5obOc8bTh6SxdU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765461822; x=1766066622; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=CLsfEOgWDExJ3WhLf4sg7dIsOWS7vuQUYkgDX6My13A=; b=gekPisgqE3cntJ+mZek+1hmn/Tu2RyMdNhxbqJQeSipiV7lL7Jom4unjJ74q6ExeZ8 K0PDE0qW58k3zOHeuC/8TU/5skvD/LXKZTZhiOah5G1qTvM0yDgtufkPMeKOOydwam2H D27Hjwz1EzPQgC2CVZQwu+Dhq34I1fsD6tEgUFG+ngKqdS1jx1CQR0T7X+zdgVjvVqJ5 Cz5nYuR+zX1J18sqa+mcddwDPxFNZOrjYFWQ47osacZPii631KtEIxlqrdupfZvQIZkg EIcg0n2vaLHwgBdg74pKbmA2nrYSq2SBa2uhRmD5m/ZfF3+kZCq+SXT2/XZ2t7qQdIIk CCIQ== X-Forwarded-Encrypted: i=1; AJvYcCVfIMONq3G5nnroyZcQVgsOM70ofeyWszc0H2qaECbnB4lhecpwPjWo2OUqg415Iq+x9pAkGi5IcP3tuRQ=@vger.kernel.org X-Gm-Message-State: AOJu0YwgQvli4R+IERAZSjr5YmFJT2rpHs6lZE5QRQ6QlKaYsDx7NITf +fVu+xrj6alI1hFO0064qx2WR6Ct7YBkGqSE/l9/avwgTm64tNCt8bOAcFJLL5FSYg== X-Gm-Gg: AY/fxX6+zMwGzSRieFJGYZqRU93FMCmKaBcdKXAwaTAmms+7N/hc7c9ACHFD6oleKIF 0RrDPoDfz5bcMGqZWBrbuOFWqsAwOgJjkqxXQ4fhBjAyFfpaQxdhyWTFZb6b9GeKYtjFsyaVWt9 V3sb2NEnx/tAu+g7N+F7zeJDbqX8Af8KkwnKS1T040n8/cZNUZuXnSuQ8+pH4yVc0KKL2YxBab8 fBisc5PCa/CG7tXE8fdCtWYmmU0xt6dfURumMcPEWBOICN2DHwIg0ZGley2SKPZxSpCFvMoI6iA 5D6s12bAlckl39OwFuw85h21FJz1Z5j7LuhmDkd1r70lt8l1ZDGswlB2PVMpePHIUcR3TI4yWKo dwlRaAzC01T6Vf+iBQfE+y6JWwyaibkj8mpx2BI6i+ihOnSrgb+xqRAPhhKObRB09V9e/yJIdZl HzaFkiVv9LFRpduZ4= X-Google-Smtp-Source: AGHT+IEa1BeEGmi4St54QeGtSB8Z4r5rsDPCPbjRjNJyjfsp44pJXGn7gep4nq/PLQSPhCGdL61Z+w== X-Received: by 2002:a05:6808:1710:b0:450:b781:3731 with SMTP id 5614622812f47-455863b7c8emr3319159b6e.26.1765461821361; Thu, 11 Dec 2025 06:03:41 -0800 (PST) Received: from chromium.org ([73.34.74.121]) by smtp.gmail.com with ESMTPSA id 5614622812f47-45598d6e789sm1130030b6e.22.2025.12.11.06.03.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Dec 2025 06:03:40 -0800 (PST) From: Simon Glass To: linux-arm-kernel@lists.infradead.org Cc: Nicolas Schier , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , =?UTF-8?q?J=20=2E=20Neusch=C3=A4fer?= , Masahiro Yamada , Tom Rini , Ahmad Fatoum , Chen-Yu Tsai , Simon Glass , Nicolas Schier , linux-kernel@vger.kernel.org Subject: [PATCH v7 1/6] scripts/make_fit: Speed up operation Date: Thu, 11 Dec 2025 07:02:58 -0700 Message-ID: <20251211140309.1910613-2-sjg@chromium.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251211140309.1910613-1-sjg@chromium.org> References: <20251211140309.1910613-1-sjg@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The kernel is likely at least 16MB so we may as well use that as a step size when reallocating space for the FIT in memory. Pack the FIT at the end, so there is no wasted space. This reduces the time to pack by an order of magnitude, or so. Signed-off-by: Simon Glass Reviewed-by: Nicolas Schier Reviewed-by: Ahmad Fatoum Tested-by: Chen-Yu Tsai --- (no changes since v3) Changes in v3: - Move the ramdisk chunk into the correct patch scripts/make_fit.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/scripts/make_fit.py b/scripts/make_fit.py index 1683e5ec6e67..0f5e7c4b8aed 100755 --- a/scripts/make_fit.py +++ b/scripts/make_fit.py @@ -98,7 +98,7 @@ def setup_fit(fsw, name): fsw (libfdt.FdtSw): Object to use for writing name (str): Name of kernel image """ - fsw.INC_SIZE =3D 65536 + fsw.INC_SIZE =3D 16 << 20 fsw.finish_reservemap() fsw.begin_node('') fsw.property_string('description', f'{name} with devicetree set') @@ -299,7 +299,9 @@ def build_fit(args): finish_fit(fsw, entries) =20 # Include the kernel itself in the returned file count - return fsw.as_fdt().as_bytearray(), seq + 1, size + fdt =3D fsw.as_fdt() + fdt.pack() + return fdt.as_bytearray(), seq + 1, size =20 =20 def run_make_fit(): --=20 2.43.0 From nobody Tue Dec 16 16:38:54 2025 Received: from mail-oi1-f181.google.com (mail-oi1-f181.google.com [209.85.167.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 865C82C15A8 for ; Thu, 11 Dec 2025 14:03:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765461827; cv=none; b=QqZPmv1UDV9HsDUovb3LDz+4ibaAKXdu3BdbChnAeFr0LMgOJ0KroOK0oqkN2Cfi59fzuQ1OK/sKSD64EgO+9gX4BXPDolpXiACifTq34Y6+aLHVj96F1byhj4YoamdxtBdXpDTMaS0927zklA1ZpWmE/UabqHMcERdcDI7XRZE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765461827; c=relaxed/simple; bh=JDHwg4UH314RvplNCRL91Oy7GIV8cCK8/8DcmuuMJd8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Ej8iJT8inPpJWml+Wz+kLu/1JbYZbCzgIX5XAxAo4aWmJn1dJUF/+1Lk/TBwt03kYGreG0mzT9zyt6U/OdvOTMTpXqyap2aQd5bK45n74FbyBQiq1x5TW+4HxMVzakqvTPVyu0mBmggHVj0eTGKBBnyEI4GZajcMSq7MsUXUhT0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=ZLu7tJM7; arc=none smtp.client-ip=209.85.167.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="ZLu7tJM7" Received: by mail-oi1-f181.google.com with SMTP id 5614622812f47-4503ee5c160so51600b6e.1 for ; Thu, 11 Dec 2025 06:03:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1765461824; x=1766066624; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=uf5OzsvzLJF7JMEHMJOMsmKpYuDb1ZnOHsvKTOsoYw0=; b=ZLu7tJM7NeakSp9xvG0cVTMYjCcB/AKRHwB00GMsnc1mkvgH0gXT24ysLzTi+IM7j6 lFp7wx1Pr1b8nn0vNhENkYylrqUd3JlVSJNgARAXGQoX82w4ps40U0wDn9lgzZq7Cge3 3J7nPRclaoTokteoyMAIouc4f2z2UqhfIEXVQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765461824; x=1766066624; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=uf5OzsvzLJF7JMEHMJOMsmKpYuDb1ZnOHsvKTOsoYw0=; b=vWuCHj84ukq41DbU2nnkW7gz2K8+sj72eWnpegzIISggDdpOvL1ND4hT/nf42hRgoE QubDqOpAGJVZLTDQNvn6NCvSANbu1/NGd1LmYqjb5TfnLGY3dQN5iL+8ew1BV1Og4Z8j eclq9C9NCf663DcgWnR+HdW2AFjbeKyFSsT3N7eAAZi0IIhHhVfB/S/8t0JGB1LDVfxi orncjlRGuejiercY16T2wRyGLbV5COE6aOu44j7i24f1Zzj+w6b0825V1M7TFALygVaw DyRkA6siOkJ78vi71H3Jk+ARNxA3QvdNjhCuR5imxRdUgxqJnxj8gKWLSctfeGMzYIXS cFaw== X-Forwarded-Encrypted: i=1; AJvYcCVJuKua+c+UQdrsJkBkP2p6l5YbqKlOCREnc7f53s+eyJvLefiCiWCxvMFhVmjzvUgTGZyTcOioWy3AiyA=@vger.kernel.org X-Gm-Message-State: AOJu0YzBW7FrvvLePNmb2NANd70nhHUzNR/0TkUQbEF2VBqMnACVytzQ 1drxjiu20sHTcDJt7m2XTBnAZldCnaSHa7G10gSSJvAvKIWV/LpwWwEVrRqiOlF6Fg== X-Gm-Gg: AY/fxX6A8G7NvYa+piogLxiIcE/NtI57rHIMa1eLoolqfaZt0Q8oCliW8MfokA4Krr5 jJHypFajp4h7yAg3mZgbogr0AyIcnEBOvJHRWpdcsZ9ewhzBE+vumU/+H4GeAFBVu4ZN6AL+P/s fAIawbb4CZYgh1EaF/3FAhUIsdCqx/wWB5UleUvSkJSIY0wIj3GsLKovR3gwS3+TUfU6SVfV0Rj aHmnKC7TFichCDboypGXr1aWyCt/TY38OELMv486/ygGZPlqCgdAURmkD4GY98TlibOpBc8Gh5M SQ02FA246yDqKueu1dJILhj/aTFu+T8zQwSg/2ACF1uRP5KZXBdQ/zt6y1fvLMilDTMKaZDaNaB f9DENqR0Zewdso8Be8dyb+EZtCfnpi13gGBXptwLysA0/w2fElC585+wjWTRAYnM6NvEWp+NQQR wey74V X-Google-Smtp-Source: AGHT+IGAhfe/8+QVBKrlzKF3W05U1ybOZGuxX3rXL5y843UymGBcnVJuY3dUEiFiCqu+eUImkPT9HA== X-Received: by 2002:a05:6808:1507:b0:450:d3af:ba4c with SMTP id 5614622812f47-45586723052mr3049979b6e.52.1765461823858; Thu, 11 Dec 2025 06:03:43 -0800 (PST) Received: from chromium.org ([73.34.74.121]) by smtp.gmail.com with ESMTPSA id 5614622812f47-45598d6e789sm1130030b6e.22.2025.12.11.06.03.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Dec 2025 06:03:43 -0800 (PST) From: Simon Glass To: linux-arm-kernel@lists.infradead.org Cc: Nicolas Schier , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , =?UTF-8?q?J=20=2E=20Neusch=C3=A4fer?= , Masahiro Yamada , Tom Rini , Ahmad Fatoum , Chen-Yu Tsai , Simon Glass , Nicolas Schier , linux-kernel@vger.kernel.org Subject: [PATCH v7 2/6] scripts/make_fit: Support an initial ramdisk Date: Thu, 11 Dec 2025 07:02:59 -0700 Message-ID: <20251211140309.1910613-3-sjg@chromium.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251211140309.1910613-1-sjg@chromium.org> References: <20251211140309.1910613-1-sjg@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" FIT (Flat Image Tree) allows a ramdisk to be included in each configuration. Add support for this to the script. This feature is not available via 'make image.fit' since the ramdisk likely needs to be built separately anyway, e.g. using modules from the kernel build. Future work may provide support for doing that. Note that the uncompressed size is not correct when a ramdisk is used, since it is too expensive to decompress the ramdisk. Signed-off-by: Simon Glass Reviewed-by: Nicolas Schier Reviewed-by: Ahmad Fatoum --- (no changes since v6) Changes in v6: - Drop the load address for ramdisk, as it is not needed - Bring back the ramdisk 'compression' property, set to 'none' Changes in v5: - Fix 'use' typo Changes in v4: - Update the commit message Changes in v3: - Add a comment at the top of the file about the -r option - Count the ramdisk in the total files - Update the commit message Changes in v2: - Don't compress the ramdisk as it is already compressed scripts/make_fit.py | 52 ++++++++++++++++++++++++++++++++++++++------- 1 file changed, 44 insertions(+), 8 deletions(-) diff --git a/scripts/make_fit.py b/scripts/make_fit.py index 0f5e7c4b8aed..66cd73711506 100755 --- a/scripts/make_fit.py +++ b/scripts/make_fit.py @@ -10,10 +10,14 @@ Usage: make_fit.py -A arm64 -n 'Linux-6.6' -O linux -o arch/arm64/boot/image.fit -k /tmp/kern/arch/arm64/boot/image.itk - @arch/arm64/boot/dts/dtbs-list -E -c gzip + -r /boot/initrd.img-6.14.0-27-generic @arch/arm64/boot/dts/dtbs-li= st + -E -c gzip =20 -Creates a FIT containing the supplied kernel and a set of devicetree files, -either specified individually or listed in a file (with an '@' prefix). +Creates a FIT containing the supplied kernel, an optional ramdisk, and a s= et of +devicetree files, either specified individually or listed in a file (with = an +'@' prefix). + +Use -r to specify an existing ramdisk/initrd file. =20 Use -E to generate an external FIT (where the data is placed after the FIT data structure). This allows parsing of the data without loading @@ -29,8 +33,6 @@ looks at the .cmd files produced by the kernel build. =20 The resulting FIT can be booted by bootloaders which support FIT, such as U-Boot, Linuxboot, Tianocore, etc. - -Note that this tool does not yet support adding a ramdisk / initrd. """ =20 import argparse @@ -81,6 +83,8 @@ def parse_args(): help=3D'Specifies the operating system') parser.add_argument('-k', '--kernel', type=3Dstr, required=3DTrue, help=3D'Specifies the (uncompressed) kernel input file (.itk)') + parser.add_argument('-r', '--ramdisk', type=3Dstr, + help=3D'Specifies the ramdisk/initrd input file') parser.add_argument('-v', '--verbose', action=3D'store_true', help=3D'Enable verbose output') parser.add_argument('dtbs', type=3Dstr, nargs=3D'*', @@ -133,7 +137,28 @@ def write_kernel(fsw, data, args): fsw.property_u32('entry', 0) =20 =20 -def finish_fit(fsw, entries): +def write_ramdisk(fsw, data, args): + """Write out the ramdisk image + + Writes a ramdisk node along with the required properties + + Args: + fsw (libfdt.FdtSw): Object to use for writing + data (bytes): Data to write (possibly compressed) + args (Namespace): Contains necessary strings: + arch: FIT architecture, e.g. 'arm64' + fit_os: Operating Systems, e.g. 'linux' + """ + with fsw.add_node('ramdisk'): + fsw.property_string('description', 'Ramdisk') + fsw.property_string('type', 'ramdisk') + fsw.property_string('arch', args.arch) + fsw.property_string('compression', 'none') + fsw.property_string('os', args.os) + fsw.property('data', data) + + +def finish_fit(fsw, entries, has_ramdisk=3DFalse): """Finish the FIT ready for use =20 Writes the /configurations node and subnodes @@ -143,6 +168,7 @@ def finish_fit(fsw, entries): entries (list of tuple): List of configurations: str: Description of model str: Compatible stringlist + has_ramdisk (bool): True if a ramdisk is included in the FIT """ fsw.end_node() seq =3D 0 @@ -154,6 +180,8 @@ def finish_fit(fsw, entries): fsw.property_string('description', model) fsw.property('fdt', bytes(''.join(f'fdt-{x}\x00' for x in = files), "ascii")) fsw.property_string('kernel', 'kernel') + if has_ramdisk: + fsw.property_string('ramdisk', 'ramdisk') fsw.end_node() =20 =20 @@ -274,6 +302,14 @@ def build_fit(args): size +=3D os.path.getsize(args.kernel) write_kernel(fsw, comp_data, args) =20 + # Handle the ramdisk if provided. Compression is not supported as it is + # already compressed. + if args.ramdisk: + with open(args.ramdisk, 'rb') as inf: + data =3D inf.read() + size +=3D len(data) + write_ramdisk(fsw, data, args) + for fname in args.dtbs: # Ignore non-DTB (*.dtb) files if os.path.splitext(fname)[1] !=3D '.dtb': @@ -296,12 +332,12 @@ def build_fit(args): =20 entries.append([model, compat, files_seq]) =20 - finish_fit(fsw, entries) + finish_fit(fsw, entries, bool(args.ramdisk)) =20 # Include the kernel itself in the returned file count fdt =3D fsw.as_fdt() fdt.pack() - return fdt.as_bytearray(), seq + 1, size + return fdt.as_bytearray(), seq + 1 + bool(args.ramdisk), size =20 =20 def run_make_fit(): --=20 2.43.0 From nobody Tue Dec 16 16:38:54 2025 Received: from mail-oi1-f176.google.com (mail-oi1-f176.google.com [209.85.167.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 476B72C324C for ; Thu, 11 Dec 2025 14:03:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765461828; cv=none; b=kho1oOWzLr3j4PNOAL/c9jj0zR7U1HxFiFiQhHDvHo+6vVApVGeLP3t9MBDZsO3+ssyvoEnJeSBikpN/Vk1a5HkxNoar3/vOT3Ak0+htV6XZAWT0JTm3lo3xI7aBc8lTLW9da+u2RKxnmbmJJcUalesEnbeMyydr0KlD+XXRKpA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765461828; c=relaxed/simple; bh=UqfoVLVav09I0aw2CGKUg7qsVn5uYMmSpEa3teQI+zA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=llpQXHrAkP0aIbKqV620Vkl9JehG/swGrc/Lndzqt8wpS3SEOYKgjCs3n55SrA+BS2+FwEyRM6KJrZBK3OYH2Mx+CZo2W26M6e2/iFbz5OzhvD0halLtDfAJXx69rB7h+ZG0TJsT8hk1mznFlQSC7MkUNZRcm28Kz94tbrtBS64= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=hRewFKgp; arc=none smtp.client-ip=209.85.167.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="hRewFKgp" Received: by mail-oi1-f176.google.com with SMTP id 5614622812f47-4557f0e5ed2so44093b6e.1 for ; Thu, 11 Dec 2025 06:03:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1765461826; x=1766066626; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=NGboLX11faY20GW7EG69bVQ+HimuuG86Fz7KsZ0Cjec=; b=hRewFKgpfmHk48yrSe2H/oVfe9/kp+gzNAClf1u16EhurQlwqxG8wi7ZhC7KTkavMK mVjeWF2m2/mu1IQ7tyUklQlPd77lderGgmto2uqwCDGy7LVEN/UQG1gqil4lDBZGlrda 7ipRcjPKClks/6pIK7T3DGr8mh0+pzIWfd6sM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765461826; x=1766066626; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=NGboLX11faY20GW7EG69bVQ+HimuuG86Fz7KsZ0Cjec=; b=n5EactsIk4yrKYkyekWn6HHIAVo6R+19Wtbo3flza2tExs2lm7iAXNo68NLH077dSx oX0SKKUDgdfGHt2U3030rFtF4VLjmVSdn7Vn4ZStqFMDPfzu3Gg6rUUhWqOL2KbhyPvt 07wvZZlFtPWDIJei4mOIF/77gNtpi5VooMW8FWtM5AECNHmeOsw5L2PuqVNhK0PNddy/ SX7AYGGgq7YPkfZzehpW43kmmA5/YVsEmtyziuApMBafREqduRAb1hk9dHDvoHytm5DI TmgNhziUMKFsm3GltacEltu8Pjtt1fBqzCt1QXMFqaBYqu1j4LjHuivcKvg62OI22uI/ W6hg== X-Forwarded-Encrypted: i=1; AJvYcCVfc9pjbM3VjEbNf3sOfdvr/mxZQvbTXuqFFE/7BbEZrlCCoAPyzns0Abp8/E6iNj6ej5NKN43EpiY74OM=@vger.kernel.org X-Gm-Message-State: AOJu0YySjTQ2KlayZr/JsRBQUnx6hMXVg5EyHtK9q6Dc1Z51cSFrrBeJ p+8NGMFmIGzJho49F/k5rh6WKblvIeJEhy7RiDS9NfhzIetMMrv9jMPyVqDapi4bPg== X-Gm-Gg: AY/fxX7ViY1YVlqTO61G0zPS/ZriH9zjgWxkYnquWwGjGYipFNqUi6vG79CoNrR3uEC 3HO/77QpwIPndJmjaTjkFZANYS/T4vTKYA6/HDr/vtlwFkt1YPlncCHl1CnHiiGHaSQV/M14EQS z5uTY8GttYW9/WfvIhHVNyNao9AV58+uJAUh3jfE1Z0CQqUSIlvg/BBkBYvSabXjS+pVFeWQ3PG 222zpDOh0irnLvzSo4dj8VXJa/XIKww39lLvmc4E1mi7wG3Ixy89e4b2Z8/1eF8RJ9Ox0q10ZWR geOEMgSlyvr4HdRz/E6OR/cKzytv+jFautvyzLmtZwgwuxA+II9XSVoI1542D7YDtJNaI81kaFo 47HHtEzNvfVIuotWYePNNy4HbZw6b3D0PIwn1EwpunlR8iP3Of33AfGlWL9J3v3us/0S1GAYtdy OPC3sEP+aHxdj3ugY= X-Google-Smtp-Source: AGHT+IFW4sXgXHRcnLBTV5SDzKKgVwFFYBppoyZ7iYax8w6vu6HYQtjGylKmpzENOm340PSo/tAJkg== X-Received: by 2002:a05:6808:222a:b0:450:89ee:922b with SMTP id 5614622812f47-4559bf3ef85mr1042177b6e.22.1765461826269; Thu, 11 Dec 2025 06:03:46 -0800 (PST) Received: from chromium.org ([73.34.74.121]) by smtp.gmail.com with ESMTPSA id 5614622812f47-45598d6e789sm1130030b6e.22.2025.12.11.06.03.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Dec 2025 06:03:44 -0800 (PST) From: Simon Glass To: linux-arm-kernel@lists.infradead.org Cc: Nicolas Schier , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , =?UTF-8?q?J=20=2E=20Neusch=C3=A4fer?= , Masahiro Yamada , Tom Rini , Ahmad Fatoum , Chen-Yu Tsai , Simon Glass , Nicolas Schier , linux-kernel@vger.kernel.org Subject: [PATCH v7 3/6] scripts/make_fit: Move dtb processing into a function Date: Thu, 11 Dec 2025 07:03:00 -0700 Message-ID: <20251211140309.1910613-4-sjg@chromium.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251211140309.1910613-1-sjg@chromium.org> References: <20251211140309.1910613-1-sjg@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Since build_fit() is getting quite long, move the dtb processing into a separate function. Change the double quotes in the write() call to single, to match the rest of the script. Signed-off-by: Simon Glass Reviewed-by: Nicolas Schier Reviewed-by: Ahmad Fatoum Reviewed-by: Chen-Yu Tsai --- Changes in v7: - Add a note about the " -> ' change scripts/make_fit.py | 67 +++++++++++++++++++++++++++++---------------- 1 file changed, 44 insertions(+), 23 deletions(-) diff --git a/scripts/make_fit.py b/scripts/make_fit.py index 66cd73711506..7ddaf3c0d783 100755 --- a/scripts/make_fit.py +++ b/scripts/make_fit.py @@ -277,6 +277,47 @@ def process_dtb(fname, args): =20 return (model, compat, files) =20 + +def _process_dtbs(args, fsw, entries, fdts): + """Process all DTB files and add them to the FIT + + Args: + args: Program arguments + fsw: FIT writer object + entries: List to append entries to + fdts: Dictionary of processed DTBs + + Returns: + tuple: + Number of files processed + Total size of files processed + """ + seq =3D 0 + size =3D 0 + for fname in args.dtbs: + # Ignore non-DTB (*.dtb) files + if os.path.splitext(fname)[1] !=3D '.dtb': + continue + + try: + (model, compat, files) =3D process_dtb(fname, args) + except Exception as e: + sys.stderr.write(f'Error processing {fname}:\n') + raise e + + for fn in files: + if fn not in fdts: + seq +=3D 1 + size +=3D os.path.getsize(fn) + output_dtb(fsw, seq, fn, args.arch, args.compress) + fdts[fn] =3D seq + + files_seq =3D [fdts[fn] for fn in files] + entries.append([model, compat, files_seq]) + + return seq, size + + def build_fit(args): """Build the FIT from the provided files and arguments =20 @@ -289,7 +330,6 @@ def build_fit(args): int: Number of configurations generated size: Total uncompressed size of data """ - seq =3D 0 size =3D 0 fsw =3D libfdt.FdtSw() setup_fit(fsw, args.name) @@ -310,34 +350,15 @@ def build_fit(args): size +=3D len(data) write_ramdisk(fsw, data, args) =20 - for fname in args.dtbs: - # Ignore non-DTB (*.dtb) files - if os.path.splitext(fname)[1] !=3D '.dtb': - continue - - try: - (model, compat, files) =3D process_dtb(fname, args) - except Exception as e: - sys.stderr.write(f"Error processing {fname}:\n") - raise e - - for fn in files: - if fn not in fdts: - seq +=3D 1 - size +=3D os.path.getsize(fn) - output_dtb(fsw, seq, fn, args.arch, args.compress) - fdts[fn] =3D seq - - files_seq =3D [fdts[fn] for fn in files] - - entries.append([model, compat, files_seq]) + count, fdt_size =3D _process_dtbs(args, fsw, entries, fdts) + size +=3D fdt_size =20 finish_fit(fsw, entries, bool(args.ramdisk)) =20 # Include the kernel itself in the returned file count fdt =3D fsw.as_fdt() fdt.pack() - return fdt.as_bytearray(), seq + 1 + bool(args.ramdisk), size + return fdt.as_bytearray(), count + 1 + bool(args.ramdisk), size =20 =20 def run_make_fit(): --=20 2.43.0 From nobody Tue Dec 16 16:38:54 2025 Received: from mail-oi1-f177.google.com (mail-oi1-f177.google.com [209.85.167.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3C04F2D0C62 for ; Thu, 11 Dec 2025 14:03:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765461830; cv=none; b=PZRnExPYhOuRGs1bWXA8It/83CVnkEQYqoOMjBzCs+xnxjxHc1rxLPR7B4ZAGIwQgYsHAggB0jNnp47tRbbHNUAzNx9JGNlBJ7EPI+qs9S2GJZKMaaEu3vIV0h1UUJwsRJmRYPKbscYUeRBypem670t7BBUxPwrqwp1+CNO6wko= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765461830; c=relaxed/simple; bh=h3wjZGG48fZ9pxWsr3LfkOz8cQcq5N+oV3tr1nZ5BOI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IjsARKuUVWpdvFueJOAP8vAvNRjiLSTu4eW0AVh5dZ3tnHcWBLvPWRhdUSOJiAzETPpRtjhUv2JIsxrFQ7hqdiSYExv1G02q+3tnbeIciZzqbs5YhblrJQaw54xdvOn+fONhvTapC2WxtvLaROj48ivfMTb6wmvkjmZqN0g7J04= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=dpNBIpQ2; arc=none smtp.client-ip=209.85.167.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="dpNBIpQ2" Received: by mail-oi1-f177.google.com with SMTP id 5614622812f47-450ac3ed719so83571b6e.0 for ; Thu, 11 Dec 2025 06:03:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1765461828; x=1766066628; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ocxfd2riU5l6RNdagWaPymuHyCDM3y1Y0L0w/bDOMGU=; b=dpNBIpQ2FgaS1DHIOTcCYfjxLJEOGEUbt0+JswT4dsswDFAwpB9MfpqBANbOBB/ZNJ QTELh/zHYSGbXeQLHTLx5DshX3n/eBY2Y6ADXJed2kjRe9zUY6CFaqE0EPY1z9NEh2nb 7yEgNUEU2wTz7Iar3wa/lxNmSczkg/I3n9hSo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765461828; x=1766066628; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=ocxfd2riU5l6RNdagWaPymuHyCDM3y1Y0L0w/bDOMGU=; b=nkNvZ1SRVI8pyqi346EwOdG9jh9Ns5WIlGA8mVlz+OtPvkwoCx7p+N8TsuqHUMpSIW d1cX9erR17QZQRh4Gh4Um7XkOkbdoG7ZGW6rup9jfhhf3mnRIdhqFyl+5XUaT8Aid5cW pIgAIZUosBbMjxdC9dqxgrIc7X9D+oBrDzgwPSfoTSrx7tLUSBdXEsPIVXDRCjNRfaqk wIl+3uHcWhx92ELX6j00bbEhGLzbZ5rB4vozjT+D854jSAC4Mz55dkcQrnlIGKFYpjod pVsXClT2hPxGR9w+pCHecUH2nYMDGU1e2UWteT0u97IeAtez15u6V5oY7r8QY4okmVwI dLuA== X-Forwarded-Encrypted: i=1; AJvYcCWTzbtZ4Wj4pQP0YpyDxHIA5NxY6GjijIQAMNcWn1U9m5jePlfuoSJ0NCrdP7ju800RKUxKPOoSFTB3YJw=@vger.kernel.org X-Gm-Message-State: AOJu0Yw+Dmlie20DR9fgUEfyluTno1qhtb5vMYxR8SW7FToe/9qQPp4N YoSVh9k0nIAG0zD/6hSuSJGd9cydo9Pb/bWRFyNUucmycm8aS2etiyLvlngNOkeOng== X-Gm-Gg: AY/fxX5fImN6XdMpss/R8xt3uUWBJXrGXPJM1Yp6Jv+bgWPiWfIIELuoGPDN8E9gkNO nw69QKyrqxSFnvJ3w5EhLsVfasrd6btQKmoDpucJTc4xvknxfocBvLFMlL3dS/ZuqSYrxUNDWOn gEyDJqPK4K4M3myXpbAepOnZUV+K1DK5J7UI5hULQPdwoLBFeXsNhTL8SacjpxrdlSlv95MByPI 8KtIdfHOvAXWzDlR09fBNPNXZcXtIva47sF/0bMhrS6fvFDZV4sJcd7rJaWE4jihecvePTwLGAe h7/V8YC0fOWGX5FNrPueFPd1nNKkmspNe2p3xx9wskFz3lojZiQZ7zyAi8QdeEDPz+bLhaAFcLH 25KXcqtbfcbdrheLUY/GDi1GejVqa38Oz/jQBDQgh9jT8XuTc4AcNqdRZ3vAzQyj2dc2fnxv3X/ 229ym+ X-Google-Smtp-Source: AGHT+IHtuzi/39tczpLCYlwV1m0CpQpVxWVqkNMlvivSc8jFvfBYvWlon4SeWF8dguVxgleSft/DYA== X-Received: by 2002:a05:6808:bc2:b0:44d:b8d3:3214 with SMTP id 5614622812f47-45586581c02mr2763469b6e.3.1765461828234; Thu, 11 Dec 2025 06:03:48 -0800 (PST) Received: from chromium.org ([73.34.74.121]) by smtp.gmail.com with ESMTPSA id 5614622812f47-45598d6e789sm1130030b6e.22.2025.12.11.06.03.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Dec 2025 06:03:47 -0800 (PST) From: Simon Glass To: linux-arm-kernel@lists.infradead.org Cc: Nicolas Schier , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , =?UTF-8?q?J=20=2E=20Neusch=C3=A4fer?= , Masahiro Yamada , Tom Rini , Ahmad Fatoum , Chen-Yu Tsai , Simon Glass , Alice Ryhl , Ard Biesheuvel , Josh Poimboeuf , Kees Cook , Nathan Chancellor , Nicolas Schier , Oleh Zadorozhnyi , Petr Mladek , linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v7 4/6] kbuild: Support a MAKE_FIT_FLAGS environment variable Date: Thu, 11 Dec 2025 07:03:01 -0700 Message-ID: <20251211140309.1910613-5-sjg@chromium.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251211140309.1910613-1-sjg@chromium.org> References: <20251211140309.1910613-1-sjg@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In some cases it is useful to be able to pass additional flags to the make_fit.py script. For example, since ramdisks are typically large, passing -E to use external data can be helpful. Add a new MAKE_FIT_FLAGS variable for this. Signed-off-by: Simon Glass --- Changes in v7: - Add a new patch with the MAKE_FIT_FLAGS change scripts/Makefile.lib | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib index 28a1c08e3b22..a5641a6538b1 100644 --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -400,7 +400,7 @@ FIT_COMPRESSION ?=3D gzip =20 quiet_cmd_fit =3D FIT $@ cmd_fit =3D $(MAKE_FIT) -o $@ --arch $(UIMAGE_ARCH) --os linux \ - --name '$(UIMAGE_NAME)' \ + --name '$(UIMAGE_NAME)' $(MAKE_FIT_FLAGS) \ $(if $(findstring 1,$(KBUILD_VERBOSE)),-v) \ $(if $(FIT_DECOMPOSE_DTBS),--decompose-dtbs) \ --compress $(FIT_COMPRESSION) -k $< @$(word 2,$^) --=20 2.43.0 From nobody Tue Dec 16 16:38:54 2025 Received: from mail-oi1-f182.google.com (mail-oi1-f182.google.com [209.85.167.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 514112D0603 for ; Thu, 11 Dec 2025 14:03:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765461834; cv=none; b=cznYxgQarbxs4wVDSeLUgt52ZvO06KrNLmyrC9/LuKON8WOVy55L/XJYw4RNvaGeILdnPpC0n244trBjJSWtkNUquxnAjdE5gWTHo6JYeyfQ8AswooXaS8yOJ0a/fleyxLMUzt1oldunLJpFYVH4lXc6S3RfIewVfOAleu5eY/g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765461834; c=relaxed/simple; bh=KD2qhSXDFwAeANzTRbX5+dmS3r4HqQiCInVKrJHu/3U=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dfJRJpATyvzroKH2sSXgn+74JeKM2/aK9w8TF+kTa26tqpyTm6Ut1tEyMBUsnK0duF13z8bScHIsIfRoaesoOdAXvRdaFFfH1NTMjSdMnmPtT4leqCUU0WAJVV76WROmqn6PT8cD7MyRjzrSkwCadvbQm5HXv702/oKzx3iAV/I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=Pq4qCSaB; arc=none smtp.client-ip=209.85.167.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="Pq4qCSaB" Received: by mail-oi1-f182.google.com with SMTP id 5614622812f47-45090ef26c6so36094b6e.2 for ; Thu, 11 Dec 2025 06:03:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1765461831; x=1766066631; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=aipOaJuScwfExD6+bFm/ZBqxUEz6heXJaJqy4EsM7Tg=; b=Pq4qCSaBriTu48W4twwtbRGfeYviXaCmES7iND+sEvdb6nxFTVpkxrvDVniunwc9SB ly1KthMifJp5hJ7lmamvEWzq5vwwAxatuMw8GM+5BqNPOxYL4RQl0o1YnTf54apFdaYX Vcuw+vt2fvGvkiGMqImX1pZe12ZtZphgnFOqY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765461831; x=1766066631; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=aipOaJuScwfExD6+bFm/ZBqxUEz6heXJaJqy4EsM7Tg=; b=wUFrmXJ8qxHNAHngK6Bjk5bO6Y3dpArEveL8slK765Vd0qMi9/qQUP9vy7sjrrqHLP pgLEDsoFZMRhOpjhIOe30/F9SdkWjK4HamFYgGWFe1qEYgV9hDH5aO3POzgQ++J9NFYp y/nzAmxHnr4ie/zIbceAXq4Ktb/XXXbx7d+aGMCIePlTA9TB2KBizOp9WkEX/nJ1FjOM E6sKXjVzg+tq/pQwB4PMepSS3VwVMbNJNh+eGNBxBf5KkH4GB7hAbKPpYFQzvxBTregQ ud4Xo7jlCQj77vBrp/s7B4D9ES/NOYqS4uVYH6snRbr7YB7F5PlW+AW3gKLdWaUTeTux P7+Q== X-Forwarded-Encrypted: i=1; AJvYcCXiJ46u8q+rnrKs5QVWSZUZHBndEtTrOFb7epvNt54eWaOmWXFaZ6W6yTErD/k3dsC1Av5PUrFitmV+TpE=@vger.kernel.org X-Gm-Message-State: AOJu0YwzRpg926/CVbkpZVyPrb6sByoixj+uvqBUeS5G9TFojV7YoryM 3ysqCpBr9nmwVUlO05P55yYfS1//eg0ElKHiWKo0NqoMJuPAYgfmKzKF6eQRZreSLQ== X-Gm-Gg: AY/fxX4ZEhaxNx1VgbTlp3v2jymWu5MUnWQssR+8MX/cFj3N99EzNsQSjXxiiAwhYCR TZ26brUJ/SFVTFIDwd3PNaH81ck/fULAHHRgJDBVpamPMmHXCB4kLVzU2yrB7h4f3Mh2S4Ta5VI JYe9sWuV3UhC8EbBdz19O11HhU84LMRMx1RksZ02NTsWYQwhiVZwJunWWlvj81OJ39Br6uQbwy5 hNKDC+gTbXyLBpuAVd0voWrnr1mAymId/+oF2ufROazDAIsQsy/daitLhauUgz2XJyg61Fmc7bl bOBVi6ibMOWBonKD3C4fh1o5rMMXdRpYd7uVpl+P10ebfJE8IRutoIwk4GRaWry6/qqu3XcgXMq WzN3yO6kCakhpsWQWljJWPHwPUSANn8j2XjSxD6yzrvSqFqjtkCkgTiy+3KigV/kJe73KClERut 8WvfolSSYj5CvQvCQ= X-Google-Smtp-Source: AGHT+IHC3573QohLmv8SOACLNjH9ajppgT1cVKZJbAnApK3+sKvHBpgpk8c0AWII7l6JWhGkYhKPCQ== X-Received: by 2002:a05:6808:894c:b0:453:10f0:a345 with SMTP id 5614622812f47-455863e1496mr2740090b6e.30.1765461830659; Thu, 11 Dec 2025 06:03:50 -0800 (PST) Received: from chromium.org ([73.34.74.121]) by smtp.gmail.com with ESMTPSA id 5614622812f47-45598d6e789sm1130030b6e.22.2025.12.11.06.03.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Dec 2025 06:03:50 -0800 (PST) From: Simon Glass To: linux-arm-kernel@lists.infradead.org Cc: Nicolas Schier , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , =?UTF-8?q?J=20=2E=20Neusch=C3=A4fer?= , Masahiro Yamada , Tom Rini , Ahmad Fatoum , Chen-Yu Tsai , Simon Glass , David Sterba , Nick Terrell , linux-kernel@vger.kernel.org Subject: [PATCH v7 5/6] scripts/make_fit: Support a few more parallel compressors Date: Thu, 11 Dec 2025 07:03:02 -0700 Message-ID: <20251211140309.1910613-6-sjg@chromium.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251211140309.1910613-1-sjg@chromium.org> References: <20251211140309.1910613-1-sjg@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add support for pbzip2, xz and plzip which can compress in parallel. This speeds up the ramdisk compression. Signed-off-by: Simon Glass --- Changes in v7: - Mention xz as well, in the commit message scripts/make_fit.py | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/scripts/make_fit.py b/scripts/make_fit.py index 7ddaf3c0d783..defb87f223d7 100755 --- a/scripts/make_fit.py +++ b/scripts/make_fit.py @@ -50,11 +50,12 @@ import libfdt CompTool =3D collections.namedtuple('CompTool', 'ext,tools') =20 COMP_TOOLS =3D { - 'bzip2': CompTool('.bz2', 'bzip2'), + 'bzip2': CompTool('.bz2', 'pbzip2,bzip2'), 'gzip': CompTool('.gz', 'pigz,gzip'), 'lz4': CompTool('.lz4', 'lz4'), - 'lzma': CompTool('.lzma', 'lzma'), + 'lzma': CompTool('.lzma', 'plzip,lzma'), 'lzo': CompTool('.lzo', 'lzop'), + 'xz': CompTool('.xz', 'xz'), 'zstd': CompTool('.zstd', 'zstd'), } =20 @@ -207,7 +208,12 @@ def compress_data(inf, compress): done =3D False for tool in comp.tools.split(','): try: - subprocess.call([tool, '-c'], stdin=3Dinf, stdout=3Dou= tf) + # Add parallel flags for tools that support them + cmd =3D [tool] + if tool in ('zstd', 'xz'): + cmd.extend(['-T0']) # Use all available cores + cmd.append('-c') + subprocess.call(cmd, stdin=3Dinf, stdout=3Doutf) done =3D True break except FileNotFoundError: --=20 2.43.0 From nobody Tue Dec 16 16:38:54 2025 Received: from mail-ot1-f44.google.com (mail-ot1-f44.google.com [209.85.210.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2E6752D0C62 for ; Thu, 11 Dec 2025 14:03:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765461836; cv=none; b=BHZMOoVC0lMbyeG5a+aVgaEFpQgnebWFe6IC7lmeDu9/GDLoS/TrRSSO94nW1tSi1yDJfqQHJPvlBgsEWXwgwN1eAQeA1UFl9wLk3ypaY3gAgk0dF1HUiWcLFA5G+eNWQmTDcNfT3PoGHEgAYWfC4pFxgGGXeXgUsG/i9nMQ+Qk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765461836; c=relaxed/simple; bh=qFAoyq72kc1ep7bQhDi6ZfBb7dgniIZQAAK+V+ilCKA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=geRILAjzBDHhYPL7TIvQAIM1N5976rmK74mib6IyLYrh40oAl1hBlQnvRmHI+Y/Y071aH79xM6kMpIM+E/V56JFzLRyMwnD18IcGzvykO4EjExVG+npJuyyLhC11SX/uWYAr7hS5alnboLN/RW42CSTUGSdL0glAtgc4/TpMrVA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=Wz0YYk65; arc=none smtp.client-ip=209.85.210.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="Wz0YYk65" Received: by mail-ot1-f44.google.com with SMTP id 46e09a7af769-7c6d13986f8so109868a34.0 for ; Thu, 11 Dec 2025 06:03:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1765461834; x=1766066634; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YrEZrghrc2EX7rimqehQ8Km0GewP4FP/AwInEozlUrg=; b=Wz0YYk65B0tuvrtHUK2wf3/EBvDut8fnTfM/9T6g/1AnXQEinJBvrNA5E6OjlS1YY5 HNjMxnKXYcpgKvJSEokpXbz8EiqIgVMLh1taFOIpQOGuSMTlB0erVWMLPjtJYV8SwrCP Gzer7UCDycAtgKFlONhqmmEE87BzQNEc/wzsg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765461834; x=1766066634; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=YrEZrghrc2EX7rimqehQ8Km0GewP4FP/AwInEozlUrg=; b=dUbtzOTrwXUDSoAwSE7tZFR8a7Tj2pNwCsOxyqxzD6cmrhMnf2FcNzBjgaIyHF45e9 M+fIwGlyEaNCpajnb6t3HjvdztPvHINWHw6ap2e/sLE8pmTH041mp1kIsyfhuz0CmGKu J2EEdKDnR1kTU5qq/ayjhjdtGMeD+G9DI5p2SQmCZQJ2W4mbhnd05qb4N23utSptFG70 yx2tezdfErSbKmIjnQfbPQVz0SdVBOxPLMBQ2SADTVGoOOZ7qn5Tc6cd6W4NYQqW/k5q 4UiLF4JPlpptDHR15u6JX9fwJbcp3/zKnMqc3TE58MupIGST2i7PS5VQK+BXpPj0teV4 fTwg== X-Forwarded-Encrypted: i=1; AJvYcCXEtutgq9RkTeK1LrbwWYwGQFRbZZZbH3vy7fJT92oV+EoDLzXR8kH88DipYepXwZKDt5MOyeODj5n9NZU=@vger.kernel.org X-Gm-Message-State: AOJu0Yz4ccgJTxKKsIWQ5pInq86OF6c1r5KURJYoXZcY0nHAPXGApVWL OVGoO+NOT1cEiopXyAucukwsIGtTWj6Zukzsfwh6/izr+V6qrcKL8i41j8yLoourzDkN0dMv8IT zKbg= X-Gm-Gg: AY/fxX6vchHb0AFWXXwK9kxnBuheTbz7Y7XQj9t2lacJMTlAYtBzUtGH3xf+VkGEdBJ MA/nancn+bBAHUsnJG03WZcEKJRG1gE1+RoWlpCJedvjOZxNfO7qTNEHtjH1GOlDXSu/QdsMyvB WslQyIraZEydgicVKpkNA65fNyogapGI2xcIeZ5iYse/odcRfUCHbMZgn6IAKHVPk7BSgD81Q+X 4ajN8kg55PArzs5GdzEG+lQ7q9rSfLpPV1pPKB3/YUAGW8Wco98rwZy5khIm91yGw4eTmYBFMte GUGIkAX1dgWZ21R2k4u0AQXW2OdFFahrmndtr8OrA/Vklob7rE6ZTM65wrgj6xw+DL/wcwBwmQ6 pTg7Nknf0zE7jb6vBu+oDqXdGHY9BM1ClkVGx/GcZTWzmdgt0WamULgElFP3g92VFciUAxgdadM /XxEVZ X-Google-Smtp-Source: AGHT+IGdhTYXa00yC9lHlR47tVgHZ0qHU5gLHDx5CEQuAyMBzVTh1Mkta8Jby7+utQw5FLyd4TFLDg== X-Received: by 2002:a05:6808:c2bf:b0:43f:68b:acb3 with SMTP id 5614622812f47-4559befdfa3mr681670b6e.17.1765461833902; Thu, 11 Dec 2025 06:03:53 -0800 (PST) Received: from chromium.org ([73.34.74.121]) by smtp.gmail.com with ESMTPSA id 5614622812f47-45598d6e789sm1130030b6e.22.2025.12.11.06.03.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Dec 2025 06:03:52 -0800 (PST) From: Simon Glass To: linux-arm-kernel@lists.infradead.org Cc: Nicolas Schier , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , =?UTF-8?q?J=20=2E=20Neusch=C3=A4fer?= , Masahiro Yamada , Tom Rini , Ahmad Fatoum , Chen-Yu Tsai , Simon Glass , linux-kernel@vger.kernel.org Subject: [PATCH v7 6/6] scripts/make_fit: Compress dtbs in parallel Date: Thu, 11 Dec 2025 07:03:03 -0700 Message-ID: <20251211140309.1910613-7-sjg@chromium.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251211140309.1910613-1-sjg@chromium.org> References: <20251211140309.1910613-1-sjg@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When there are 1500 device tree files it takes quite a while to compress them. Do it in parallel. kbuild: Allow adding modules into the FIT ramdisk Signed-off-by: Simon Glass --- (no changes since v1) scripts/make_fit.py | 56 ++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 50 insertions(+), 6 deletions(-) diff --git a/scripts/make_fit.py b/scripts/make_fit.py index defb87f223d7..e923cc8b05b7 100755 --- a/scripts/make_fit.py +++ b/scripts/make_fit.py @@ -37,6 +37,7 @@ as U-Boot, Linuxboot, Tianocore, etc. =20 import argparse import collections +import multiprocessing import os import subprocess import sys @@ -225,15 +226,31 @@ def compress_data(inf, compress): return comp_data =20 =20 -def output_dtb(fsw, seq, fname, arch, compress): +def compress_dtb(fname, compress): + """Compress a single DTB file + + Args: + fname (str): Filename containing the DTB + compress (str): Compression algorithm, e.g. 'gzip' + + Returns: + tuple: (str: fname, bytes: compressed_data) + """ + with open(fname, 'rb') as inf: + compressed =3D compress_data(inf, compress) + return fname, compressed + + +def output_dtb(fsw, seq, fname, arch, compress, data=3DNone): """Write out a single devicetree to the FIT =20 Args: fsw (libfdt.FdtSw): Object to use for writing seq (int): Sequence number (1 for first) fname (str): Filename containing the DTB - arch: FIT architecture, e.g. 'arm64' + arch (str): FIT architecture, e.g. 'arm64' compress (str): Compressed algorithm, e.g. 'gzip' + data (bytes): Pre-compressed data (optional) """ with fsw.add_node(f'fdt-{seq}'): fsw.property_string('description', os.path.basename(fname)) @@ -241,9 +258,10 @@ def output_dtb(fsw, seq, fname, arch, compress): fsw.property_string('arch', arch) fsw.property_string('compression', compress) =20 - with open(fname, 'rb') as inf: - compressed =3D compress_data(inf, compress) - fsw.property('data', compressed) + if data is None: + with open(fname, 'rb') as inf: + data =3D compress_data(inf, compress) + fsw.property('data', data) =20 =20 def process_dtb(fname, args): @@ -300,6 +318,11 @@ def _process_dtbs(args, fsw, entries, fdts): """ seq =3D 0 size =3D 0 + + # First figure out the unique DTB files that need compression + todo =3D [] + file_info =3D [] # List of (fname, model, compat, files) tuples + for fname in args.dtbs: # Ignore non-DTB (*.dtb) files if os.path.splitext(fname)[1] !=3D '.dtb': @@ -311,11 +334,32 @@ def _process_dtbs(args, fsw, entries, fdts): sys.stderr.write(f'Error processing {fname}:\n') raise e =20 + file_info.append((fname, model, compat, files)) + for fn in files: + if fn not in fdts and fn not in todo: + todo.append(fn) + + # Compress all DTBs in parallel + cache =3D {} + if todo and args.compress !=3D 'none': + if args.verbose: + print(f'Compressing {len(todo)} DTBs...') + + with multiprocessing.Pool() as pool: + compress_args =3D [(fn, args.compress) for fn in todo] + # unpacks each tuple, calls compress_dtb(fn, compress) in para= llel + results =3D pool.starmap(compress_dtb, compress_args) + + cache =3D dict(results) + + # Now write all DTBs to the FIT using pre-compressed data + for fname, model, compat, files in file_info: for fn in files: if fn not in fdts: seq +=3D 1 size +=3D os.path.getsize(fn) - output_dtb(fsw, seq, fn, args.arch, args.compress) + output_dtb(fsw, seq, fn, args.arch, args.compress, + cache.get(fn)) fdts[fn] =3D seq =20 files_seq =3D [fdts[fn] for fn in files] --=20 2.43.0