From nobody Sun Feb 8 18:18:48 2026 Received: from mail-ot1-f46.google.com (mail-ot1-f46.google.com [209.85.210.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8A54E31A553 for ; Sat, 27 Dec 2025 14:22:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.46 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1766845340; cv=none; b=mzhoWpEQvlAOBOGNnKl+K8ZueT1P88nWraGT7vAMkZcPpf14+ZyOuaT6EzrhcD803IEGIgYeVptiTOIfw3EcKfW30jEXJzeT+vOCwfT+2j5im+WVqTkdvrTLV7Q+RmT/r0XWH6aGkdm1sUvP1zXl1lflFJwgCtesQclaVZI6uZc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1766845340; c=relaxed/simple; bh=dANt1IxVBpejBPX1GyE+hxJwTnKsLrEyXQVIM/d+pnQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=EGVwjXseFj4JwkSQ49XGvAzH/I3M1zOecOldG6TyoBHnI9GonIiywYp2Mje4pC/FoD86gnpYcMKyX5Ezn83qo1UT1Bp+08miAd483UWsNJKCAAewqZjy7WMqGsYjc2MXEdslxszrd7AAgIGF+ciXCeKkEBbNnTXq9Oio3MMhdk8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=cc+uzldL; arc=none smtp.client-ip=209.85.210.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="cc+uzldL" Received: by mail-ot1-f46.google.com with SMTP id 46e09a7af769-7c7545310b8so3928184a34.1 for ; Sat, 27 Dec 2025 06:22:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1766845337; x=1767450137; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CLsfEOgWDExJ3WhLf4sg7dIsOWS7vuQUYkgDX6My13A=; b=cc+uzldLhh/cYO86wQVb/K6jVUffLoleSeDvZXrLrDuDPFUqm1Ivxt4DZ389F77eGX HsKcJ2zesEB9TbuQwtwyVpa45CH9eFRgrI5dKembkrRRCYwV58+M6wDZdYuk6cL3nJmu tegZuBRw6qDpZgT4N7JPK13fwKjFvpJfcLQwE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1766845337; x=1767450137; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=CLsfEOgWDExJ3WhLf4sg7dIsOWS7vuQUYkgDX6My13A=; b=UOg3oXOgTNifwVqKizVo2Cvkp03Bw1ohEhNeI89gMu+qhbyGS9ul0LzcddFTGGv5Xx svgTAlWA+i37ersVivhdzB6X3kGCeNSoljvItmKv36a7+TzhRiDUmx2RBH3diNXYZH1b EYFg9we5U6bgcM/LFcQMwDZSIRgpIp8VdCt6UG59e0hBaIwXt5fIvL/Hbtb1c9xTwZKT v1mHXHEtqgUQ8KCyEPjjp6x7++RC5V+6+oaIo7Z3ga35vTfuhLTrmZDJz8xl9Pd6v8aC WYJJN8ewHySpGoThwinFvZ7F55KXESQBC7wrpLAZOWVwmVqb9CSJFdVadjV3p4198NIW X3Gw== X-Forwarded-Encrypted: i=1; AJvYcCUvl8GqhcpNxFaDmdHmlDfcvFggZ96gGVX5jhkt4/NMuea5g57gf2WCFKRs3kBe3RvJd+wAu4BQmjmEXQM=@vger.kernel.org X-Gm-Message-State: AOJu0YwO+83LCTX56HQbf+n1oGcOy2MxgAwMKS7lE/j3szAIjkK/bH7t cu/pJ3I2pxZJLYX120YP1IEL2DQNwu4OVfhOku7g6yjCL14jYdUPF5ZtKRPYisRzBw== X-Gm-Gg: AY/fxX6iwt58vvWQcnpGKMAsW7gVoxnlWj1XmBNe+LPBVfoN7SZi7vL5cKThnYM45F9 9EIl9pcrCyCd4IHzPzt3wr3d7f7IvDXq1w+BXsyExy7dRDJQu2KsOl4rUGLvi6KgJOJkOwFfJbL aCQUvzbK2AUPQo4c15CHU5NklvJVVnJ2iX7GhygE/fhBQB/eVOxTRjd3WBMVs/0BETtq8I11sBR m3/sJr7LOJxXX+N/xrJzy+hux7mv5E6mu3dEciFLxsqR8TJ3XN4X+sl3JGYKAkYHPVlTvSwZY/d y7kw3N/yjIQjm6JLUwmJhVafn7ACux3xA2a1x6ojBs1KqcmEz4lmJ05W9EjEuzILDCRqrXRIHI7 dtPugpxgVlffSOy0jA53Chvms787AoJVNeuqwWA/sTC/VJFaGz4khQxU0XK6maXVIAcedOoZ7aL 3Hb/OE X-Google-Smtp-Source: AGHT+IERloq1wbu6MMPNkQSZUoJASFP8MfNfOubJP/R1dV7qustFVakA5w7Kz2wH7QU+n8t6pVo8uw== X-Received: by 2002:a05:6830:4c0e:b0:7c7:827f:872f with SMTP id 46e09a7af769-7cc66a5a9d7mr15858241a34.37.1766845337511; Sat, 27 Dec 2025 06:22:17 -0800 (PST) Received: from chromium.org ([73.34.74.121]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-7cc66727e11sm17352660a34.3.2025.12.27.06.22.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 27 Dec 2025 06:22:15 -0800 (PST) From: Simon Glass To: linux-arm-kernel@lists.infradead.org Cc: Masahiro Yamada , Chen-Yu Tsai , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , Nicolas Schier , =?UTF-8?q?J=20=2E=20Neusch=C3=A4fer?= , Ahmad Fatoum , Tom Rini , Simon Glass , Nicolas Schier , linux-kernel@vger.kernel.org Subject: [PATCH v8 1/6] scripts/make_fit: Speed up operation Date: Sat, 27 Dec 2025 07:21:45 -0700 Message-ID: <20251227142200.2241198-2-sjg@chromium.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251227142200.2241198-1-sjg@chromium.org> References: <20251227142200.2241198-1-sjg@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The kernel is likely at least 16MB so we may as well use that as a step size when reallocating space for the FIT in memory. Pack the FIT at the end, so there is no wasted space. This reduces the time to pack by an order of magnitude, or so. Signed-off-by: Simon Glass Reviewed-by: Nicolas Schier Reviewed-by: Ahmad Fatoum Tested-by: Chen-Yu Tsai --- (no changes since v3) Changes in v3: - Move the ramdisk chunk into the correct patch scripts/make_fit.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/scripts/make_fit.py b/scripts/make_fit.py index 1683e5ec6e67..0f5e7c4b8aed 100755 --- a/scripts/make_fit.py +++ b/scripts/make_fit.py @@ -98,7 +98,7 @@ def setup_fit(fsw, name): fsw (libfdt.FdtSw): Object to use for writing name (str): Name of kernel image """ - fsw.INC_SIZE =3D 65536 + fsw.INC_SIZE =3D 16 << 20 fsw.finish_reservemap() fsw.begin_node('') fsw.property_string('description', f'{name} with devicetree set') @@ -299,7 +299,9 @@ def build_fit(args): finish_fit(fsw, entries) =20 # Include the kernel itself in the returned file count - return fsw.as_fdt().as_bytearray(), seq + 1, size + fdt =3D fsw.as_fdt() + fdt.pack() + return fdt.as_bytearray(), seq + 1, size =20 =20 def run_make_fit(): --=20 2.43.0 From nobody Sun Feb 8 18:18:48 2026 Received: from mail-ot1-f54.google.com (mail-ot1-f54.google.com [209.85.210.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0498931B807 for ; Sat, 27 Dec 2025 14:22:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1766845342; cv=none; b=hpJSajC/omcgy26oniquyrlZWesYSZU8Z+UpjNN/UeOJBOHz5LXzfdppf67QEsRc9DZkFiw4vfD2GljlHSvtStnWUoq2jYJwD8rXe6ecIyZPPYLONNm2L4dKClOv+jzVWUgOlTZcaSbzFCcxtO3Y81RzG/V8usoKWFss7iBBgkg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1766845342; c=relaxed/simple; bh=JDHwg4UH314RvplNCRL91Oy7GIV8cCK8/8DcmuuMJd8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qPdbnY6dQEHEexA+JbgPDSnPR88DhgR3BFug4qV5UR1EKZ4Eqf9QbNdoh4gKMGiXmTKpp1il1UDrskeQXNWA89Q+Tm38SJaSqFTtmZTJJneW/z04XF+xiQbgvveQbEwZnBeS4I9BAexX9PkWNGbENsjc71STkKBgKFYvFTDPfMQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=N2vTFX3U; arc=none smtp.client-ip=209.85.210.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="N2vTFX3U" Received: by mail-ot1-f54.google.com with SMTP id 46e09a7af769-7cac9225485so6274495a34.3 for ; Sat, 27 Dec 2025 06:22:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1766845340; x=1767450140; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=uf5OzsvzLJF7JMEHMJOMsmKpYuDb1ZnOHsvKTOsoYw0=; b=N2vTFX3U6GEngaNMgNJYlPfoIKsY7ai6KUa4Jawf2gYD7vcCcOKsCZvfE48mkGyIZ3 UsPxVr5O69pDAmmJv95Q3ZuQsp6Z8vUQzv0yg3D8Nk5aVECv8N1Fcko9ZpuFysklpRJp rMkDjvHWtY0bYqsy6vXKjjElGL/ur43DhdUwQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1766845340; x=1767450140; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=uf5OzsvzLJF7JMEHMJOMsmKpYuDb1ZnOHsvKTOsoYw0=; b=bhbCuhovfdJzYkHBk/wxvike9KFvpBKCI3y56JgwFj/IMT4XEbfp/+R3RsKPk56N6L J0AkUf11Ld/3KvYXXdScIK1nWUByhdG6lcdSDvXTpRQ9GD9HYCH5RYN3/Ff7bl9wYi45 tDI3gBd7TbSlz/xDuDOg18FbDKcR0LPnqJkiB8eNLOHoRi3Rr2ASnPAYXAEoCe5P5Hhg CgKr9pu6TA2X+R3McbpR1DuYDbIZDzHke1y/TegJd4Wnb90oUw5B83PqBU1vmz1AiYXd hYyEk/5G91vi6nwLOhBwWcHxI9/qWt5I/LLFw+p7Xo735idV3cWOOSeLpewk6FJbhtC/ HbKQ== X-Forwarded-Encrypted: i=1; AJvYcCXl9K8XKsDvBh7IwHcQfHvAGr94V40PZsAkRhng+fKTlUjYOvbQJZIC9S8Y+axrp9EMNGNmR927jaIwbR8=@vger.kernel.org X-Gm-Message-State: AOJu0YwxR8Zlg01uidFNOoms2uyUmQB+776eXxgGnZOe3zxH/BGvMlCt 3VEGSgUNr0XW+kSs0Dfm3bnNb8+dBApRGhArsgLfac6ZmdiSK09Lfol25W7j+5zVOA== X-Gm-Gg: AY/fxX66CrjQApsxUkdpM83/QzwPjQmR6Gi02ZOp9rHhQK/Nm+VLVpaVxu6stro2OBb h6xueVZpVxXGYC4ZXvyzLVnHkvKppZFqWRAMnSZFoQJVH714Jq8sEzpkf7bPLyutS7298+aKd4M gbec4nqPb3kbXafglH7qcKwT5+ZP+WzBm1ey3D16DQ24vH3opyxnkJW9kMiB57dnILKQs7MJxGn sC95DN89R/aXQVYQ1plwfZhh1kNzmVA/gQtevCfaKGePBSm+pC6sVG7siCzCWsWSQTCeEy81U81 veqcOz/517vOFA/VSa6RClSl1OpEIfbBk9WskaTeAxgm7gFejLs7zOHUu+N+oFZOFNJA6nvqLJ4 T+FsPAVSmzRXYNo0WnMuL/w9IUL6kSjX3pn+pQidn3Acs0rxr76tqOrzaV3svGdGBG8w+65x/xu /vr7Rw X-Google-Smtp-Source: AGHT+IFB9fWalUcUiO3rdGujfqPvSy2uyVV1y9oBi7yYhQ7EZmFI5catOG8/EQRfew7LrIG6+51fPQ== X-Received: by 2002:a05:6830:2b2b:b0:7c7:701:fb10 with SMTP id 46e09a7af769-7cc668ffd39mr16342194a34.7.1766845339868; Sat, 27 Dec 2025 06:22:19 -0800 (PST) Received: from chromium.org ([73.34.74.121]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-7cc66727e11sm17352660a34.3.2025.12.27.06.22.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 27 Dec 2025 06:22:18 -0800 (PST) From: Simon Glass To: linux-arm-kernel@lists.infradead.org Cc: Masahiro Yamada , Chen-Yu Tsai , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , Nicolas Schier , =?UTF-8?q?J=20=2E=20Neusch=C3=A4fer?= , Ahmad Fatoum , Tom Rini , Simon Glass , Nicolas Schier , linux-kernel@vger.kernel.org Subject: [PATCH v8 2/6] scripts/make_fit: Support an initial ramdisk Date: Sat, 27 Dec 2025 07:21:46 -0700 Message-ID: <20251227142200.2241198-3-sjg@chromium.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251227142200.2241198-1-sjg@chromium.org> References: <20251227142200.2241198-1-sjg@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" FIT (Flat Image Tree) allows a ramdisk to be included in each configuration. Add support for this to the script. This feature is not available via 'make image.fit' since the ramdisk likely needs to be built separately anyway, e.g. using modules from the kernel build. Future work may provide support for doing that. Note that the uncompressed size is not correct when a ramdisk is used, since it is too expensive to decompress the ramdisk. Signed-off-by: Simon Glass Reviewed-by: Nicolas Schier Reviewed-by: Ahmad Fatoum --- (no changes since v6) Changes in v6: - Drop the load address for ramdisk, as it is not needed - Bring back the ramdisk 'compression' property, set to 'none' Changes in v5: - Fix 'use' typo Changes in v4: - Update the commit message Changes in v3: - Add a comment at the top of the file about the -r option - Count the ramdisk in the total files - Update the commit message Changes in v2: - Don't compress the ramdisk as it is already compressed scripts/make_fit.py | 52 ++++++++++++++++++++++++++++++++++++++------- 1 file changed, 44 insertions(+), 8 deletions(-) diff --git a/scripts/make_fit.py b/scripts/make_fit.py index 0f5e7c4b8aed..66cd73711506 100755 --- a/scripts/make_fit.py +++ b/scripts/make_fit.py @@ -10,10 +10,14 @@ Usage: make_fit.py -A arm64 -n 'Linux-6.6' -O linux -o arch/arm64/boot/image.fit -k /tmp/kern/arch/arm64/boot/image.itk - @arch/arm64/boot/dts/dtbs-list -E -c gzip + -r /boot/initrd.img-6.14.0-27-generic @arch/arm64/boot/dts/dtbs-li= st + -E -c gzip =20 -Creates a FIT containing the supplied kernel and a set of devicetree files, -either specified individually or listed in a file (with an '@' prefix). +Creates a FIT containing the supplied kernel, an optional ramdisk, and a s= et of +devicetree files, either specified individually or listed in a file (with = an +'@' prefix). + +Use -r to specify an existing ramdisk/initrd file. =20 Use -E to generate an external FIT (where the data is placed after the FIT data structure). This allows parsing of the data without loading @@ -29,8 +33,6 @@ looks at the .cmd files produced by the kernel build. =20 The resulting FIT can be booted by bootloaders which support FIT, such as U-Boot, Linuxboot, Tianocore, etc. - -Note that this tool does not yet support adding a ramdisk / initrd. """ =20 import argparse @@ -81,6 +83,8 @@ def parse_args(): help=3D'Specifies the operating system') parser.add_argument('-k', '--kernel', type=3Dstr, required=3DTrue, help=3D'Specifies the (uncompressed) kernel input file (.itk)') + parser.add_argument('-r', '--ramdisk', type=3Dstr, + help=3D'Specifies the ramdisk/initrd input file') parser.add_argument('-v', '--verbose', action=3D'store_true', help=3D'Enable verbose output') parser.add_argument('dtbs', type=3Dstr, nargs=3D'*', @@ -133,7 +137,28 @@ def write_kernel(fsw, data, args): fsw.property_u32('entry', 0) =20 =20 -def finish_fit(fsw, entries): +def write_ramdisk(fsw, data, args): + """Write out the ramdisk image + + Writes a ramdisk node along with the required properties + + Args: + fsw (libfdt.FdtSw): Object to use for writing + data (bytes): Data to write (possibly compressed) + args (Namespace): Contains necessary strings: + arch: FIT architecture, e.g. 'arm64' + fit_os: Operating Systems, e.g. 'linux' + """ + with fsw.add_node('ramdisk'): + fsw.property_string('description', 'Ramdisk') + fsw.property_string('type', 'ramdisk') + fsw.property_string('arch', args.arch) + fsw.property_string('compression', 'none') + fsw.property_string('os', args.os) + fsw.property('data', data) + + +def finish_fit(fsw, entries, has_ramdisk=3DFalse): """Finish the FIT ready for use =20 Writes the /configurations node and subnodes @@ -143,6 +168,7 @@ def finish_fit(fsw, entries): entries (list of tuple): List of configurations: str: Description of model str: Compatible stringlist + has_ramdisk (bool): True if a ramdisk is included in the FIT """ fsw.end_node() seq =3D 0 @@ -154,6 +180,8 @@ def finish_fit(fsw, entries): fsw.property_string('description', model) fsw.property('fdt', bytes(''.join(f'fdt-{x}\x00' for x in = files), "ascii")) fsw.property_string('kernel', 'kernel') + if has_ramdisk: + fsw.property_string('ramdisk', 'ramdisk') fsw.end_node() =20 =20 @@ -274,6 +302,14 @@ def build_fit(args): size +=3D os.path.getsize(args.kernel) write_kernel(fsw, comp_data, args) =20 + # Handle the ramdisk if provided. Compression is not supported as it is + # already compressed. + if args.ramdisk: + with open(args.ramdisk, 'rb') as inf: + data =3D inf.read() + size +=3D len(data) + write_ramdisk(fsw, data, args) + for fname in args.dtbs: # Ignore non-DTB (*.dtb) files if os.path.splitext(fname)[1] !=3D '.dtb': @@ -296,12 +332,12 @@ def build_fit(args): =20 entries.append([model, compat, files_seq]) =20 - finish_fit(fsw, entries) + finish_fit(fsw, entries, bool(args.ramdisk)) =20 # Include the kernel itself in the returned file count fdt =3D fsw.as_fdt() fdt.pack() - return fdt.as_bytearray(), seq + 1, size + return fdt.as_bytearray(), seq + 1 + bool(args.ramdisk), size =20 =20 def run_make_fit(): --=20 2.43.0 From nobody Sun Feb 8 18:18:48 2026 Received: from mail-ot1-f67.google.com (mail-ot1-f67.google.com [209.85.210.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EBCD031E11C for ; Sat, 27 Dec 2025 14:22:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.67 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1766845346; cv=none; b=NC+YcQMPZ179gHEvHdAa38dZ8gnM7YjnZqmvKL/kO/w4icBOrODIcrPL166RSjZVJwKzBkF4Y2lJwJgcTHBpOxuaBKWL6E/Xyjt4g+2IYWT9zxQUF4TavWKzsnoaMAjSvexZKMWL7/2rzIE5LzyTAZHvwfW75J29EsqCoKojczY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1766845346; c=relaxed/simple; bh=OJ5gjSQWKx5C84AvdM6xqDc/SoTZsEG7agwwRllLSl4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=goBondBPQrtlRIU03rY3iEckjxVCSR/tMaasv0Mr+ixgHumJqehLwtuj6ObYW8b4LvPtc6IVwe/hJ+y988yBIreHIGGAtYdH70x2hILw4VBSxLTPujcorMMiVqlc00Uteanjs7WjBfOPzNFP/Hy2wLdEohybSqLtcWQz8RALe6I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=BsifQ6g0; arc=none smtp.client-ip=209.85.210.67 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="BsifQ6g0" Received: by mail-ot1-f67.google.com with SMTP id 46e09a7af769-7c6dbdaced8so6381087a34.1 for ; Sat, 27 Dec 2025 06:22:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1766845344; x=1767450144; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=OQo1oqJAVo44ptfZ9oLDxIXzqeCSewp3mycJJep0ovk=; b=BsifQ6g0IlOoCvN7kkqClQU5VoPtuIC+dKTv6AESOqgZ+TxQdgcEswLqLbD5UskFOR Kl8hE8WWFkFR4L8i9Vua2xq0eW/sY9GKspsAsNcsw+fhWWq1l0eEnhtELIKQ5suygwb5 r+fNXJ3dsRAVrHIXeAf8fmviigYzx6gJIB9j0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1766845344; x=1767450144; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=OQo1oqJAVo44ptfZ9oLDxIXzqeCSewp3mycJJep0ovk=; b=HU0fODKkvuCPwmqWxGcvC/bM5uIMGq9xe05s/llY08beIcYNKNQuTBD8w8cFtm9mR8 PQXv/AjuspBB+0G9D0s7Yxgv3ndbvlbHfedwjQeapuljL7JxF70DgfUFtBvpo5FS740K r/ZkrOYJXKWvfXYuTR4tSGGhO32NZBKGATRtxwjMHEpOG0UXUncpB+iQ4Q3RFNYPIW3O OHbfJWoheIEXPwtbsb5dVSwz49GJsBNsBoJs4BdKyimWDjDEYgKCU3ZU9LEf3wVTy9Fo yI9K0Llt+ydBJgd2DYvELxjT82VyUZjDmbS8w+D2fv5PQ3m0X36RYDzii3VnU8VC1FSg Q0IA== X-Forwarded-Encrypted: i=1; AJvYcCU5jNnar7eDZ3rii+/2ZTW0Dl7zXC+64A/cbD4PiSGfk3+Aso5OaaC0Gm97VcKD7IPNQ6zXnsPEc+3qj/0=@vger.kernel.org X-Gm-Message-State: AOJu0Yx/w2fiCYwU27x/NQ27Jq1cuI1Fk9VHoDvxhmGYrCaCl4owaamE 1JjoHas4mzxE8bioCct5IsHVdS+aMosy8d+g5Sb4M7Q7SUyNC0kz3dzIU6GfHdtBzQ== X-Gm-Gg: AY/fxX4TNdkjRHCAg9l+wL57/Y0SZhfaMnql5FjQQ0dAQvpVEaGoqbo17pBGJDRSFP0 gNJpcb0LnzC4L5g3Pxlj3Pf4xxOI/3XjlShHrZgq9UrOK61JU7RHX1QTiIoXZEAlTBRKvOf1vbL 9kdz+PUCQmqbw8fsj8TUSkDwAIg0lQQ83H6TCzrT6fYUZ0whah6estqCSSZAFeiIYcQkS7nrxw9 5ckrJSI8e7zHpAYhFCBAVv+VY+8Ll3CIEcAE0qmIGCDM0p/cKPnwIvA1wttRFKnZkSX4WKFM6+Y cVMkfpmd+26EyPsMDHwpjWX+22qhdtz/q/5Wf6lmHE3zXCVpdDoJ60x75/MlYX3wZU0i4jUdyzX Ej6+QoLJyWlJtg7vAImd3UvZiS18bFI98Ny+/68NP//jRlRrn22MbpL8sSVKZjSpHG3gUpnKd1K COUyo68rfS44bSMWA= X-Google-Smtp-Source: AGHT+IG/iPHYfwLVw3hTqVWk+/TFfy/39UQV5YqNYUp6ccKtSuEg3+Tg7hchdtSmxedRas5yzGbUtg== X-Received: by 2002:a05:6830:3894:b0:7c7:501f:4bce with SMTP id 46e09a7af769-7cc66a952d3mr11728229a34.25.1766845343827; Sat, 27 Dec 2025 06:22:23 -0800 (PST) Received: from chromium.org ([73.34.74.121]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-7cc66727e11sm17352660a34.3.2025.12.27.06.22.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 27 Dec 2025 06:22:20 -0800 (PST) From: Simon Glass To: linux-arm-kernel@lists.infradead.org Cc: Masahiro Yamada , Chen-Yu Tsai , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , Nicolas Schier , =?UTF-8?q?J=20=2E=20Neusch=C3=A4fer?= , Ahmad Fatoum , Tom Rini , Simon Glass , Nicolas Schier , linux-kernel@vger.kernel.org Subject: [PATCH v8 3/6] scripts/make_fit: Move dtb processing into a function Date: Sat, 27 Dec 2025 07:21:47 -0700 Message-ID: <20251227142200.2241198-4-sjg@chromium.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251227142200.2241198-1-sjg@chromium.org> References: <20251227142200.2241198-1-sjg@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Since build_fit() is getting quite long, move the dtb processing into a separate function. Change the double quotes in the write() call to single, to match the rest of the script. Signed-off-by: Simon Glass Reviewed-by: Nicolas Schier Reviewed-by: Ahmad Fatoum Reviewed-by: Chen-Yu Tsai --- (no changes since v7) Changes in v7: - Add a note about the " -> ' change scripts/make_fit.py | 67 +++++++++++++++++++++++++++++---------------- 1 file changed, 44 insertions(+), 23 deletions(-) diff --git a/scripts/make_fit.py b/scripts/make_fit.py index 66cd73711506..7ddaf3c0d783 100755 --- a/scripts/make_fit.py +++ b/scripts/make_fit.py @@ -277,6 +277,47 @@ def process_dtb(fname, args): =20 return (model, compat, files) =20 + +def _process_dtbs(args, fsw, entries, fdts): + """Process all DTB files and add them to the FIT + + Args: + args: Program arguments + fsw: FIT writer object + entries: List to append entries to + fdts: Dictionary of processed DTBs + + Returns: + tuple: + Number of files processed + Total size of files processed + """ + seq =3D 0 + size =3D 0 + for fname in args.dtbs: + # Ignore non-DTB (*.dtb) files + if os.path.splitext(fname)[1] !=3D '.dtb': + continue + + try: + (model, compat, files) =3D process_dtb(fname, args) + except Exception as e: + sys.stderr.write(f'Error processing {fname}:\n') + raise e + + for fn in files: + if fn not in fdts: + seq +=3D 1 + size +=3D os.path.getsize(fn) + output_dtb(fsw, seq, fn, args.arch, args.compress) + fdts[fn] =3D seq + + files_seq =3D [fdts[fn] for fn in files] + entries.append([model, compat, files_seq]) + + return seq, size + + def build_fit(args): """Build the FIT from the provided files and arguments =20 @@ -289,7 +330,6 @@ def build_fit(args): int: Number of configurations generated size: Total uncompressed size of data """ - seq =3D 0 size =3D 0 fsw =3D libfdt.FdtSw() setup_fit(fsw, args.name) @@ -310,34 +350,15 @@ def build_fit(args): size +=3D len(data) write_ramdisk(fsw, data, args) =20 - for fname in args.dtbs: - # Ignore non-DTB (*.dtb) files - if os.path.splitext(fname)[1] !=3D '.dtb': - continue - - try: - (model, compat, files) =3D process_dtb(fname, args) - except Exception as e: - sys.stderr.write(f"Error processing {fname}:\n") - raise e - - for fn in files: - if fn not in fdts: - seq +=3D 1 - size +=3D os.path.getsize(fn) - output_dtb(fsw, seq, fn, args.arch, args.compress) - fdts[fn] =3D seq - - files_seq =3D [fdts[fn] for fn in files] - - entries.append([model, compat, files_seq]) + count, fdt_size =3D _process_dtbs(args, fsw, entries, fdts) + size +=3D fdt_size =20 finish_fit(fsw, entries, bool(args.ramdisk)) =20 # Include the kernel itself in the returned file count fdt =3D fsw.as_fdt() fdt.pack() - return fdt.as_bytearray(), seq + 1 + bool(args.ramdisk), size + return fdt.as_bytearray(), count + 1 + bool(args.ramdisk), size =20 =20 def run_make_fit(): --=20 2.43.0 From nobody Sun Feb 8 18:18:48 2026 Received: from mail-ot1-f52.google.com (mail-ot1-f52.google.com [209.85.210.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CCD9B3203B4 for ; Sat, 27 Dec 2025 14:22:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1766845350; cv=none; b=NaxB/rdFbW6NLAk8tQ/2trMrrwO9KdZzR3EJKhvGITq9Mz8WCf6QInaXIW+91S5E9zca6DjUTdTpYfA/d7UlkUB0I0YBjjZN6GYb7FwTcHxsQorFV2GnSeQmikQmNmXIZFvCGl5JAiS5QVuMXdL3oeczrylETNg0BYn9srviCC4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1766845350; c=relaxed/simple; bh=XjhzIln6T98OU+/144ye0mPj2kCYhj5pKwfpf9/qz28=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=PwMOvoP6vCnbZJrbUab4i6c1eNqdpzVV0v1xMXLVcbNgLjNA9jbOx8CJMS2/gC/xQeR4p9KXdN1cZEndSGyt/UHfuiy/Z+JkyBGux8CM7WJBQTMLVEGWshSB+4vO55IWdrQ8VLCTfHG/tdU8MS8P9OXiGFlNJKlRKbI1p5ne/WM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=CHvQerfY; arc=none smtp.client-ip=209.85.210.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="CHvQerfY" Received: by mail-ot1-f52.google.com with SMTP id 46e09a7af769-7c7aee74dceso3032420a34.2 for ; Sat, 27 Dec 2025 06:22:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1766845348; x=1767450148; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Stn7BG5p2kiWu3fO1SYV4KqjZkimP4vmWf/OzcEQFxc=; b=CHvQerfYIuw6GzNk8b+BV6Fbc5OmUXBL5xxw//8aqvEVtCsprTLnO+SPpenDCwaJRp gdeXLRW1t9YG0Wmup9DgFDuZZSG+qEdoNky5wXlUwH5lMO6OwBDD1H1P1jCOfqKHQvl8 mU/cXYvEtC53JnPuY5nrsxSDFF0rI3vsGVkw8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1766845348; x=1767450148; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=Stn7BG5p2kiWu3fO1SYV4KqjZkimP4vmWf/OzcEQFxc=; b=k2fx7JFCG2dKjboAzxx/Z8b/bdHPsr11FN4GkTrUBgJS9RB0Is1FVvUS9B+Y7zwatd O+4Mw0vEN/ofBj+US663nWU1nMmn5ZATfvzPKiGX8YR/rfgOWgC0iBbsSFy3Dn0FHcaf C04k0+OvXWoH5CmsHIaTPDF5iinjDWGEZzc0GArXWl78Ovln5aP+Up6/7vzVkwvi+zh2 ofzTx5b3yNc3gSdtVMJXtJLd7vKfVuyMyP8Dlar+4ciasEC5to2ZG9F7adtLMF+STk9r EBAJdxXKM28RTaoNOa3NJz8zzNCwS9CQD+oFe9KBHRcr8FCfdGFDM4udRbwbfDB8XjdL 0UUA== X-Forwarded-Encrypted: i=1; AJvYcCUZSCW/MRmjTrelyuKY9D9flJq7er0SVvZ4v2WHfxs9agih9SHBteo24JOjERytyqAMi3rHmLvTFbma3Lc=@vger.kernel.org X-Gm-Message-State: AOJu0YzYfj6t58Fe6jXMSLtWRn3P21AJXGVDpHpfCFp08MdLyPa9Zc0r r3vXVMUIBFdUp83vnQBgtZjYzfX44zw/z6kT8aryyYxx5X29e0TouLPXPfu+Xu7/1w== X-Gm-Gg: AY/fxX71p/6B4vLp6WVN1KVgPUOUb7q0k3Q6VhkJTXd7cAH1LAtnuAfWoQiQn38/q9z gHeLgDZZdFf6YLSE52njwkgdOCwvcjwC1ZmHCUPsQytsPO998UxWfP2fGysutalTG1QKWdOJcg/ B2PsnCNc/9GQZjj6Edbqpw1JdjUwxjKX6Eg/NH0UDUt0O4XcO6O3LuojjuPHBSVY2x7+EB3eVOK G3kzG029kn/4DzXez5cQeKzo1Nw9QcwnewB8ck1l2fUecawITFYHjIi27y3rDFgLhSh7RxSpLk5 J5Dq3DlypzAc3b3+VG5GBftpHsScQ5f6qrT363qAAjjzV6/768B+GYjiG6ZCgtFyaSgtQRIE5Iq LmjGh3AVT9N7aBUS3snnoTEEVKHYxMCGZjfq64CF7SpyZvlC5NU86MZknxqGFtR7eK97QFaVMZW 2z1Miu2MKNVU/H0lE= X-Google-Smtp-Source: AGHT+IGLZpCOF9onuPISW/tPeWU8M37UAPoGGRxrM8fdwUy2auYhL1uc1u62/1fsGdJy6tTGv+8Z1A== X-Received: by 2002:a05:6808:4a52:10b0:459:b530:28ae with SMTP id 5614622812f47-459b5302fb7mr2050092b6e.63.1766845347754; Sat, 27 Dec 2025 06:22:27 -0800 (PST) Received: from chromium.org ([73.34.74.121]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-7cc66727e11sm17352660a34.3.2025.12.27.06.22.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 27 Dec 2025 06:22:25 -0800 (PST) From: Simon Glass To: linux-arm-kernel@lists.infradead.org Cc: Masahiro Yamada , Chen-Yu Tsai , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , Nicolas Schier , =?UTF-8?q?J=20=2E=20Neusch=C3=A4fer?= , Ahmad Fatoum , Tom Rini , Simon Glass , Alice Ryhl , Ard Biesheuvel , Ingo Molnar , Josh Poimboeuf , Kees Cook , Nathan Chancellor , Nicolas Schier , Oleh Zadorozhnyi , Petr Mladek , linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v8 4/6] kbuild: Support a MAKE_FIT_FLAGS environment variable Date: Sat, 27 Dec 2025 07:21:48 -0700 Message-ID: <20251227142200.2241198-5-sjg@chromium.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251227142200.2241198-1-sjg@chromium.org> References: <20251227142200.2241198-1-sjg@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In some cases it is useful to be able to pass additional flags to the make_fit.py script. For example, since ramdisks are typically large, passing -E to use external data can be helpful. Add a new MAKE_FIT_FLAGS variable for this. Signed-off-by: Simon Glass Reviewed-by: Nicolas Schier Reviewed-by: Thomas Wei=C3=9Fschuh --- (no changes since v7) Changes in v7: - Add a new patch with the MAKE_FIT_FLAGS change scripts/Makefile.lib | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib index 28a1c08e3b22..a5641a6538b1 100644 --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -400,7 +400,7 @@ FIT_COMPRESSION ?=3D gzip =20 quiet_cmd_fit =3D FIT $@ cmd_fit =3D $(MAKE_FIT) -o $@ --arch $(UIMAGE_ARCH) --os linux \ - --name '$(UIMAGE_NAME)' \ + --name '$(UIMAGE_NAME)' $(MAKE_FIT_FLAGS) \ $(if $(findstring 1,$(KBUILD_VERBOSE)),-v) \ $(if $(FIT_DECOMPOSE_DTBS),--decompose-dtbs) \ --compress $(FIT_COMPRESSION) -k $< @$(word 2,$^) --=20 2.43.0 From nobody Sun Feb 8 18:18:48 2026 Received: from mail-oi1-f180.google.com (mail-oi1-f180.google.com [209.85.167.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4CC97227E95 for ; Sat, 27 Dec 2025 14:22:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1766845355; cv=none; b=fFPcMR2CJqwxTz2346WylPVle34Ya8q/D3UgPto44kbnoNFkzDys3g23EcQJ18MRYx9bAcQXdzIr8XkWaq9yY0qM4OQJVR5KQ1Lf2QVMgpJOH5ggB4yiD8/Li3Y7A1jFWEx7lwmByK8OW6Xu9p8hxK2Ta+HLR3w34slBrsLRhTs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1766845355; c=relaxed/simple; bh=EMPPl1WC5jzc2li05ovAeRNEiHN208aqLkzCt8HS/CQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=aQOGod/dBVpAv3AYsVH0Mv8fOfdzqAU6QT2uYtw9igcks4REAqnYh4ZPbYxg+vzdhcXLdCtZH5JiHCsm9482eCRaBB57smfWWZ4S9fqAgR0s7aDB1mIsBhxG1NzbvBGcgopHgwY42gzEVCXLdF4htYeURm1+DxKwMo5L3/9FSt4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=lkbKLi2W; arc=none smtp.client-ip=209.85.167.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="lkbKLi2W" Received: by mail-oi1-f180.google.com with SMTP id 5614622812f47-455749af2e1so5486872b6e.1 for ; Sat, 27 Dec 2025 06:22:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1766845352; x=1767450152; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ev5dZu+MNaNiCXAmHypJMGG2mZAu4GrojXqEHePJJcQ=; b=lkbKLi2WQnorOODRstfHuA9fRDQ0r0CyJpOBUIiN6IexvwS13vwevfQn9H5V74slTW 26qqiQ1+o1mDH5svrhAdi57m9oxK4UKdOQsVJwfzlIo2uxYefkiEa90DWspbRsiuXORg L+L0bWCx1oID0jmlUP9qDdrdVgprtXXEHaYhA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1766845352; x=1767450152; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=ev5dZu+MNaNiCXAmHypJMGG2mZAu4GrojXqEHePJJcQ=; b=I9NTzPvYV13QBb5vb4+GB3iJM8powPqUT/gXhF6oZngxYXW3asNPLKKGTZBhcrNhKG ue60AcPWBKS7/DZPZlER3XlC+7vQSuq+U79FU6NcvnxgC2u4HBxQl6gicPcwx+Vo2ap/ EYs6MVeoZbSyAfiOhP5TCtvt4dF2n/ZiNcJ2lxV9tSUB9g2jEoE8+ReCRyDgtQXT+TNf qG6J3kTB1x61Jd3vGxrBtdl8569FanJQbjlMBP4PPFenCvwtupHLRUjf+yNVU2OxuOsH KwN3FkiNU9d54FovQD5U7Nly/YUTsC7T9Wa7HzeAYSEqwF26HNV0eFr9FfzgOGwzHxvl 9OoA== X-Forwarded-Encrypted: i=1; AJvYcCWjoR4Ef4XIYFE6gdQwhktYazErWDGCCPgyawCZPMjQVcq+iU5hr3D9jauu8oJBUJ4gZPeMIo3oM47iCw0=@vger.kernel.org X-Gm-Message-State: AOJu0Yxht5UVh14c6tyhZmDV4HgPJviptRiz8MXAxVG0lY5ABZU7jKvp i4zFmFIdj6cabOum2mkG3fEPoWXcItHIYrs+HR5UMEDUw7n0PdzhtQKhdJiG8af9EydFNCAdzQX k/vU= X-Gm-Gg: AY/fxX5vvmzXVF3+YXSmhLVppnu1qacjHJWRNidfd9d6EmtN1/oYTy0cLUTvX8fenNE 1cszljokxbXuInXTylLRMQ7K2nRUYzo2oOaqg70bxk115fPb7KEhnfo6nw5F6UaLphuVYVCeNx6 G4liAT5ODVFHBKL3m5FIEy6zCOZ6sUNS6iJUoIwPJ1UMj+wCbBDKyfx61pP/oOJZ5hf1VWwBLLt V9gvZXhR0bfhanANeUCbmlAbL+/yPpiCY9qI9/UNmIqyKpYQ3zQgH5ZfVxUK7JOV5n65o/jGc1X tfj4UGIPy2e8fmcIVg8FblaVRO4T9jkvjqmfDJjJyBoMg5PyZ6Io8plWx2Vly8figcpL0aRHOl9 iLJgHh1zuIs6hrFf2f2MVKViVQmEkxnDemSF/EbSBT/CjbP3yS5Cw3htVZKjehMNEFQcrkVwV8z 5XIWzn X-Google-Smtp-Source: AGHT+IFbD2T68RWby1LSxTFnHXAreDJGvHAACCzMPzQU2qMbAT3eAIYfVlwPjWmULM1RMKcUvRYWpA== X-Received: by 2002:a05:6808:6a83:b0:450:b92c:aaa2 with SMTP id 5614622812f47-457a292c4d7mr13567904b6e.18.1766845352225; Sat, 27 Dec 2025 06:22:32 -0800 (PST) Received: from chromium.org ([73.34.74.121]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-7cc66727e11sm17352660a34.3.2025.12.27.06.22.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 27 Dec 2025 06:22:29 -0800 (PST) From: Simon Glass To: linux-arm-kernel@lists.infradead.org Cc: Masahiro Yamada , Chen-Yu Tsai , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , Nicolas Schier , =?UTF-8?q?J=20=2E=20Neusch=C3=A4fer?= , Ahmad Fatoum , Tom Rini , Simon Glass , David Sterba , Nick Terrell , linux-kernel@vger.kernel.org Subject: [PATCH v8 5/6] scripts/make_fit: Support a few more parallel compressors Date: Sat, 27 Dec 2025 07:21:49 -0700 Message-ID: <20251227142200.2241198-6-sjg@chromium.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251227142200.2241198-1-sjg@chromium.org> References: <20251227142200.2241198-1-sjg@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add support for pbzip2, xz and plzip which can compress in parallel. This speeds up the ramdisk compression. Signed-off-by: Simon Glass --- (no changes since v7) Changes in v7: - Mention xz as well, in the commit message scripts/make_fit.py | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/scripts/make_fit.py b/scripts/make_fit.py index 7ddaf3c0d783..defb87f223d7 100755 --- a/scripts/make_fit.py +++ b/scripts/make_fit.py @@ -50,11 +50,12 @@ import libfdt CompTool =3D collections.namedtuple('CompTool', 'ext,tools') =20 COMP_TOOLS =3D { - 'bzip2': CompTool('.bz2', 'bzip2'), + 'bzip2': CompTool('.bz2', 'pbzip2,bzip2'), 'gzip': CompTool('.gz', 'pigz,gzip'), 'lz4': CompTool('.lz4', 'lz4'), - 'lzma': CompTool('.lzma', 'lzma'), + 'lzma': CompTool('.lzma', 'plzip,lzma'), 'lzo': CompTool('.lzo', 'lzop'), + 'xz': CompTool('.xz', 'xz'), 'zstd': CompTool('.zstd', 'zstd'), } =20 @@ -207,7 +208,12 @@ def compress_data(inf, compress): done =3D False for tool in comp.tools.split(','): try: - subprocess.call([tool, '-c'], stdin=3Dinf, stdout=3Dou= tf) + # Add parallel flags for tools that support them + cmd =3D [tool] + if tool in ('zstd', 'xz'): + cmd.extend(['-T0']) # Use all available cores + cmd.append('-c') + subprocess.call(cmd, stdin=3Dinf, stdout=3Doutf) done =3D True break except FileNotFoundError: --=20 2.43.0 From nobody Sun Feb 8 18:18:48 2026 Received: from mail-ot1-f51.google.com (mail-ot1-f51.google.com [209.85.210.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B4F2031D726 for ; Sat, 27 Dec 2025 14:22:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1766845357; cv=none; b=dDBwnH+gt4V+/ExnSOkNAL3fqYlMOzakJtWRUm08m08FRrIOTpyJkUQPAqCCniVwuAEiv+LFAO79jyUlHF/m92OWpOr7aIRkZAx3fwYTFUOIaxpw86VcIzfnJyphriFqs9H15sNqsVsZGJYgbuZsAPKkbkeamqjLPJ+SYz4PxFc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1766845357; c=relaxed/simple; bh=2v9Ov13bzeY9YTAeY6kGNre6+aj8ZdSVZ65J0kCe5fM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=dUKYuV7oJd/odQajocprxGYwc0DDRHxup4XYQfWPjeHBfuGpltU9HXCsFRtI8pD+tMZ0/K0jqdBVLcZb2lsr+gEc8SoE9nUDH4Watxmp7lDf4hhEaT813DXpwT0p61osrW+xaGhsgmuRsV1eXNSr0BIaRLsfOpih4Qf/PsA17Yo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=gIdweX3u; arc=none smtp.client-ip=209.85.210.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="gIdweX3u" Received: by mail-ot1-f51.google.com with SMTP id 46e09a7af769-7cac9225485so6274577a34.3 for ; Sat, 27 Dec 2025 06:22:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1766845354; x=1767450154; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=dkOl3+2E5uBM0wxWZy4TsfgVfQXwCyjeP1JIoSVkCIk=; b=gIdweX3uFZUegh/3JjfcS/SgtjJfYd+kv85cmA4wXBE2T+IT5F4OkTp8oYibcvzckZ JRYAMtw5wK33VwgX095LcmpW/kkbVlAsCIbbI3BrbyTmCHD7cz4S2EwfYSxNTA4F6YbD C/pyX6Vqr5Ywk83aBQ9c/mKO5xPYSzb+VjpQI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1766845354; x=1767450154; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=dkOl3+2E5uBM0wxWZy4TsfgVfQXwCyjeP1JIoSVkCIk=; b=Twq2J8cQReSIoIpBSpKOBCIkbg1FTCrjBieUE2Y4NT0XGPFoqLy05Bu0cR5rojLlSS 3GPCxtYdr2E1sjbRne86WdnpX/oa93XoDoTEm2P6mWcnNdGf1x/9kzPZ7Y+O4P23+mJ8 uG5x+v3hmqe3J1QTUjx1wY60cvsRV0y3cKvv8es5noi9+7lDOXOdDicjxSJB3kmkqpDA w6sB9WdKzC+V8DKplPSlh0yYlKPFC1EQ+0LT1jGNNeHFJdwQFghmMzBp086yR82dnv2E /cWkyHTUof3xTJ5KUy5dntP4D2fMyTNGxoMG8YXpLdYuy3u+y1CnJG3zmqnfYYNVRDou XimA== X-Forwarded-Encrypted: i=1; AJvYcCUpeq1Saxj+ZKugbJZt1GS/LfAuNqOU2Zkjhg9NbUm8cWX/Ilc5CsLHmBJZYstSuWaomRha3jgzlq9pMUY=@vger.kernel.org X-Gm-Message-State: AOJu0YxoGKPijGuBsMJMtsLaWvmCFpe8S4Hpjrxdl/fDh5neQxN+Dz8y nskWc20xJvAXnSsT4zHU1tTo06BqT03ay8/aMVsD8kx2KihlgjPnzgIgzIOeqosmTA== X-Gm-Gg: AY/fxX7+wGhb/wG1nCJYzTKesbw1xY7tMi3cNDsn9SeiWolToXQAqI7ghiPekdKX734 JT6m+Q18UR3OSKo200NPEWjAswd4sTO0dm+G+kvXCmsNwvimnxCqKgo7OcbbjVF02gCbs55+WHp WEbxPXPLoucyb2QjUpmCm/llKYkPPld7KfQcT6FmiAbKU19qGE4Gxg2QQGA7uxVrGzlweA50RHF R995JfstXQuMgi8F+3pnCb4TwRHMkeES7pXfodXm1ZqjSzxAQT3Zx7xNNQe/O1RQ8ExT6HhhQtO FrgUhh3vv8VfqxiV2LdJghlZxZAS0NOKUaGcY5JJE1rS6KPTMT1cqEwBAF7XYli/ObHWO7rV1Mf 6gtEz07UmP7qL6EtoGRIvL1h8TFZSMCuTWl7Xl9/60OnXJmdDkSfGwXmmqpya156aSFIeVGKScn m5PglB X-Google-Smtp-Source: AGHT+IH+2fFfhpMMxFeukqxtBdzObwyT8Ia+HitEAnOFLETYjHsbEDshOLHe3rAQCd4FWWRGCErwog== X-Received: by 2002:a05:6830:4104:b0:7c7:62d0:b462 with SMTP id 46e09a7af769-7cc668ffda5mr13302862a34.6.1766845354627; Sat, 27 Dec 2025 06:22:34 -0800 (PST) Received: from chromium.org ([73.34.74.121]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-7cc66727e11sm17352660a34.3.2025.12.27.06.22.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 27 Dec 2025 06:22:33 -0800 (PST) From: Simon Glass To: linux-arm-kernel@lists.infradead.org Cc: Masahiro Yamada , Chen-Yu Tsai , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , Nicolas Schier , =?UTF-8?q?J=20=2E=20Neusch=C3=A4fer?= , Ahmad Fatoum , Tom Rini , Simon Glass , linux-kernel@vger.kernel.org Subject: [PATCH v8 6/6] scripts/make_fit: Compress dtbs in parallel Date: Sat, 27 Dec 2025 07:21:50 -0700 Message-ID: <20251227142200.2241198-7-sjg@chromium.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251227142200.2241198-1-sjg@chromium.org> References: <20251227142200.2241198-1-sjg@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When there are 1500 device tree files it takes quite a while to compress them. Do it in parallel. Signed-off-by: Simon Glass --- Changes in v8: - Drop erroneous line 'kbuild: Allow adding modules into the FIT ...' scripts/make_fit.py | 56 ++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 50 insertions(+), 6 deletions(-) diff --git a/scripts/make_fit.py b/scripts/make_fit.py index defb87f223d7..e923cc8b05b7 100755 --- a/scripts/make_fit.py +++ b/scripts/make_fit.py @@ -37,6 +37,7 @@ as U-Boot, Linuxboot, Tianocore, etc. =20 import argparse import collections +import multiprocessing import os import subprocess import sys @@ -225,15 +226,31 @@ def compress_data(inf, compress): return comp_data =20 =20 -def output_dtb(fsw, seq, fname, arch, compress): +def compress_dtb(fname, compress): + """Compress a single DTB file + + Args: + fname (str): Filename containing the DTB + compress (str): Compression algorithm, e.g. 'gzip' + + Returns: + tuple: (str: fname, bytes: compressed_data) + """ + with open(fname, 'rb') as inf: + compressed =3D compress_data(inf, compress) + return fname, compressed + + +def output_dtb(fsw, seq, fname, arch, compress, data=3DNone): """Write out a single devicetree to the FIT =20 Args: fsw (libfdt.FdtSw): Object to use for writing seq (int): Sequence number (1 for first) fname (str): Filename containing the DTB - arch: FIT architecture, e.g. 'arm64' + arch (str): FIT architecture, e.g. 'arm64' compress (str): Compressed algorithm, e.g. 'gzip' + data (bytes): Pre-compressed data (optional) """ with fsw.add_node(f'fdt-{seq}'): fsw.property_string('description', os.path.basename(fname)) @@ -241,9 +258,10 @@ def output_dtb(fsw, seq, fname, arch, compress): fsw.property_string('arch', arch) fsw.property_string('compression', compress) =20 - with open(fname, 'rb') as inf: - compressed =3D compress_data(inf, compress) - fsw.property('data', compressed) + if data is None: + with open(fname, 'rb') as inf: + data =3D compress_data(inf, compress) + fsw.property('data', data) =20 =20 def process_dtb(fname, args): @@ -300,6 +318,11 @@ def _process_dtbs(args, fsw, entries, fdts): """ seq =3D 0 size =3D 0 + + # First figure out the unique DTB files that need compression + todo =3D [] + file_info =3D [] # List of (fname, model, compat, files) tuples + for fname in args.dtbs: # Ignore non-DTB (*.dtb) files if os.path.splitext(fname)[1] !=3D '.dtb': @@ -311,11 +334,32 @@ def _process_dtbs(args, fsw, entries, fdts): sys.stderr.write(f'Error processing {fname}:\n') raise e =20 + file_info.append((fname, model, compat, files)) + for fn in files: + if fn not in fdts and fn not in todo: + todo.append(fn) + + # Compress all DTBs in parallel + cache =3D {} + if todo and args.compress !=3D 'none': + if args.verbose: + print(f'Compressing {len(todo)} DTBs...') + + with multiprocessing.Pool() as pool: + compress_args =3D [(fn, args.compress) for fn in todo] + # unpacks each tuple, calls compress_dtb(fn, compress) in para= llel + results =3D pool.starmap(compress_dtb, compress_args) + + cache =3D dict(results) + + # Now write all DTBs to the FIT using pre-compressed data + for fname, model, compat, files in file_info: for fn in files: if fn not in fdts: seq +=3D 1 size +=3D os.path.getsize(fn) - output_dtb(fsw, seq, fn, args.arch, args.compress) + output_dtb(fsw, seq, fn, args.arch, args.compress, + cache.get(fn)) fdts[fn] =3D seq =20 files_seq =3D [fdts[fn] for fn in files] --=20 2.43.0