From nobody Sun Feb 8 11:16:49 2026 Received: from mail-io1-f44.google.com (mail-io1-f44.google.com [209.85.166.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EE968326939 for ; Fri, 14 Nov 2025 14:28:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763130485; cv=none; b=U0MiLzJV18w4uvNDBesZ95f5HngiVoWt8hOF4p0vFEyLLNdimr/lOQjCS6Mr4Vdh/YNNFzMb8ux27gkj9BoUuVqbMankP4xKlzgNRW+RQnpcJMnA/9O/mydcvdLmYxh1AIp4yVZD7U13T9GwUmypybhE/quaK/8qstZvADIh2X0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763130485; c=relaxed/simple; bh=WxmuUnG9mgIBxKipIANrDQZ0nIjxvThuVKA/zPa2iQ4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ACvRTKf206LFCzIAg1wuF6IGC0Cs3d/9rmb2zeoObLDsR+LvzBhzEkugS49wv5eTaKu8ZbzxsIYA0XNH/LqLjPrPv2Np+qz/xwcemvExtrKh9NJ4asLPVexe6F2N6zHS185viSavrb7wkNsOusA5werxsTE6a0KB2lRE3NeB1KM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=QVRdIOkV; arc=none smtp.client-ip=209.85.166.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="QVRdIOkV" Received: by mail-io1-f44.google.com with SMTP id ca18e2360f4ac-948a50b8ea6so142657439f.1 for ; Fri, 14 Nov 2025 06:28:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1763130483; x=1763735283; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VOwDYlRH5DhNzOzjWWbwzUOQiOeiolgsqjgpBqGQ14g=; b=QVRdIOkVLQSxIoEqHfBv3mF2pRLlruNqQ/tUeJ9cPJLWohTVCZPT3Pj0yXk4rdyclO LN5X3q2c9MKyAkdjTiq2k2FA7Uw1DrgeV9Owb/dh109kznCpGTsjwVITxmLFYiZFi8pP /jkKvZkf+gvyMBQF94ebfHCRX5NBrfMoZqRfc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1763130483; x=1763735283; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=VOwDYlRH5DhNzOzjWWbwzUOQiOeiolgsqjgpBqGQ14g=; b=PvxVapTskuK5hfVe0PfUE2fgGx42xOTBqDowSQaGUuI9169icRW4QrqQ4P16zhdQdY DD9dCKwGiPGeLbQ/uKf5D+2XSJPI0/a2ABoPWPrWHlUc7Bbtl6nHXBNztDEU+ocp3Xio U/NQZ76G23UvJbpGmUDlobHsaEdpf2472GBl3I1nC+ayqQIsnRCEVdOqYQhW77hZpSrI AEpxNh4WsVDhi59W2YWKGae/Gliwlon0E7XlKCCkiWC7DoQTpr/c/ZsdE+oL7d/5TUHK oVWbgKDZa1QzdJTDL3i/RHgN4L0ZS4iN4PEkFJoLjl4hyQ1CVQMHB6Vzl/dPdWfqWK2K 7Isg== X-Forwarded-Encrypted: i=1; AJvYcCVMeX7OgsJUi6F8oQwAVCTzWE4FsKITjZ19n1arOAoUAmhjAmtb3jvIY0p7ERsCYGX2EWtGieUGRKpHsUo=@vger.kernel.org X-Gm-Message-State: AOJu0Yx4YwBTw4ABCn6QNCDfk6q31oPty2q/ILFO3LYALcA9HV3v2qsT M12eUw2/V4CQwhIMByFNtqaisB69LQghKuTLhx3nkrbFQoS7Qdcy7DU7SJk4k0XhxA== X-Gm-Gg: ASbGnctsk/wf3ntV5kFMIhzqJ3IUb+Q43UoWhEbUyCePF6NLWXo3NYkDJgCW2oR36m/ Brmv3kgXBDuD/QYUp5FcoaP0lwx67VCDsQ0p59rGsW3OtaeMQkt8V9+aet8A3d71n0xgLpnFJcH XCBKhb78VjkNkSZn1bnkH4N6n1yeJRSte9ZZm8VVm4OAlFj5H0SGfid2kLWxHTNTXRLuEQohNJk mal/LfOiQoK1XP7rAsg6A4g9S2nYlKSrrW11+nyNvlMSoUx9Ws8gZ1+qLNKODEEyNwyASI4eExz D14vlVaueSe5UCWdXBs9YrKiGrowALMOQcHXDsbPeSHGJPw3kb4rp51BDxhBTBeIfiQE3JisXxZ SmJ5q988vsv37qQgzPjCmCSod7MKxaC0KWEYiS3VjWWySGE9OhDJ1SzFPNiihizDWF9VrzWa4r1 moC1n6H+Xq8YiBWyE= X-Google-Smtp-Source: AGHT+IHKQCuiUr5zChKS0MHb/W5O2DKQjIJyTMquCSOnoatcp/1EvxDxrceULeEYGZCSDbMoVvwx8A== X-Received: by 2002:a05:6e02:1a4c:b0:434:7caa:8947 with SMTP id e9e14a558f8ab-4348c949d5bmr50346505ab.28.1763130483086; Fri, 14 Nov 2025 06:28:03 -0800 (PST) Received: from chromium.org ([73.34.74.121]) by smtp.gmail.com with ESMTPSA id e9e14a558f8ab-434839cdb10sm27500205ab.32.2025.11.14.06.28.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Nov 2025 06:28:02 -0800 (PST) From: Simon Glass To: linux-arm-kernel@lists.infradead.org Cc: Nicolas Schier , Masahiro Yamada , Chen-Yu Tsai , Tom Rini , Ahmad Fatoum , =?UTF-8?q?J=20=2E=20Neusch=C3=A4fer?= , Simon Glass , Nicolas Schier , linux-kernel@vger.kernel.org Subject: [PATCH v5 1/8] scripts/make_fit: Speed up operation Date: Fri, 14 Nov 2025 07:27:27 -0700 Message-ID: <20251114142741.1919072-2-sjg@chromium.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251114142741.1919072-1-sjg@chromium.org> References: <20251114142741.1919072-1-sjg@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The kernel is likely at least 16MB so we may as well use that as a step size when reallocating space for the FIT in memory. Pack the FIT at the end, so there is no wasted space. This reduces the time to pack by an order of magnitude, or so. Signed-off-by: Simon Glass Reviewed-by: Nicolas Schier --- (no changes since v3) Changes in v3: - Move the ramdisk chunk into the correct patch scripts/make_fit.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/scripts/make_fit.py b/scripts/make_fit.py index 1683e5ec6e67..0f5e7c4b8aed 100755 --- a/scripts/make_fit.py +++ b/scripts/make_fit.py @@ -98,7 +98,7 @@ def setup_fit(fsw, name): fsw (libfdt.FdtSw): Object to use for writing name (str): Name of kernel image """ - fsw.INC_SIZE =3D 65536 + fsw.INC_SIZE =3D 16 << 20 fsw.finish_reservemap() fsw.begin_node('') fsw.property_string('description', f'{name} with devicetree set') @@ -299,7 +299,9 @@ def build_fit(args): finish_fit(fsw, entries) =20 # Include the kernel itself in the returned file count - return fsw.as_fdt().as_bytearray(), seq + 1, size + fdt =3D fsw.as_fdt() + fdt.pack() + return fdt.as_bytearray(), seq + 1, size =20 =20 def run_make_fit(): --=20 2.43.0 From nobody Sun Feb 8 11:16:49 2026 Received: from mail-il1-f175.google.com (mail-il1-f175.google.com [209.85.166.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 49248326D51 for ; Fri, 14 Nov 2025 14:28:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763130488; cv=none; b=G043EAAWvdqqvq75JURFPMWUSE/xMt2/VZSnn/3JKKTeC9icGXiFN9/4FUQS60HAFFWDwsgaoJ/9/KEu17c12HfjVPKV2x5BI88abfj4+rxMm9iulULuAJUay4sbM03sgmMaSJWdZxDqEj6VYiS07SB1arq5saPhzoSWFMNHQEY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763130488; c=relaxed/simple; bh=w8LHtzEq1PNAXKD5/lQkltdJlt5cI5xPZVrdBhE0b0Q=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JEqGKAIj4OM0mIyd/Cao/QOvoHIV9kDAanAddf7RECGq/qiodgYpbA8oebkMJTBff7EgGgrzly7RA+eycb/1jdOSCsB5Ee+ksl09RudUH2+X11XsyvYzCz4D8TsKaK8DJmUdqZiYpVJX5Z+23tk0qeD5Wdiq3hCgg/yeHjpWKAk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=hVWh1jEL; arc=none smtp.client-ip=209.85.166.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="hVWh1jEL" Received: by mail-il1-f175.google.com with SMTP id e9e14a558f8ab-434709e7cc9so9991545ab.1 for ; Fri, 14 Nov 2025 06:28:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1763130485; x=1763735285; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=SbwiEPUMKFzWHVRkANDWXLiE9C+Y1AwxTIo0xTKShXQ=; b=hVWh1jELsMk9IdS13E5PuSZx6zgDq6+hPOUwLkzRZTk4t4LdX1OLMZSxRw0WBdSPis R1zR1ZCv4M/srcV55tjTH2kKseUXckuhyUZIAUcuusiWM8Hw+qDonhlUIq4DfjA++y3R uohfT9BZQveSgZJUQD0Kieebqr99eCVEXRG9A= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1763130485; x=1763735285; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=SbwiEPUMKFzWHVRkANDWXLiE9C+Y1AwxTIo0xTKShXQ=; b=KHQAuqzDPGKPM7oWk6DxMek08GnqAxJb2ya+Sx/YPBGzz/lYzj2k2nxealLyE68/6r S5hprbFLQCxSiexzyEEq8SrLXhzD2jmSrxfkzu0rQqLUGGISxGZjBQbrd43UIqLNX9+b ZA4Ks0XJ/1xn46kgavkvXC6F7iOe4JBZoZUideLb5gY6yr3QhDEgVgcGm9AjRv+ZOpPX OHVa0jR7UDAr6yCdcF2SWBnLfLjEqFXUyoW+fWJY3V4OUTLQKjSKhmE5UsimXvRlNi3q u6PGVdF7rqc/sCKgJswhr+7nqtn3uPkY7F6IgeUj90ylKFUSjJFeap0nnFOYI72ZsYPc 2bZw== X-Forwarded-Encrypted: i=1; AJvYcCVMt8zNL/Mu9WGy8ETC2HCdGI9T3WW0Ws3La/uIaAhI9ng5TBWWJ8jn0Nsy/uNzvEGuCYj+wu8yU3r8oJs=@vger.kernel.org X-Gm-Message-State: AOJu0YzGOXjUIZIgCRvIqGvohPP+ZBxP2+yOX+Xkzk4oQ859cAErHSsB gwfhtwpb0uk0GYaMlo5HZVta49AQKiLv0hnDyxn3C9EkXcjs2cu6xKjj5k37f7vEkA== X-Gm-Gg: ASbGnctCakW5KJ/lW6m+iVlI9Qx1RrzlfMU225wNC8jM9lBHTrnI4LOS2tMt9V+0hge vcH9z2Gf+Me49RBwSz5C1vBd+1QczA4XI2TklCZ2Ji4/GFrD4ebBYS9CfXbzY8tXP/QtEBSRvQe gmpDlJzsKAtFlnfYiSP6jCRSN2uUqR4IqsfL7YTlwoXOIFbRzgg26ArHyn2okxo1b7ahH2w16E7 VOUTDtQVPy/ppFb5XjiAFJW+0UNtgRykxbFiLmlvU456bPA/vnPA4/8GKYkgISQK6yaWRfc/fZU B3HqQx6TXps3gbtFST2NU8/b1uMkCuObRp3HgLwZUvu9x/1CaxaNOIk+f0sLlDDI3AAPOfiqM+s 9jNn8vGYhMyTc2t3tbQpJVN55+5zAd1u+opa7PZPJvpnWXKPO+vcxx50K9C+Iq52UmnJUsF3OQz QOZfwP X-Google-Smtp-Source: AGHT+IGsFSLfQsHTQySP1q0sttqWWUuCQB9YY1khd7e5giMT7AMOylsJY1uQiJSgSiyY72LImJOkkQ== X-Received: by 2002:a05:6e02:3090:b0:433:7a94:6fcc with SMTP id e9e14a558f8ab-4348c8503camr48779555ab.2.1763130485241; Fri, 14 Nov 2025 06:28:05 -0800 (PST) Received: from chromium.org ([73.34.74.121]) by smtp.gmail.com with ESMTPSA id e9e14a558f8ab-434839cdb10sm27500205ab.32.2025.11.14.06.28.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Nov 2025 06:28:04 -0800 (PST) From: Simon Glass To: linux-arm-kernel@lists.infradead.org Cc: Nicolas Schier , Masahiro Yamada , Chen-Yu Tsai , Tom Rini , Ahmad Fatoum , =?UTF-8?q?J=20=2E=20Neusch=C3=A4fer?= , Simon Glass , Nicolas Schier , linux-kernel@vger.kernel.org Subject: [PATCH v5 2/8] scripts/make_fit: Support an initial ramdisk Date: Fri, 14 Nov 2025 07:27:28 -0700 Message-ID: <20251114142741.1919072-3-sjg@chromium.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251114142741.1919072-1-sjg@chromium.org> References: <20251114142741.1919072-1-sjg@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" FIT (Flat Image Tree) allows a ramdisk to be included in each configuration. Add support for this to the script. This feature is not available via 'make image.fit' since the ramdisk likely needs to be built separately anyway, e.g. using modules from the kernel build. A later patch in this series provides support for doing that. Note that the uncompressed size is not correct when a ramdisk is used, since it is too expensive to decompress the ramdisk. Signed-off-by: Simon Glass Reviewed-by: Nicolas Schier --- Changes in v5: - Fix 'use' typo Changes in v4: - Update the commit message Changes in v3: - Add a comment at the top of the file about the -r option - Count the ramdisk in the total files - Update the commit message Changes in v2: - Don't compress the ramdisk as it is already compressed scripts/make_fit.py | 52 ++++++++++++++++++++++++++++++++++++++------- 1 file changed, 44 insertions(+), 8 deletions(-) diff --git a/scripts/make_fit.py b/scripts/make_fit.py index 0f5e7c4b8aed..984371f505bc 100755 --- a/scripts/make_fit.py +++ b/scripts/make_fit.py @@ -10,10 +10,14 @@ Usage: make_fit.py -A arm64 -n 'Linux-6.6' -O linux -o arch/arm64/boot/image.fit -k /tmp/kern/arch/arm64/boot/image.itk - @arch/arm64/boot/dts/dtbs-list -E -c gzip + -r /boot/initrd.img-6.14.0-27-generic @arch/arm64/boot/dts/dtbs-li= st + -E -c gzip =20 -Creates a FIT containing the supplied kernel and a set of devicetree files, -either specified individually or listed in a file (with an '@' prefix). +Creates a FIT containing the supplied kernel, an optional ramdisk, and a s= et of +devicetree files, either specified individually or listed in a file (with = an +'@' prefix). + +Use -r to specify an existing ramdisk/initrd file. =20 Use -E to generate an external FIT (where the data is placed after the FIT data structure). This allows parsing of the data without loading @@ -29,8 +33,6 @@ looks at the .cmd files produced by the kernel build. =20 The resulting FIT can be booted by bootloaders which support FIT, such as U-Boot, Linuxboot, Tianocore, etc. - -Note that this tool does not yet support adding a ramdisk / initrd. """ =20 import argparse @@ -81,6 +83,8 @@ def parse_args(): help=3D'Specifies the operating system') parser.add_argument('-k', '--kernel', type=3Dstr, required=3DTrue, help=3D'Specifies the (uncompressed) kernel input file (.itk)') + parser.add_argument('-r', '--ramdisk', type=3Dstr, + help=3D'Specifies the ramdisk/initrd input file') parser.add_argument('-v', '--verbose', action=3D'store_true', help=3D'Enable verbose output') parser.add_argument('dtbs', type=3Dstr, nargs=3D'*', @@ -133,7 +137,28 @@ def write_kernel(fsw, data, args): fsw.property_u32('entry', 0) =20 =20 -def finish_fit(fsw, entries): +def write_ramdisk(fsw, data, args): + """Write out the ramdisk image + + Writes a ramdisk node along with the required properties + + Args: + fsw (libfdt.FdtSw): Object to use for writing + data (bytes): Data to write (possibly compressed) + args (Namespace): Contains necessary strings: + arch: FIT architecture, e.g. 'arm64' + fit_os: Operating Systems, e.g. 'linux' + """ + with fsw.add_node('ramdisk'): + fsw.property_string('description', 'Ramdisk') + fsw.property_string('type', 'ramdisk') + fsw.property_string('arch', args.arch) + fsw.property_string('os', args.os) + fsw.property('data', data) + fsw.property_u32('load', 0) + + +def finish_fit(fsw, entries, has_ramdisk=3DFalse): """Finish the FIT ready for use =20 Writes the /configurations node and subnodes @@ -143,6 +168,7 @@ def finish_fit(fsw, entries): entries (list of tuple): List of configurations: str: Description of model str: Compatible stringlist + has_ramdisk (bool): True if a ramdisk is included in the FIT """ fsw.end_node() seq =3D 0 @@ -154,6 +180,8 @@ def finish_fit(fsw, entries): fsw.property_string('description', model) fsw.property('fdt', bytes(''.join(f'fdt-{x}\x00' for x in = files), "ascii")) fsw.property_string('kernel', 'kernel') + if has_ramdisk: + fsw.property_string('ramdisk', 'ramdisk') fsw.end_node() =20 =20 @@ -274,6 +302,14 @@ def build_fit(args): size +=3D os.path.getsize(args.kernel) write_kernel(fsw, comp_data, args) =20 + # Handle the ramdisk if provided. Compression is not supported as it is + # already compressed. + if args.ramdisk: + with open(args.ramdisk, 'rb') as inf: + data =3D inf.read() + size +=3D len(data) + write_ramdisk(fsw, data, args) + for fname in args.dtbs: # Ignore non-DTB (*.dtb) files if os.path.splitext(fname)[1] !=3D '.dtb': @@ -296,12 +332,12 @@ def build_fit(args): =20 entries.append([model, compat, files_seq]) =20 - finish_fit(fsw, entries) + finish_fit(fsw, entries, bool(args.ramdisk)) =20 # Include the kernel itself in the returned file count fdt =3D fsw.as_fdt() fdt.pack() - return fdt.as_bytearray(), seq + 1, size + return fdt.as_bytearray(), seq + 1 + bool(args.ramdisk), size =20 =20 def run_make_fit(): --=20 2.43.0 From nobody Sun Feb 8 11:16:49 2026 Received: from mail-io1-f49.google.com (mail-io1-f49.google.com [209.85.166.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DB61A326D76 for ; Fri, 14 Nov 2025 14:28:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763130489; cv=none; b=Xh8NeViIvecB89vvCWFmCruRqgSIAv8pbXpglg3Rr2AyScKMO9KWLlDbAUEgIsCix64BsCGkd7ToqNUpLkkYvSjcfRig6TB6cjaZcuclzJyeJ8CaGIHlFw3hPal6/awwx/tRvbbhD+YcOUlQ4MrY1DUoMC/jwDz1bCO1sEmAbDw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763130489; c=relaxed/simple; bh=BGaCAKlcyk5xO8jgGliNm42nPEYthddki81jtkWbq08=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=GX51VV53iyAXwX0YphWipYa5PT0d7r6b7AKdtsGy8H8DoTitQ/lGdTVdz3ZWFbOS1umQo70JBM9YHLjYU/o6wMYIZzHM7P05WxZ7OFfZAL6O7M7ra4zTC7tocl1b+OCBEdvQSte6tMFOGHsV3aDJkjrckC+7Py/aDI8/e+t5H68= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=QbpsnYMr; arc=none smtp.client-ip=209.85.166.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="QbpsnYMr" Received: by mail-io1-f49.google.com with SMTP id ca18e2360f4ac-94861514a5cso168399539f.1 for ; Fri, 14 Nov 2025 06:28:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1763130487; x=1763735287; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=t+WQ+WlElZR6b8foj+jTnRrBNTpepaWdsCyG6VTxCeY=; b=QbpsnYMrCwjh6HOQeMgyEx27+eIDeyJvKGCrwwKTbUclEcJ+yoNI+W+mGI2YyWY2aU rWzTflbpwZ5Bm3q+xJn0puhd4MoxXbZ16fwhdmCc8qrUmTLi+Ai5GR4HhxflV4Jn9Fw2 MujlnnEADq43M6fneR9qcQzHN5niXN9pAHg0w= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1763130487; x=1763735287; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=t+WQ+WlElZR6b8foj+jTnRrBNTpepaWdsCyG6VTxCeY=; b=eGiiMzHjyq2PjzGOXiqMi0RIPl1voVw9Z0dVad0JdLjgxBsQixecCoBwhQ4wcApLQY bF4YXFi62NfLG3GNOrhZ4aO51BGgaB0Rf2bhqLb5LKpggx2AEkQEj79KqIQw2cF7ISU7 uABIoP0XkxK0euETL/e8ocfW6VpFPE/cLJjoWoTEkCXKef1Gu0mioU8fDJ0c+sEFaIfp SZxIGIuhsAhGKZNxgzhYMD1xKwobZvuFe5q0jw9l6sk6mg+S+QZfiVUVhduh/Nb6953O B6fSbeEmhEHGAzvCwJ7qqBII8ScnKMDybZP02u8cd6wOg+llKXjwNMpFIIV/SCW1tl47 r/nA== X-Forwarded-Encrypted: i=1; AJvYcCWnPorhZ1mqJIEU1QIzpLkty9V48Fzr/30hq8vxZFVjOogwYiA0tOp+kHAiP57v05ImcsWZk5nhFzwZVxA=@vger.kernel.org X-Gm-Message-State: AOJu0Yz8RuxIc1tVeS9dr9SWsrpi1SvvMoppcvJjDI0y7C3Jg4lep9Rl Be2Ml4AKVGdtirgS5IAlnc8iV/0avNmju5O7yMowSqUFEkup2VMoqjE2ef6iN8KpTQ== X-Gm-Gg: ASbGncsjTfrbYTFB0v+BxqOUfveiIMSp/CMSqPy17OK/OO5T5jevJ05awWdfhG3W6hm w25KAA/v6i7gCEef98DOmQNsUtrCPo8gGXOvXJr9EWs83/z/jxEx8M6ltydTyFi4WZ08G5N5edD rndFeGnPxnpdMb2bQ4HpBbq4T9sgwoT8p4QlG6e0j+ojJ7t3xcAi6tTK9Jb7ucTW5Q8pJxfu6Ln sctvT5H3sxjAt9PiaEn87pxyOKHtrhWU1NfkgfxvZxEpFYVixtVr6PIDwbRC00tMkLdzsXKcDrQ 7Qmp1PubYs4U77ZmGhTSmnxIiEWyjPQ9cnf+6GoziVMw32m1niRVrHAj/BThX1u5pIMWcQKGKqQ 7qX0iCuuJD3owpinP269FuZIv1k2giVDKJD763F8X4WBNaFRZq1K7ktCKX1fddYzWMylD0BA2rY esSgNMvgjP/qGEruo= X-Google-Smtp-Source: AGHT+IGHyVy5O73G/VZLCysyXwbrLp5vRyiOAKVWO/iDDKm1SSBWJajVWW6I/83NMUwWoVuzzbgYUQ== X-Received: by 2002:a92:c26c:0:b0:433:8161:872c with SMTP id e9e14a558f8ab-4348c88f748mr53671975ab.1.1763130487022; Fri, 14 Nov 2025 06:28:07 -0800 (PST) Received: from chromium.org ([73.34.74.121]) by smtp.gmail.com with ESMTPSA id e9e14a558f8ab-434839cdb10sm27500205ab.32.2025.11.14.06.28.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Nov 2025 06:28:06 -0800 (PST) From: Simon Glass To: linux-arm-kernel@lists.infradead.org Cc: Nicolas Schier , Masahiro Yamada , Chen-Yu Tsai , Tom Rini , Ahmad Fatoum , =?UTF-8?q?J=20=2E=20Neusch=C3=A4fer?= , Simon Glass , Nicolas Schier , linux-kernel@vger.kernel.org Subject: [PATCH v5 3/8] scripts/make_fit: Move dtb processing into a function Date: Fri, 14 Nov 2025 07:27:29 -0700 Message-ID: <20251114142741.1919072-4-sjg@chromium.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251114142741.1919072-1-sjg@chromium.org> References: <20251114142741.1919072-1-sjg@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Since build_fit() is getting quite long, move the dtb processing into a separate function. Signed-off-by: Simon Glass Reviewed-by: Nicolas Schier --- (no changes since v1) scripts/make_fit.py | 67 +++++++++++++++++++++++++++++---------------- 1 file changed, 44 insertions(+), 23 deletions(-) diff --git a/scripts/make_fit.py b/scripts/make_fit.py index 984371f505bc..1a74a9dcd85e 100755 --- a/scripts/make_fit.py +++ b/scripts/make_fit.py @@ -277,6 +277,47 @@ def process_dtb(fname, args): =20 return (model, compat, files) =20 + +def _process_dtbs(args, fsw, entries, fdts): + """Process all DTB files and add them to the FIT + + Args: + args: Program arguments + fsw: FIT writer object + entries: List to append entries to + fdts: Dictionary of processed DTBs + + Returns: + tuple: + Number of files processed + Total size of files processed + """ + seq =3D 0 + size =3D 0 + for fname in args.dtbs: + # Ignore non-DTB (*.dtb) files + if os.path.splitext(fname)[1] !=3D '.dtb': + continue + + try: + (model, compat, files) =3D process_dtb(fname, args) + except Exception as e: + sys.stderr.write(f'Error processing {fname}:\n') + raise e + + for fn in files: + if fn not in fdts: + seq +=3D 1 + size +=3D os.path.getsize(fn) + output_dtb(fsw, seq, fn, args.arch, args.compress) + fdts[fn] =3D seq + + files_seq =3D [fdts[fn] for fn in files] + entries.append([model, compat, files_seq]) + + return seq, size + + def build_fit(args): """Build the FIT from the provided files and arguments =20 @@ -289,7 +330,6 @@ def build_fit(args): int: Number of configurations generated size: Total uncompressed size of data """ - seq =3D 0 size =3D 0 fsw =3D libfdt.FdtSw() setup_fit(fsw, args.name) @@ -310,34 +350,15 @@ def build_fit(args): size +=3D len(data) write_ramdisk(fsw, data, args) =20 - for fname in args.dtbs: - # Ignore non-DTB (*.dtb) files - if os.path.splitext(fname)[1] !=3D '.dtb': - continue - - try: - (model, compat, files) =3D process_dtb(fname, args) - except Exception as e: - sys.stderr.write(f"Error processing {fname}:\n") - raise e - - for fn in files: - if fn not in fdts: - seq +=3D 1 - size +=3D os.path.getsize(fn) - output_dtb(fsw, seq, fn, args.arch, args.compress) - fdts[fn] =3D seq - - files_seq =3D [fdts[fn] for fn in files] - - entries.append([model, compat, files_seq]) + count, fdt_size =3D _process_dtbs(args, fsw, entries, fdts) + size +=3D fdt_size =20 finish_fit(fsw, entries, bool(args.ramdisk)) =20 # Include the kernel itself in the returned file count fdt =3D fsw.as_fdt() fdt.pack() - return fdt.as_bytearray(), seq + 1 + bool(args.ramdisk), size + return fdt.as_bytearray(), count + 1 + bool(args.ramdisk), size =20 =20 def run_make_fit(): --=20 2.43.0 From nobody Sun Feb 8 11:16:49 2026 Received: from mail-io1-f49.google.com (mail-io1-f49.google.com [209.85.166.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 44FBE327213 for ; Fri, 14 Nov 2025 14:28:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763130492; cv=none; b=ZZvGgsDiG7fmq7esImZuJOEmGzF8MRKiNfDoSVy2ad8+uDxBdLrLXKWBVHYt3a+lSIc+lyR8JGw28I0XXIsq3a2iXlIh43iLZnhHeMQLQ5gI2kprZHzk1keNJEyNhgt2cSCU0eiq85m0oI09cJyBPrHtXxMzkhcT2KtoCL7aLTo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763130492; c=relaxed/simple; bh=zBAjQh9JCzwABFilcP24spbYzKjpbpx0VEEAWxnlGS0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BjyMk3NfcqDQpo3LsQyCgJC2qC8PkqDmPxhcOy5xut1sK6uJVo9ZmsRlpgDmwvOIRB81S8WoSW5pcay8F0f0LnH5lRHsXRwZ1FPQPPFeATDb01rX+kb0KUE3KpW8GGhHlsvwV3EMVm/QM5TzERXjR3n+jKFb3aQiI/vzpZdPpmo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=dQrDanze; arc=none smtp.client-ip=209.85.166.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="dQrDanze" Received: by mail-io1-f49.google.com with SMTP id ca18e2360f4ac-9489dc9845dso81363839f.0 for ; Fri, 14 Nov 2025 06:28:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1763130489; x=1763735289; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0Kcz22F2HSffVxv8SHD4ALbiSJ858QNwAp4RJw9tXbw=; b=dQrDanzeanmNIP5kFNS23XhSOfrUgLK/M+7emVniRQYgkqSrexkWXwXihzOsUilywY yhrXAMpnG20jcyUdzvxYDx2wD/4FyoBd9fmDJVqsaMuw9nS2JQcpMSNRkdCNFWOcdpv7 Dfi12c7vB4FE3f7zytlhQdd2OSm2SUUPg3iNo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1763130489; x=1763735289; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=0Kcz22F2HSffVxv8SHD4ALbiSJ858QNwAp4RJw9tXbw=; b=AzkUn11J1rLEsD4RjkAZDQukyt841H4+XuZzp5/eRn8VBzShiKKF+JEAZaTUuZoFhR l3uez6MuejS1tUoiGL7XLuiqYzRHtbAu4+vGQk1U00XSTA9PmF5fAWnBG+4Fr4slLyan ptyrjz8o7LkYJfKVLqb/ecSCz69qJ0gkIAKcxR4tXk+6aauJRSP9i7jE/LrMHpLfMnnk Veo5oGLglaOc5Y0A4LdjAQAtoO7K29VT4y8jeny1QWplubSsxZsefkaJpob28RGL4i51 zfI1vh65bRa40TV8/0BVOVI4Ydl0tQRjlBWo/VjPTVsFOkJzmEC/1Q3/AXcUolJv2TbT HdTg== X-Forwarded-Encrypted: i=1; AJvYcCVcxtUHdV8oOW+N0W0/V5pDgCeSBlZtrKqwzKWQYhpKZa5RJBjAaU+38X8VaJCBDicKoKu0pgW/gzpZbzU=@vger.kernel.org X-Gm-Message-State: AOJu0Yzdky4Hk4LVQqFu/wq05Eg/r6kJptl6nXZiBs4JLrDWAHNv0QyA wAnCwbSCxOI4HEsRKbA8sjBZ8Mu2rWDCpOYlDQ/Rwhv8ozcNHdg0oAmeckGCb3vqTg== X-Gm-Gg: ASbGncsVm1uUcPrdXe+oDagk0QcxhKdAPtMvSsbykTiF53DClDNNwliXd8CU/k64HtR Nm7LaQyb3JOPTE/T7U2XNJKl0xSX21avVbohCoz2OgEpZwZP7QdWRZjBosEy7rvye1yX42AlR/Q xfXba5/XI5zCH+peuoxxorBt0MFymA5HFGQOLgAApVZGA8LOGsuW20t6yO+ch2VPoD7lEUg8EuO 0HhRxflRSGpMVF+WVdjPsQVjxIe1xWK6qArfVhE9NDHWFgwvTr7s00FQIZugPaB5yWQg5r/afzv 8sil3nzKSycKv6UL5IX1YBU2w8JQWE4oscSUS4q0ls4omVijIHDNLZxfGkqxGeWTKFrex2VMw8I RoNOJJYTT0cjX1lZzR5gQlO5wGVifC1lBb9kVXfsOwKxhjjagIrrYCOi05e71TwUjmOrfM68u9Y M6ApQD X-Google-Smtp-Source: AGHT+IFHOZJj2lH8HRzO4I7qi0KILDpyFlaNnMyv9yoDbXQXY3BIaEqV9liLUCD0alvtY81aHaZwsg== X-Received: by 2002:a05:6e02:258b:b0:434:6ef2:a43d with SMTP id e9e14a558f8ab-4348c938f48mr42073985ab.19.1763130489362; Fri, 14 Nov 2025 06:28:09 -0800 (PST) Received: from chromium.org ([73.34.74.121]) by smtp.gmail.com with ESMTPSA id e9e14a558f8ab-434839cdb10sm27500205ab.32.2025.11.14.06.28.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Nov 2025 06:28:07 -0800 (PST) From: Simon Glass To: linux-arm-kernel@lists.infradead.org Cc: Nicolas Schier , Masahiro Yamada , Chen-Yu Tsai , Tom Rini , Ahmad Fatoum , =?UTF-8?q?J=20=2E=20Neusch=C3=A4fer?= , Simon Glass , Nicolas Schier , linux-kernel@vger.kernel.org Subject: [PATCH v5 4/8] scripts/make_fit: Provide a way to add built modules Date: Fri, 14 Nov 2025 07:27:30 -0700 Message-ID: <20251114142741.1919072-5-sjg@chromium.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251114142741.1919072-1-sjg@chromium.org> References: <20251114142741.1919072-1-sjg@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Provide arguments to support building a ramdisk from a directory tree of modules. This is a convenient way to try out a kernel with its modules. This makes use of the cpio tool rather than attempting to use a python module or our own code. The list of modules is provided in a file. Signed-off-by: Simon Glass Reviewed-by: Nicolas Schier Suggested-by: Ahmad Fatoum --- (no changes since v4) Changes in v4: - Provide the list of modules from the Makefile - Reduce verbosity (don't print every module filename) Changes in v3: - Add a way to add built modules into the FIT scripts/make_fit.py | 98 +++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 90 insertions(+), 8 deletions(-) diff --git a/scripts/make_fit.py b/scripts/make_fit.py index 1a74a9dcd85e..3db129f40b20 100755 --- a/scripts/make_fit.py +++ b/scripts/make_fit.py @@ -13,11 +13,17 @@ Usage: -r /boot/initrd.img-6.14.0-27-generic @arch/arm64/boot/dts/dtbs-li= st -E -c gzip =20 + # Build with modules ramdisk instead of external ramdisk: + make_fit.py -A arm64 -n 'Linux-6.17' -O linux + -o arch/arm64/boot/image.fit -k /tmp/kern/arch/arm64/boot/image.itk + -m module1.ko module2.ko module3.ko @arch/arm64/boot/dts/dtbs-list + Creates a FIT containing the supplied kernel, an optional ramdisk, and a s= et of devicetree files, either specified individually or listed in a file (with = an '@' prefix). =20 Use -r to specify an existing ramdisk/initrd file. +Use -m to build a ramdisk from specified kernel module files. =20 Use -E to generate an external FIT (where the data is placed after the FIT data structure). This allows parsing of the data without loading @@ -38,6 +44,7 @@ as U-Boot, Linuxboot, Tianocore, etc. import argparse import collections import os +import shutil import subprocess import sys import tempfile @@ -83,8 +90,14 @@ def parse_args(): help=3D'Specifies the operating system') parser.add_argument('-k', '--kernel', type=3Dstr, required=3DTrue, help=3D'Specifies the (uncompressed) kernel input file (.itk)') - parser.add_argument('-r', '--ramdisk', type=3Dstr, + + # Create mutually exclusive group for ramdisk options + rd_group =3D parser.add_mutually_exclusive_group() + rd_group.add_argument('-r', '--ramdisk', type=3Dstr, help=3D'Specifies the ramdisk/initrd input file') + rd_group.add_argument('-m', '--modules', type=3Dstr, nargs=3D'+', + help=3D'List of module filenames to include in ramdisk') + parser.add_argument('-v', '--verbose', action=3D'store_true', help=3D'Enable verbose output') parser.add_argument('dtbs', type=3Dstr, nargs=3D'*', @@ -240,6 +253,60 @@ def output_dtb(fsw, seq, fname, arch, compress): fsw.property('data', compressed) =20 =20 +def build_ramdisk(args, tmpdir): + """Build a cpio ramdisk containing kernel modules + + Similar to mkinitramfs, this creates a compressed cpio-archive contain= ing + the kernel modules for the current kernel version. + + Args: + args (Namespace): Program arguments + tmpdir (str): Temporary directory to use for modules installation + + Returns: + tuple: + bytes: Compressed cpio data containing modules + int: total uncompressed size + """ + suppress =3D None if args.verbose else subprocess.DEVNULL + + if args.verbose: + print(f'Copying {len(args.modules)} modules to ramdisk') + + # Create output-directory structure + outdir =3D os.path.join(tmpdir, 'initramfs') + modules_dir =3D os.path.join(outdir, 'usr', 'lib', 'modules') + os.makedirs(modules_dir, exist_ok=3DTrue) + + # Copy in the specified modules + for module in args.modules: + dest_path =3D os.path.join(modules_dir, os.path.basename(module)) + shutil.copy2(module, dest_path) + + if args.verbose: + print(f'Creating cpio archive from {outdir}') + + with tempfile.NamedTemporaryFile() as cpio_file: + # Change to initramfs directory and create cpio archive + with subprocess.Popen(['find', '.', '-print0'], cwd=3Doutdir, + stdout=3Dsubprocess.PIPE) as find: + with subprocess.Popen(['cpio', '-o', '-0', '-H', 'newc'], + stdin=3Dfind.stdout, stdout=3Dcpio_file, + stderr=3Dsuppress, cwd=3Doutdir) as cpio: + find.stdout.close() + cpio.wait() + find.wait() + + if cpio.returncode !=3D 0: + raise RuntimeError('Failed to create cpio archive') + + cpio_file.seek(0) # Reset to beginning for reading + if args.verbose: + print('Reading ramdisk...' if args.compress =3D=3D 'none' else + f'Compressing ramdisk with {args.compress}...') + return compress_data(cpio_file, args.compress), cpio_file.tell() + + def process_dtb(fname, args): """Process an input DTB, decomposing it if requested and is possible =20 @@ -318,11 +385,12 @@ def _process_dtbs(args, fsw, entries, fdts): return seq, size =20 =20 -def build_fit(args): +def build_fit(args, tmpdir): """Build the FIT from the provided files and arguments =20 Args: args (Namespace): Program arguments + tmpdir (str): Temporary directory for any temporary files =20 Returns: tuple: @@ -344,20 +412,29 @@ def build_fit(args): =20 # Handle the ramdisk if provided. Compression is not supported as it is # already compressed. + ramdisk_data =3D None if args.ramdisk: with open(args.ramdisk, 'rb') as inf: - data =3D inf.read() - size +=3D len(data) - write_ramdisk(fsw, data, args) + ramdisk_data =3D inf.read() + size +=3D len(ramdisk_data) + elif args.modules: + if args.verbose: + print('Building modules ramdisk...') + ramdisk_data, uncomp_size =3D build_ramdisk(args, tmpdir) + size +=3D uncomp_size + + if ramdisk_data: + write_ramdisk(fsw, ramdisk_data, args) =20 count, fdt_size =3D _process_dtbs(args, fsw, entries, fdts) size +=3D fdt_size =20 - finish_fit(fsw, entries, bool(args.ramdisk)) + finish_fit(fsw, entries, has_ramdisk=3Dbool(ramdisk_data)) =20 - # Include the kernel itself in the returned file count fdt =3D fsw.as_fdt() fdt.pack() + + # Count FDT files, kernel, plus ramdisk if present return fdt.as_bytearray(), count + 1 + bool(args.ramdisk), size =20 =20 @@ -365,7 +442,12 @@ def run_make_fit(): """Run the tool's main logic""" args =3D parse_args() =20 - out_data, count, size =3D build_fit(args) + tmpdir =3D tempfile.mkdtemp(prefix=3D'make_fit_') + try: + out_data, count, size =3D build_fit(args, tmpdir) + finally: + shutil.rmtree(tmpdir) + with open(args.output, 'wb') as outf: outf.write(out_data) =20 --=20 2.43.0 From nobody Sun Feb 8 11:16:49 2026 Received: from mail-il1-f173.google.com (mail-il1-f173.google.com [209.85.166.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3328932AAA8 for ; Fri, 14 Nov 2025 14:28:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763130493; cv=none; b=jLPoNNbkSjSWF6fpKxP7XvGR6sBsrJ88zRj7AHCx5jaSyQMEexK3zHV37w1lhecqq/HdHRNuJzMYafJ3Ff2MkK7FiKmI5Roo0BbfcVCK4qNH8onWn5FeiJgmN+HISBWx0wb5DdpPmL9pBa7Dh8YFDk4zyuVN4N9YiuBrhqM9E+A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763130493; c=relaxed/simple; bh=vtbE4iQccIDPAbt51GtKugnCRkvsnDiu597g0q1p1vA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=O93GOdg3CXdqs9a8CkFm/2+n5ebIkEUPAwOpVrepdIa1W6SM6ZDOMwMGmXs+gJLt8RUEv+4Pn/2doT74ZATIm/lVmeMwVuxA/lOsG2+E/o/5cMcNLc1YBBLHn+0ItczDMag7++9qB905ec+++TboU5FYHYk+QtsS5GEMvhMnJ5E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=J1Ttlg0N; arc=none smtp.client-ip=209.85.166.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="J1Ttlg0N" Received: by mail-il1-f173.google.com with SMTP id e9e14a558f8ab-4337076ae3fso9080685ab.1 for ; Fri, 14 Nov 2025 06:28:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1763130491; x=1763735291; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=g04XP369OdIGlU7nO8ZhA20Z4uYWbY8J/2UrhgEgmF8=; b=J1Ttlg0N6LKJN8NnZbsH7QqIhk3l6+pthl7S8qvWG1xbUXVbdhnmcWYHXkHTNUNlyO myNeB0dhzIOEvZPQtUD6c2brUcF2+QpBXFJxsdE8N64518kKe6nT9J6Th+l4qSkxveGT Q170jXhmSOP6fjw7Yu8W3GqRAYil3u+TYAiWA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1763130491; x=1763735291; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=g04XP369OdIGlU7nO8ZhA20Z4uYWbY8J/2UrhgEgmF8=; b=EUhask3UNhRee4YHz4+qC8s83LPD634Tex7lXuJnPSxcnphDLKWxHJZFr3IKgXh1da /N/zeYMabSscCUNC7wdgpwFFPAulPE3wbV3mMlK5IfFig7cSsE8SWSJ3udqmTpE2rHed mGtzHWKz8MCD0A63S+mXVw/NBP1/87QZxxtv4zMxd0B/jdi3wOMRrrm0+CD0d2kQdjMe S/iPPKxN3vkap33tNh45A5PP++97O/+e0IZSdDITZnzRC9bps5dfxRdSKOHbvDzr6qiA L647kFXgJKHHp7Q7NwC5Xmxgw6GYM/bdrCNagEDFBt9HzMlXu/rdGLf9yG7Fuuh4y3+F j+Xw== X-Forwarded-Encrypted: i=1; AJvYcCXngl0Gs+vJp0fevZjhIdIVCNNmjE8yr5lR7IS1DR5U+SfKD6NM0dNT9RdCwK/w/TIa12GnOoWp41TZamw=@vger.kernel.org X-Gm-Message-State: AOJu0Yz51sT0QLTNkVfG3bzMnFzQmXyvGyDaudbnApyQn9c9yTcRDGdL H1TDMq2EjIZEt/ee7B+Gy7kiHA5GQKWjdkSrJThN47NwPl6WZ34EQvqJRaF++bODKg== X-Gm-Gg: ASbGnctxo66rYqU8PN6lwd3ENVZCkGjeyRSQXwWukWcjQY39xDFailRUothuya4hrnq wwP8Qmnrsw7T7kg4AWClnQb9R8iZ8wo4UzMA0GeBFTcLBPwRZUgYh7u6Ab2iaAEnVowdTiVdjgJ QNmyvfnjJISFyxDoXVS+n/4t0R9YQ5n81j35Qk0b04zooWnPEyCCL5bPaJiNuu8z8BGk2wcnl4g LLDxnV3lUdacdG6mX/jfYUV5nBeuK9dt4TLdz+/F+8CAQ/ePUfcpPG432Owvej2xnXzT7BOHBfY aTm0AG2qs48WPD999QEggQ8jSGWRJZjaZa772654RtTKlcTFTlz17SwJfK+SiDVA54gnvuFAPWz Ye6xvlaDVzCIvomd7npmphorZLI3kJsOuJsN+dhrdxypx5V868H9mufTXrx8bED1Mleeayitb8q csPwL9nXNF8bB0J/E= X-Google-Smtp-Source: AGHT+IH9WRXDGpQxQ/K/6unFcP6iH5jd3lNOXJwjoIeZqn0+/6i1yonIkDQHVVuUwWSwfsG4BW0P5Q== X-Received: by 2002:a05:6e02:2389:b0:433:330a:a572 with SMTP id e9e14a558f8ab-4348c8e3e5amr60110435ab.13.1763130491388; Fri, 14 Nov 2025 06:28:11 -0800 (PST) Received: from chromium.org ([73.34.74.121]) by smtp.gmail.com with ESMTPSA id e9e14a558f8ab-434839cdb10sm27500205ab.32.2025.11.14.06.28.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Nov 2025 06:28:10 -0800 (PST) From: Simon Glass To: linux-arm-kernel@lists.infradead.org Cc: Nicolas Schier , Masahiro Yamada , Chen-Yu Tsai , Tom Rini , Ahmad Fatoum , =?UTF-8?q?J=20=2E=20Neusch=C3=A4fer?= , Simon Glass , Bill Wendling , Justin Stitt , Miguel Ojeda , Nathan Chancellor , Nick Desaulniers , Nicolas Schier , Tamir Duberstein , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, llvm@lists.linux.dev Subject: [PATCH v5 5/8] kbuild: Split out module targets into a variable Date: Fri, 14 Nov 2025 07:27:31 -0700 Message-ID: <20251114142741.1919072-6-sjg@chromium.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251114142741.1919072-1-sjg@chromium.org> References: <20251114142741.1919072-1-sjg@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a modules-targets variable to list the targets which cause modules to be built, since we want to add a conditional target. Signed-off-by: Simon Glass --- Changes in v5: - Add a new patch to split out module targets into a variable Makefile | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/Makefile b/Makefile index 638bc09a546a..8cd46222fc48 100644 --- a/Makefile +++ b/Makefile @@ -772,7 +772,12 @@ endif # in addition to whatever we do anyway. # Just "make" or "make all" shall build modules as well =20 -ifneq ($(filter all modules nsdeps compile_commands.json clang-%,$(MAKECMD= GOALS)),) +modules-targets :=3D all +modules-targets +=3D modules +modules-targets +=3D nsdeps +modules-targets +=3D compile_commands.json +modules-targets +=3D clang-% +ifneq ($(filter $(modules-targets),$(MAKECMDGOALS)),) KBUILD_MODULES :=3D y endif =20 --=20 2.43.0 From nobody Sun Feb 8 11:16:49 2026 Received: from mail-io1-f42.google.com (mail-io1-f42.google.com [209.85.166.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2305B328267 for ; Fri, 14 Nov 2025 14:28:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763130496; cv=none; b=n6UUTbPKjkrn/Mbooxy3pJJmsKCQjoP/+/h2/J28awXqHd2KmVEBioDX5GGLNHnWEboXJmc8dE1w3s1ujSJLphZHmvWIwoEhClfPLovJpiFLY7sI2eGdRFc0QKCrSa1/yawv5RHkppViVw2WSADXYe0KT23B2c3aHGrbVonFvWQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763130496; c=relaxed/simple; bh=Khn8xl78nWKbQ9qiwRfPIiqdd7MMix5aw37wtwcBeyI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BCB0gCqFVtxDc/VrWgLSxnfXYkqGRpXgTnmYI9d2pb7mjZM95Ap2lpK2loMVKonMUKpbAr0bkT34BOoiQWRCQQI5M7N+m+hJ54u0vWXiUO+XOhDL18qLhUVk9l6/ZAiazs2LMPUJ1yozeig/+v+IZd8xRoLQOie/+H80w+7mFO8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=RFk+7/dr; arc=none smtp.client-ip=209.85.166.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="RFk+7/dr" Received: by mail-io1-f42.google.com with SMTP id ca18e2360f4ac-948673fdc47so90641439f.3 for ; Fri, 14 Nov 2025 06:28:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1763130494; x=1763735294; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zlW9sVY3dHmO2QQaQTwq9Tqd8agWgn9nVM7+PnbVmYo=; b=RFk+7/drKj+JB0Ads7dZwQTeuDy1Ftbs8h3h2NOUILT16EiIhIUCU/hgV6V2keVPvC ZYvDJ3BqUduZgHxpy60Qem4QVY/iP899m2Tg6yYYKtjCky+YPWxmPTxuD1B4OWdk4o52 A5qcK75FpYC4pHTuegbJOpyfNLM8Ac9W5fWJ0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1763130494; x=1763735294; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=zlW9sVY3dHmO2QQaQTwq9Tqd8agWgn9nVM7+PnbVmYo=; b=SErhHeEnRa27gJ5ESDHOjwuz2iyizOi72nb6oYtfEaThKT/MmaDzoC0xV+YRRPJDYQ wkt/Ifwfa95oWuN38dFRVkR5IvIu5vv2mEz0rHmrHkmEem/wlkxxTu3U7wmD76K1ftz8 YJX/tWnfSLI8haLtclLPTsXbQ9EY6Y/+1RmDNuSzKvc21Y8rmILJvIqmEFqOEMux7NzT OPgRzJdvMIeGeTh7/Bu+ETKDmqJjcQ32YX25j4z4kGjZNuPzFm89MWbip1EMDTuu2w4v gI046u8CmdoQzW6hoQISdZZqtUfOc2xZ1pwik/oe8bkQiBsliVF///h0WTqQ3XCkgZmm Tf1A== X-Forwarded-Encrypted: i=1; AJvYcCWSiz+RoMT+4DI4X43ZEtYQ5xtCBVNrYdPYmO27eqvhN5qs/BVo2tojdmIAtlEfUKYeujB1gjJEjklBLnQ=@vger.kernel.org X-Gm-Message-State: AOJu0YyEEC/0oq9VsSjuCepRARSRcZDGTdC4yrceQT3S2f19KYAzeGau 9jkIDAO1Kg97pXxkR2DD+a1oML+T4leYq8GKa7Bv69vqJEya4vFl9hPmox+deBqaTQ== X-Gm-Gg: ASbGncvYyN8Ba/5biglZ2D/8t/UiYbWvwOGucQc8xETnVsCtfS5shU4Q2TQ2ozjLMKN YuPVvVl/bbi9JNFga2akqjrhfP7EUivMht0fKBXzJ7SuVHi0WwBdDFW5WGJxGmdFq5HfAvUXhGC kA3bmAgwZBDRGK2p3fZ5vsdxNlfvyGzsQsa5h+qP5POG6ATvOxQZCF7S5bzLVJnmBYO+UOATh6M F4fA+tNFhZFLLSjlavuHFkMMiXlgt1Rcpe53z9tyCR/ZECxmcLkEM5Um96TKEdU2iD4N8jl8oom 2J68HxKzUwCY67L8tlKXjgFs32bv0AS8qeieHcT+iShEgUDx4VnP209nPGgKI24yHqlzB0Mqdnd ojharbmXM/9dyPR+0rC7I4yV6hlRyEncBOjr/kSIPJE6qjflevxgNE+22Uo3MfKAhmq0RT23kFj ipPWQfkgVJSeunEwc= X-Google-Smtp-Source: AGHT+IHtlD8phUcrsxxNZnSEeRERBORiCvKi70lR0tOOd7wFG1c+yfIKHGrMPWetfRl4tu7NCPgRrw== X-Received: by 2002:a05:6e02:1523:b0:433:713d:a288 with SMTP id e9e14a558f8ab-4348c94ad9emr53176765ab.32.1763130494081; Fri, 14 Nov 2025 06:28:14 -0800 (PST) Received: from chromium.org ([73.34.74.121]) by smtp.gmail.com with ESMTPSA id e9e14a558f8ab-434839cdb10sm27500205ab.32.2025.11.14.06.28.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Nov 2025 06:28:12 -0800 (PST) From: Simon Glass To: linux-arm-kernel@lists.infradead.org Cc: Nicolas Schier , Masahiro Yamada , Chen-Yu Tsai , Tom Rini , Ahmad Fatoum , =?UTF-8?q?J=20=2E=20Neusch=C3=A4fer?= , Simon Glass , "Reviewed-by : Nicolas Schier" , Nathan Chancellor , Ard Biesheuvel , Catalin Marinas , Josh Poimboeuf , Kees Cook , Miguel Ojeda , Nicolas Schier , Parth Pancholi , Rong Xu , Tamir Duberstein , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , Will Deacon , linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 6/8] kbuild: Allow adding modules into the FIT ramdisk Date: Fri, 14 Nov 2025 07:27:32 -0700 Message-ID: <20251114142741.1919072-7-sjg@chromium.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251114142741.1919072-1-sjg@chromium.org> References: <20251114142741.1919072-1-sjg@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Support 'make image.fit FIT_MODULES=3D1' to put all the modules into a ramdisk image within the FIT. Add image.fit as a target which requires modules, so that modules will built automatically when using FIT_MODULES=3D1 Signed-off-by: Simon Glass Suggested-by: Ahmad Fatoum Suggested-by: Reviewed-by: Nicolas Schier Acked-by: Nathan Chancellor --- Changes in v5: - Build modules automatically if needed (fix from Nicolas Schier) Changes in v4: - Rename the Makefile variable from 'EXTRA' to 'MAKE_FIT_FLAGS' - Use an empty FIT_MODULES to disable the feature, instead of '0' - Make use of the 'modules' dependency to ensure modules are built - Pass the list of modules to the script Makefile | 1 + arch/arm64/Makefile | 1 + scripts/Makefile.lib | 10 ++++++++-- 3 files changed, 10 insertions(+), 2 deletions(-) diff --git a/Makefile b/Makefile index 8cd46222fc48..4eccaef95826 100644 --- a/Makefile +++ b/Makefile @@ -773,6 +773,7 @@ endif # Just "make" or "make all" shall build modules as well =20 modules-targets :=3D all +modules-targets +=3D $(if $(FIT_MODULES),image.fit) modules-targets +=3D modules modules-targets +=3D nsdeps modules-targets +=3D compile_commands.json diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile index 73a10f65ce8b..7036f251ab40 100644 --- a/arch/arm64/Makefile +++ b/arch/arm64/Makefile @@ -174,6 +174,7 @@ endif all: $(notdir $(KBUILD_IMAGE)) =20 image.fit: dtbs +image.fit: $(if $(FIT_MODULES),modules) =20 vmlinuz.efi image.fit: Image $(BOOT_TARGETS): vmlinux diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib index 1d581ba5df66..c6a3aa653035 100644 --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -398,11 +398,17 @@ MAKE_FIT :=3D $(srctree)/scripts/make_fit.py # Use this to override the compression algorithm FIT_COMPRESSION ?=3D gzip =20 +# Set this to 1 to include an initrd with all the kernel modules +FIT_MODULES ?=3D + quiet_cmd_fit =3D FIT $@ - cmd_fit =3D $(MAKE_FIT) -o $@ --arch $(UIMAGE_ARCH) --os linux \ - --name '$(UIMAGE_NAME)' \ + cmd_fit =3D $(if $(FIT_MODULES), \ + find $(objtree) -name '*.ko' > $(objtree)/.modules-list 2>/dev/null &&) \ + $(MAKE_FIT) -o $@ --arch $(UIMAGE_ARCH) --os linux \ + --name '$(UIMAGE_NAME)' $(MAKE_FIT_FLAGS) \ $(if $(findstring 1,$(KBUILD_VERBOSE)),-v) \ $(if $(FIT_DECOMPOSE_DTBS),--decompose-dtbs) \ + $(if $(FIT_MODULES),--modules @$(objtree)/.modules-list) \ --compress $(FIT_COMPRESSION) -k $< @$(word 2,$^) =20 # XZ --=20 2.43.0 From nobody Sun Feb 8 11:16:49 2026 Received: from mail-il1-f178.google.com (mail-il1-f178.google.com [209.85.166.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5C51832C93D for ; Fri, 14 Nov 2025 14:28:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763130498; cv=none; b=f3hDc3MCSWvSKZbispfSlsWFf+Ye1E7fa0aIkYR3+VKoDlPJOjpRbumfYHeV4iGVwJUTr0ETrq21SDJ10UOPuNr85/GNTYYKWF/u5rHZbpwIs1RHGxYuU+VlcB2FgrrpMQ/9WWxO8LBOh2rMNTjkxEQ/iKh2Y3s0vA18H2TLVQY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763130498; c=relaxed/simple; bh=8ZGGs3PQhqVTmNk+d9LVkL248C8uJ8HMmqu52Ln87QA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=SU5lsYo+2I3+RQ0S8UlU9Jfqb2NCq+1p4Pmalh1vcxqCUZPSeLhrMUY5HoSs0Dz/iMUPxxcVqjFxveE6K80aKtt5j8z3GFfK7L0DNHJCZD4muAmNhPUWdIdv24da/Zg07tcC8Iz27pQXaja48XYaDjtzKaFF46RN8J8X5dOUnxQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=NyNaNqKL; arc=none smtp.client-ip=209.85.166.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="NyNaNqKL" Received: by mail-il1-f178.google.com with SMTP id e9e14a558f8ab-433770ba959so16225665ab.1 for ; Fri, 14 Nov 2025 06:28:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1763130496; x=1763735296; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MuN93K9Dcy3JOdIK4h07lOL7a8R3BcxISOfJinfzFRA=; b=NyNaNqKLFs4J3+7go6jer3qbr4aDxazAyjbO3G0Pf94Xac70T2IgwVdgaPzDS0//dU eEI8Y0/W4EiYF5ykyu6ZMNNibuSsSIcqMpCu8BZ4Plr6NIrdAmM6aM5pVq8Qj+mp8Dbj u2FxYS6tc+mV+3iz2m37V1xQ0Mljgl2nLnUfg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1763130496; x=1763735296; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=MuN93K9Dcy3JOdIK4h07lOL7a8R3BcxISOfJinfzFRA=; b=NNi6nH2JG1tXcc1oldpk6FA956+ECdAnx3gy001vuzGtDp8UtQlkmUD4x77ytqz+/g hR6uHg2zWHpFaEt2iP6zq1t/PJco6/68yTR3heWdr8D96hdu1T4hDh8bQshZ5LF4QwqA wDx3V4MGJQosPSbgDri1We3/LJeBEpaot2KJTLNghiOzQZhMVvQ+oOBh7GgDT1bomzD0 r0q/M/YV3nXifvHKiw6EXbmRFCjaoeiT48oC4rzrmKxjFBN5CbibeF+V/nwDYg64fDcE DqYPOBiofRLDA4CLbpzbjnVhBI3/Hyztd8WfKs4zIaQTgqNkVpKoZzemI6ATUZ9zvCi1 f0AA== X-Forwarded-Encrypted: i=1; AJvYcCXcXKgR0OE6+yMjt6KU525lAMjMzJMpsdKyGXHGCgoZllfR+FAlrJUu8Nr9o47ClOOXmeE+lKIAjazfd4w=@vger.kernel.org X-Gm-Message-State: AOJu0YwoLv6xAJ25hYZFtE4xDplgmu+xZ1pHvangN/WohrkUfH/Z2kUe m8VnUxJa65HHiSInea1aKXbxP3ihNzM3JXk1OeQDiS1qmrNdOUIxS5Nw3GwjITnSbg== X-Gm-Gg: ASbGnctuYsOtvngbtS7wfLlivjJUItxIk+wYDYOlVJYFUoZLEAP/ujdHqefqPJ70G1o RFA3Z/5hpxsQ5Iw14j6kZdZZ5dDJLGkv0q/P4p3Xeqpr+gdGls6Jmf+Y5cusKXwBn2jdkUYEvJT HDdXAY7tntiBV52Bxt7x/34ZVKBBJ3AvslntDyjf8JFUVig8+20ChjrjW1jpKNWJ/CXIFn/pnxG xZmnybS3PpYmoNNR1FCsCoyBUdREBQ+ZGvqTZhAs3QVURYN8M9MA1tpY1VCPERYUlwKOqZ64CZe f1N/v8hGlhd+mx0pkyV8+6IcxWfXfC9jMT20rw19bt5VKGcjqHjoPrL6N3u8wU+CECAe2ZXfDmu aONU9QTvtDxmBoI093u83PhyxeJAFf5J5VUDOiTwfh/3lSNVBV0FLwTM+wl1ONCa6JI+yAuFnT8 G7C8ne X-Google-Smtp-Source: AGHT+IF5K+GTrsGobgCXU6IqK3u5VkD77G/MeCEjJbWOyc35yy8/I+punAVonCr1oOz6r5VuSD8dEA== X-Received: by 2002:a05:6e02:170e:b0:431:da08:d652 with SMTP id e9e14a558f8ab-4348c7d3099mr51311895ab.0.1763130496474; Fri, 14 Nov 2025 06:28:16 -0800 (PST) Received: from chromium.org ([73.34.74.121]) by smtp.gmail.com with ESMTPSA id e9e14a558f8ab-434839cdb10sm27500205ab.32.2025.11.14.06.28.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Nov 2025 06:28:15 -0800 (PST) From: Simon Glass To: linux-arm-kernel@lists.infradead.org Cc: Nicolas Schier , Masahiro Yamada , Chen-Yu Tsai , Tom Rini , Ahmad Fatoum , =?UTF-8?q?J=20=2E=20Neusch=C3=A4fer?= , Simon Glass , David Sterba , Nick Terrell , linux-kernel@vger.kernel.org Subject: [PATCH v5 7/8] scripts/make_fit: Support a few more parallel compressors Date: Fri, 14 Nov 2025 07:27:33 -0700 Message-ID: <20251114142741.1919072-8-sjg@chromium.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251114142741.1919072-1-sjg@chromium.org> References: <20251114142741.1919072-1-sjg@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add support for pbzip2 and plzip which can compress in parallel. This speeds up the ramdisk compression. Signed-off-by: Simon Glass --- (no changes since v1) scripts/make_fit.py | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/scripts/make_fit.py b/scripts/make_fit.py index 3db129f40b20..10a040f4eb83 100755 --- a/scripts/make_fit.py +++ b/scripts/make_fit.py @@ -57,10 +57,10 @@ import libfdt CompTool =3D collections.namedtuple('CompTool', 'ext,tools') =20 COMP_TOOLS =3D { - 'bzip2': CompTool('.bz2', 'bzip2'), + 'bzip2': CompTool('.bz2', 'pbzip2,bzip2'), 'gzip': CompTool('.gz', 'pigz,gzip'), 'lz4': CompTool('.lz4', 'lz4'), - 'lzma': CompTool('.lzma', 'lzma'), + 'lzma': CompTool('.lzma', 'plzip,lzma'), 'lzo': CompTool('.lzo', 'lzop'), 'zstd': CompTool('.zstd', 'zstd'), } @@ -220,7 +220,12 @@ def compress_data(inf, compress): done =3D False for tool in comp.tools.split(','): try: - subprocess.call([tool, '-c'], stdin=3Dinf, stdout=3Dou= tf) + # Add parallel flags for tools that support them + cmd =3D [tool] + if tool in ('zstd', 'xz'): + cmd.extend(['-T0']) # Use all available cores + cmd.append('-c') + subprocess.call(cmd, stdin=3Dinf, stdout=3Doutf) done =3D True break except FileNotFoundError: --=20 2.43.0 From nobody Sun Feb 8 11:16:49 2026 Received: from mail-il1-f175.google.com (mail-il1-f175.google.com [209.85.166.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C79A832C93C for ; Fri, 14 Nov 2025 14:28:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763130501; cv=none; b=JNkqS64Lgqv+xpgKGixsRVREZnzTZINGbjGqD6ZXXw7SOb3eHUT0fjibj8mvhx3//IXpADfVBfJeEi4pfHz9rr/X9GUHkMxP9nB0i8BdPVjWHQfEAex4y6E7fKnAN4pU/aSK5pFqiRkvW+MBWFHfmHKmPe1LmjkWpAx4f5cvJTw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763130501; c=relaxed/simple; bh=iRwr4aWqYl4kIwimtlljRUSeu5ezG3oP5PG69wjW44g=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Lca6bQAaZfyz7n6KBKa9wljaSR6vBmpAhjpboJSrwZgKUz5hB3F19lVC7Gl5FYcFrXr/DSMXYpAA2C+mJWhdOdPfQcmY1H3zKXnP/KWKyJXsebZ07wDJb2vDWdWrlNkoIfzyRei/N7+pJUE3oEBuKEVnieDTnkrga+gAWSyQ0M4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=fiiNJ/nE; arc=none smtp.client-ip=209.85.166.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="fiiNJ/nE" Received: by mail-il1-f175.google.com with SMTP id e9e14a558f8ab-43470d98f77so9497955ab.2 for ; Fri, 14 Nov 2025 06:28:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1763130499; x=1763735299; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=b318BeGWA3r29XTQ0vNTU6rijD9/dIK2iMOcNHHPjBE=; b=fiiNJ/nER4yVJ6alDCyLzYJfdrlRo/XHmSxLEPiORSNQd+HGRK5hVj8chLMqHCikRZ tERfp2qfKuo3Z6wThF8Zggm8BZkuvwzVUI0UAttqDHYTlYW0fKlYuCR2W9GPnXhF05i6 YxUNHRcLrHLAa6sQL7pmSgJuNxGQCk7Kl0kD4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1763130499; x=1763735299; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=b318BeGWA3r29XTQ0vNTU6rijD9/dIK2iMOcNHHPjBE=; b=VL39vEj12GKS2GychqxGWw3HYDLgKn3MFll9ZdlLBgzuhuqQLAuAM8LP6jIwTaPPWH Y1qD48X5oR5iygQDLWk3Btkv+jiBNAAXH6geumZ9omkHFzXO4E/YM3EUJcB1Y31S7LYs ldnQUx2SVkLyMrxoF9fq+0qEWm2veFW1ytKZrXwKt0lOd1OEZyBu/4Ov3qEpzO9IXw7d YJ3TjWbDi4zgE7/NxGCScYFDmpVQWBQWJtSJj125gZ2ZiEFSzEV9/gyu/VuB2NCpcEF3 RlRo2V1pst3aWDVh5arEqGFJJrWUj6QKSD86XYoLGDeJjWqA8RzYS0q/84c6QrlTPZgo 9JjQ== X-Forwarded-Encrypted: i=1; AJvYcCU1xXk7ZytcPxOgtW9YxKQOTyQya0Rlfr7F3/OQbnpEo61rZ/tco5JqXlDmgiVw9Ey5JhwKiWXKfxMpaXY=@vger.kernel.org X-Gm-Message-State: AOJu0YyDLvTBH6AJ+cShsLVQfDL+t/3iV3i4aebH19GqySJpL7mtfa2f tsMimFCZqmwOkPCwagDDl015ecx4d1Vqg07b7mNXCF3tBECCGcjkXLCSz2Ro70J8xg== X-Gm-Gg: ASbGncs8jYrtUc8cA19vUm4UjJMPdb2Hror8/8zgSqSqSlw/g5Vd/mUDaC8kvm0u38N NRu01xm2iVnzo7BA049gNOI9dLfCMp1P9xIi5RSe9gtarAOacP6oBDJ32e2lPaG1r887RbKjkx5 gQYEEd9bNdlBjz9II29xgVD+6wg8F5S3prPhoblzWg7/9hkMil2A7lXcIkUHnruHk0o1PhL1+Lt Wd1ghdOZRQn0u4hRv+SD/+240laCuXN3Ch0xt8CESncIc2Ui045OAvP/Fmcia/lQvt4c+Fyumh5 KpzYTxRtPhMBXvF8jbwUTFtZUbbmylrcdaxKy4auREunnz8lgVyY+d4cjGHYbIbHTaO3lPRRh7d zoyAXS+Vh3Pa+fdzsDTQYpLFgc7HysB+NSot6PR8ncYwTH2cLiOGqCPfhHsUh+mYohKoGEe6SQD 96io93 X-Google-Smtp-Source: AGHT+IEsHcIFm8dXTBDkdjl5jk0rVOZMeytOzdSeMB9ux68T/mGruUqL+yzI070U3lDQqZZhTkEkSg== X-Received: by 2002:a05:6e02:5e8a:b0:434:96ea:ff56 with SMTP id e9e14a558f8ab-43496eb00b0mr156465ab.39.1763130498831; Fri, 14 Nov 2025 06:28:18 -0800 (PST) Received: from chromium.org ([73.34.74.121]) by smtp.gmail.com with ESMTPSA id e9e14a558f8ab-434839cdb10sm27500205ab.32.2025.11.14.06.28.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Nov 2025 06:28:17 -0800 (PST) From: Simon Glass To: linux-arm-kernel@lists.infradead.org Cc: Nicolas Schier , Masahiro Yamada , Chen-Yu Tsai , Tom Rini , Ahmad Fatoum , =?UTF-8?q?J=20=2E=20Neusch=C3=A4fer?= , Simon Glass , linux-kernel@vger.kernel.org Subject: [PATCH v5 8/8] scripts/make_fit: Compress dtbs in parallel Date: Fri, 14 Nov 2025 07:27:34 -0700 Message-ID: <20251114142741.1919072-9-sjg@chromium.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251114142741.1919072-1-sjg@chromium.org> References: <20251114142741.1919072-1-sjg@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When there are 1500 device tree files it takes quite a while to compress them. Do it in parallel. Signed-off-by: Simon Glass --- (no changes since v1) scripts/make_fit.py | 56 ++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 50 insertions(+), 6 deletions(-) diff --git a/scripts/make_fit.py b/scripts/make_fit.py index 10a040f4eb83..d1af496f3a57 100755 --- a/scripts/make_fit.py +++ b/scripts/make_fit.py @@ -43,6 +43,7 @@ as U-Boot, Linuxboot, Tianocore, etc. =20 import argparse import collections +import multiprocessing import os import shutil import subprocess @@ -237,15 +238,31 @@ def compress_data(inf, compress): return comp_data =20 =20 -def output_dtb(fsw, seq, fname, arch, compress): +def compress_dtb(fname, compress): + """Compress a single DTB file + + Args: + fname (str): Filename containing the DTB + compress (str): Compression algorithm, e.g. 'gzip' + + Returns: + tuple: (str: fname, bytes: compressed_data) + """ + with open(fname, 'rb') as inf: + compressed =3D compress_data(inf, compress) + return fname, compressed + + +def output_dtb(fsw, seq, fname, arch, compress, data=3DNone): """Write out a single devicetree to the FIT =20 Args: fsw (libfdt.FdtSw): Object to use for writing seq (int): Sequence number (1 for first) fname (str): Filename containing the DTB - arch: FIT architecture, e.g. 'arm64' + arch (str): FIT architecture, e.g. 'arm64' compress (str): Compressed algorithm, e.g. 'gzip' + data (bytes): Pre-compressed data (optional) """ with fsw.add_node(f'fdt-{seq}'): fsw.property_string('description', os.path.basename(fname)) @@ -253,9 +270,10 @@ def output_dtb(fsw, seq, fname, arch, compress): fsw.property_string('arch', arch) fsw.property_string('compression', compress) =20 - with open(fname, 'rb') as inf: - compressed =3D compress_data(inf, compress) - fsw.property('data', compressed) + if data is None: + with open(fname, 'rb') as inf: + data =3D compress_data(inf, compress) + fsw.property('data', data) =20 =20 def build_ramdisk(args, tmpdir): @@ -366,6 +384,11 @@ def _process_dtbs(args, fsw, entries, fdts): """ seq =3D 0 size =3D 0 + + # First figure out the unique DTB files that need compression + todo =3D [] + file_info =3D [] # List of (fname, model, compat, files) tuples + for fname in args.dtbs: # Ignore non-DTB (*.dtb) files if os.path.splitext(fname)[1] !=3D '.dtb': @@ -377,11 +400,32 @@ def _process_dtbs(args, fsw, entries, fdts): sys.stderr.write(f'Error processing {fname}:\n') raise e =20 + file_info.append((fname, model, compat, files)) + for fn in files: + if fn not in fdts and fn not in todo: + todo.append(fn) + + # Compress all DTBs in parallel + cache =3D {} + if todo and args.compress !=3D 'none': + if args.verbose: + print(f'Compressing {len(todo)} DTBs...') + + with multiprocessing.Pool() as pool: + compress_args =3D [(fn, args.compress) for fn in todo] + # unpacks each tuple, calls compress_dtb(fn, compress) in para= llel + results =3D pool.starmap(compress_dtb, compress_args) + + cache =3D dict(results) + + # Now write all DTBs to the FIT using pre-compressed data + for fname, model, compat, files in file_info: for fn in files: if fn not in fdts: seq +=3D 1 size +=3D os.path.getsize(fn) - output_dtb(fsw, seq, fn, args.arch, args.compress) + output_dtb(fsw, seq, fn, args.arch, args.compress, + cache.get(fn)) fdts[fn] =3D seq =20 files_seq =3D [fdts[fn] for fn in files] --=20 2.43.0