From nobody Tue Dec 2 02:30:06 2025 Received: from mail-io1-f50.google.com (mail-io1-f50.google.com [209.85.166.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E1569307AFB for ; Wed, 19 Nov 2025 18:14:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763576066; cv=none; b=OrZc7PhgauyzOeEN0bqS/6Q87uyoYkJ7fFnLwsrSKBo+th4HfqhDUTVJXBamKMJ+CZMbQQZyiUQKWLZVjyO0Yb3/NzauolNR+YhQCvecncTZsRXjooivxcFew8+TkhiMnMVy45E+Cji4LAn7R1h8lKLRkaS3xmVEW05p6ucb8+A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763576066; c=relaxed/simple; bh=EeemFWJ3VEPTTKqxw+38LYskKYP/EN/4v6MeWplNysA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dDBPei5Fazu+zeQaEubxyqGXQg9pw2aeeHqY8nXxQxxHLiNF4F4VPACBBNEy6BnaKeZcv9UqGh0H+QO5ByEv0HHUrh+hvjK1Ss0HEnqv7hxklxq+qpM6pOidR4GhwdxH5Ltn8m4ecPcUvu5nGdFHfmxbVruLxZDNgnQcucxs7cM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=NdmVk7k+; arc=none smtp.client-ip=209.85.166.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="NdmVk7k+" Received: by mail-io1-f50.google.com with SMTP id ca18e2360f4ac-9486adc1aa9so43139f.1 for ; Wed, 19 Nov 2025 10:14:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1763576051; x=1764180851; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zF1Pzd7Y25nbnANuZLJhVbfb7fwymTpO79JMBdCEa0k=; b=NdmVk7k+HYBFwU7tpUjd6CFFJGvSWBItbc6mGXoROAtkGwIsAwP5BRMyoOA5tXPIgl aLv4yHZukW7zYEboJyhtOHfp6ddDhNou6x2nqRsv/QH23vbQhJwR+cuKkjKzBPF6P/fZ eXMpsvLp1kFlEuQy/+nlyZeZfhCCWDZe956OE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1763576051; x=1764180851; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=zF1Pzd7Y25nbnANuZLJhVbfb7fwymTpO79JMBdCEa0k=; b=ggsH7VwdSE82e5rYFaZDDy3fiZudQbl4Dayya/PSDlM4SB4HYDV9fZxerTjUDF1kzl RYQhh98KmZsqeq1SBC/Yyi20Jp6x9tpVi+K60VNUu5ZN4RLDTGviUQFyzw2oGPgZUO6O sGtyVzmng1XpSZNJw4Kjz0VKeCB5hu+UffH0pttJqQ+EtuRqGPvbc/mAT4M3BO5GX/5r tUC6qV45e9G7jNdLl8uUYANvqyGTj4h1X0q/Ea0EBnXq63fj0JBgzYgsSYWrOH5XwntR +I1Ief3FpKBMGjs5eCnxuYTKSqujoDYAUUaY+DsjaNVjHuZ/o1Jy+1NChBLh2FItZj1H lbUQ== X-Forwarded-Encrypted: i=1; AJvYcCUfU49bx8VrzFnYK/D1RxflgN5h1kl30w9D7dq7BeVVQFx3A63MQQf9rNJNG+TgeyWieFXylokh5cgS9x0=@vger.kernel.org X-Gm-Message-State: AOJu0YxO5gk+UElPQKWZ7ma9LnIWX7/6j82/VnA1f1nq4MCGgxjQcLTm e0J55rf9QcHDprYbEQMI+HDNVokiuZYCgdIobJ7XT4VTyZidxkxEyeBSTgVRJMANKw== X-Gm-Gg: ASbGncu2IKhXG2mAtaodkRDtlYNgzOSiUr9BpBAOA4+hMXX1KBIh+lqhQpOP9nfeDSj 9LAlaoexOReilFG1tmprqNaywtsYHebOfgcKn1QJd7A2XRNOi4JHaLVPYE7ibkGAaWOMoc1J3qR JzO8ANo4Z6LRGwfrz1PllINAl/Sw0QYhOL8TPG+Eks88nf+BJv2RyvfWZTsGGk+8WKI08y9c28e jZMrYZsKjBefx3QLINMpjcHjWVfpFnjMvSibiq6750Mo7ci40gvf5DFSJhvF6Up4q5sjTUkd56b 6fW4kUcaoqhFkvTLb9/+po0VaWUb/M6W/NM2hdvtk7CJC4/uckQwbQ8buOZmM3LVHtNU8Q4xNPI gaHb9ww8/XhqU+hY+hZHhYp1MjjxIV7YQzYbqYIRIE4nznZ8b0HUf6XcnnO/iaTgIY5+YlqedTP NxkPr0 X-Google-Smtp-Source: AGHT+IGXok4pX6SwDb8ES6pk1m2C6nMJbpP1DT6LiLX3sEVL1CGlpcIazdW8D38rS+pPrrxt+7nSNQ== X-Received: by 2002:a05:6602:168a:b0:949:92c:8049 with SMTP id ca18e2360f4ac-949092c8360mr1301383839f.12.1763576050905; Wed, 19 Nov 2025 10:14:10 -0800 (PST) Received: from chromium.org ([73.34.74.121]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-949385ae254sm4838639f.1.2025.11.19.10.14.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Nov 2025 10:14:09 -0800 (PST) From: Simon Glass To: linux-arm-kernel@lists.infradead.org Cc: =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , Masahiro Yamada , Tom Rini , Ahmad Fatoum , =?UTF-8?q?J=20=2E=20Neusch=C3=A4fer?= , Nicolas Schier , Chen-Yu Tsai , Simon Glass , linux-kernel@vger.kernel.org Subject: [PATCH v6 4/8] scripts/make_fit: Provide a way to add built modules Date: Wed, 19 Nov 2025 11:13:25 -0700 Message-ID: <20251119181333.991099-5-sjg@chromium.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251119181333.991099-1-sjg@chromium.org> References: <20251119181333.991099-1-sjg@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Provide arguments to support building a ramdisk from a directory tree of modules. This is a convenient way to try out a kernel with its modules. This makes use usr/gen_initramfs.sh where it can be found. Failing that it uses the cpio tool rather than attempting to use a python module or our own code. The list of modules is provided in a file. Signed-off-by: Simon Glass Suggested-by: Ahmad Fatoum --- Note: The approach of depending on modules_install caused build problems as mentioned in the cover letter. I dropped the review tag from Nicolas Schier since there are quite a few changes in this version. Changes in v6: - Using the modules.order file instead of 'find' - Use gen_initramfs.sh where available Changes in v4: - Provide the list of modules from the Makefile - Reduce verbosity (don't print every module filename) Changes in v3: - Add a way to add built modules into the FIT scripts/make_fit.py | 127 +++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 119 insertions(+), 8 deletions(-) diff --git a/scripts/make_fit.py b/scripts/make_fit.py index 1a74a9dcd85e..9dfef11fc4b3 100755 --- a/scripts/make_fit.py +++ b/scripts/make_fit.py @@ -13,11 +13,17 @@ Usage: -r /boot/initrd.img-6.14.0-27-generic @arch/arm64/boot/dts/dtbs-li= st -E -c gzip =20 + # Build with modules ramdisk instead of external ramdisk: + make_fit.py -A arm64 -n 'Linux-6.17' -O linux + -o arch/arm64/boot/image.fit -k /tmp/kern/arch/arm64/boot/image.itk + -m module1.ko module2.ko module3.ko @arch/arm64/boot/dts/dtbs-list + Creates a FIT containing the supplied kernel, an optional ramdisk, and a s= et of devicetree files, either specified individually or listed in a file (with = an '@' prefix). =20 Use -r to specify an existing ramdisk/initrd file. +Use -m to build a ramdisk from specified kernel module files. =20 Use -E to generate an external FIT (where the data is placed after the FIT data structure). This allows parsing of the data without loading @@ -38,6 +44,7 @@ as U-Boot, Linuxboot, Tianocore, etc. import argparse import collections import os +import shutil import subprocess import sys import tempfile @@ -83,8 +90,14 @@ def parse_args(): help=3D'Specifies the operating system') parser.add_argument('-k', '--kernel', type=3Dstr, required=3DTrue, help=3D'Specifies the (uncompressed) kernel input file (.itk)') - parser.add_argument('-r', '--ramdisk', type=3Dstr, + + # Create mutually exclusive group for ramdisk options + rd_group =3D parser.add_mutually_exclusive_group() + rd_group.add_argument('-r', '--ramdisk', type=3Dstr, help=3D'Specifies the ramdisk/initrd input file') + rd_group.add_argument('-m', '--modules', type=3Dstr, nargs=3D'+', + help=3D'List of module filenames to include in ramdisk') + parser.add_argument('-v', '--verbose', action=3D'store_true', help=3D'Enable verbose output') parser.add_argument('dtbs', type=3Dstr, nargs=3D'*', @@ -240,6 +253,89 @@ def output_dtb(fsw, seq, fname, arch, compress): fsw.property('data', compressed) =20 =20 +def find_gen_initramfs(): + """Find the gen_initramfs.sh script + + Looks for usr/gen_initramfs.sh relative to the current directory, + or in the source directory if running from a build directory. + + Returns: + str: Path to gen_initramfs.sh if found, None otherwise + """ + path =3D 'usr/gen_initramfs.sh' + if not os.path.exists(path): + srctree =3D os.environ.get('srctree') + if not srctree: + return None + path =3D os.path.join(srctree, path) + if not os.path.exists(path): + return None + + return path + + +def build_ramdisk(args, tmpdir): + """Build a cpio ramdisk containing kernel modules + + Similar to mkinitramfs, this creates a compressed cpio-archive contain= ing + the kernel modules for the current kernel version. + + Args: + args (Namespace): Program arguments + tmpdir (str): Temporary directory to use for modules installation + + Returns: + tuple: + bytes: Compressed cpio data containing modules + int: total uncompressed size + """ + suppress =3D None if args.verbose else subprocess.DEVNULL + + if args.verbose: + print(f'Copying {len(args.modules)} modules to ramdisk') + + # Create output-directory structure + outdir =3D os.path.join(tmpdir, 'initramfs') + modules_dir =3D os.path.join(outdir, 'usr', 'lib', 'modules') + os.makedirs(modules_dir, exist_ok=3DTrue) + + # Copy in the specified modules, renaming .o to .ko + for module in args.modules: + name, ext =3D os.path.splitext(module) + dest =3D name + '.ko' if ext =3D=3D '.o' else module + + # Preserve directory structure under modules_dir + dest_path =3D os.path.join(modules_dir, dest) + os.makedirs(os.path.dirname(dest_path), exist_ok=3DTrue) + shutil.copy2(module, dest_path) + + if args.verbose: + print(f'Creating cpio archive from {outdir}') + + with tempfile.NamedTemporaryFile() as cpio_file: + # Use gen_initramfs.sh to create the cpio archive if possible + gen_initramfs =3D find_gen_initramfs() + if gen_initramfs: + subprocess.run([gen_initramfs, '-o', cpio_file.name, outdir], + check=3DTrue, stdout=3Dsuppress, stderr=3Dsuppr= ess) + else: + # Change to initramfs directory and create cpio archive + with subprocess.Popen(['find', '.', '-print0'], cwd=3Doutdir, + stdout=3Dsubprocess.PIPE) as find: + with subprocess.Popen(['cpio', '-o', '-0', '-H', 'newc'], + stdin=3Dfind.stdout, stdout=3Dcpio_f= ile, + stderr=3Dsuppress, cwd=3Doutdir) as = cpio: + find.stdout.close() + cpio.wait() + find.wait() + + cpio_file.seek(0) # Reset to beginning for reading + if args.verbose: + print('Reading ramdisk...' if args.compress =3D=3D 'none' else + f'Compressing ramdisk with {args.compress}...') + return compress_data(cpio_file, args.compress), cpio_file.tell() + + def process_dtb(fname, args): """Process an input DTB, decomposing it if requested and is possible =20 @@ -318,11 +414,12 @@ def _process_dtbs(args, fsw, entries, fdts): return seq, size =20 =20 -def build_fit(args): +def build_fit(args, tmpdir): """Build the FIT from the provided files and arguments =20 Args: args (Namespace): Program arguments + tmpdir (str): Temporary directory for any temporary files =20 Returns: tuple: @@ -344,20 +441,29 @@ def build_fit(args): =20 # Handle the ramdisk if provided. Compression is not supported as it is # already compressed. + ramdisk_data =3D None if args.ramdisk: with open(args.ramdisk, 'rb') as inf: - data =3D inf.read() - size +=3D len(data) - write_ramdisk(fsw, data, args) + ramdisk_data =3D inf.read() + size +=3D len(ramdisk_data) + elif args.modules: + if args.verbose: + print('Building modules ramdisk...') + ramdisk_data, uncomp_size =3D build_ramdisk(args, tmpdir) + size +=3D uncomp_size + + if ramdisk_data: + write_ramdisk(fsw, ramdisk_data, args) =20 count, fdt_size =3D _process_dtbs(args, fsw, entries, fdts) size +=3D fdt_size =20 - finish_fit(fsw, entries, bool(args.ramdisk)) + finish_fit(fsw, entries, has_ramdisk=3Dbool(ramdisk_data)) =20 - # Include the kernel itself in the returned file count fdt =3D fsw.as_fdt() fdt.pack() + + # Count FDT files, kernel, plus ramdisk if present return fdt.as_bytearray(), count + 1 + bool(args.ramdisk), size =20 =20 @@ -365,7 +471,12 @@ def run_make_fit(): """Run the tool's main logic""" args =3D parse_args() =20 - out_data, count, size =3D build_fit(args) + tmpdir =3D tempfile.mkdtemp(prefix=3D'make_fit_') + try: + out_data, count, size =3D build_fit(args, tmpdir) + finally: + shutil.rmtree(tmpdir) + with open(args.output, 'wb') as outf: outf.write(out_data) =20 --=20 2.43.0