From nobody Wed Dec 17 21:25:30 2025 Received: from mail-io1-f68.google.com (mail-io1-f68.google.com [209.85.166.68]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 84C5613AC1 for ; Mon, 6 Oct 2025 23:02:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.68 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759791766; cv=none; b=ik+0fHomtOOAY+/ggbNE3OxNERU3BzxYHz9DB858W1FCn8bVJI6W3fvWR1xa37ZVCXEjI6lbwn6pGD2Tl2dxOXCCDevK6n85a/xQq5/T+WMhqYHRE75PH2a83lDZJascW5qYck2KLn7w/XA6N2gy4eaELoR6Kj+m9T4+xnmVSrs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759791766; c=relaxed/simple; bh=STKB9sgJnmofsdL/CTVfyDM6IGK8+0SCv9N/ZkvCAiw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=chGiNZ5p4gtL+yI+jOugBvhvzgChWTmrminzVd5TF4GsonIpT9MjdBby71qM6zOHYAxpxM1AjTLDM+3P56nbVgPhAFblUBJXx3GkG/cGczYoTxf2nt+1kw1DT2s8QjPU6fMhM/DiEmC9Wvh9Bq31Uxk3O/n8gffUGEGfnf9KxYM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=obL33BdZ; arc=none smtp.client-ip=209.85.166.68 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="obL33BdZ" Received: by mail-io1-f68.google.com with SMTP id ca18e2360f4ac-911afafcc20so219673839f.3 for ; Mon, 06 Oct 2025 16:02:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1759791763; x=1760396563; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/04FcGUHZ4AKn3TuM7bYMUdqJTWV25qYG6FlFl5oan4=; b=obL33BdZKpbNvdZvJO51sB0ddOlQv04s/ntvMcnaq7IILPWkLPofGLZmQGbnZ1CXcc FvDxg4mena/seQ+hKwyRNaZNp8LTGcGmBnDkMak9VJsNoV4m/j5RF8pilLcPoSlXc0M6 03U7BPhCfAIf6+r0b1R8s/Pm3Hp25OR+Iw9o8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759791763; x=1760396563; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/04FcGUHZ4AKn3TuM7bYMUdqJTWV25qYG6FlFl5oan4=; b=Ic1xlS+x79IWbeRsvJehpPLoE8+DXqwMgaMww0ilembzIvTRuMUXfWmLXPtp0Rf8YP +0pYPGC/wVVH1iUiwjtlScTyl0/gD05ALEDhxbSCsVRlf+JYR2g0X46QJ097dml9o6+Z VaEJkchDxpSAsmgCxBx3gPObznweFytn03ROVYR0+jgCfn15H2NazLVP3Hii+FRCNzpP v+qq7jiyYTWDoQW77ZmlzA87PFlAxsXZsqT2Umx4oa92NwDI1FvhrrmS9wuZNMDrKyDb 39H+AgEC4OSBcSwht160jERgCvcRpNOh75WrQjvHkhaY90awRlfSPBGgfuFt6LmPJ4vW bNIw== X-Forwarded-Encrypted: i=1; AJvYcCVvoFEtGoTE2UAmIP5ZwvrwH3wh6OMAbHYwDDzjDlIImykq8C8vyx4DT4Ot6ogdjfdZJ7ml/rup4f3Z25k=@vger.kernel.org X-Gm-Message-State: AOJu0YyinPvyndMWPQ5A4LaMh8HTOpeQG9hCptl/ZxspVxhp0zRlaMnT weIQYPmn2eh7y4T7CmvvQoLAIat0tVb6rvmUGAj2DrmbbGpaodHnFsMr5xRAQit3mw== X-Gm-Gg: ASbGncvT2OKkIlpjYmv14VWyHM9fx/iaUOpCI6qGFereOOe/UdHIfgZQD4KoX9SuXeY dyM16ra5uzgeaZaiF5Umct6KHClLqsdulleFvnwDnQ8mwq4saAibXoR5vLdYeGzGnNA5e90VeDI GUdps9DfmUoUV962MHmzNBKFp4gvnAAk2LN2R1NkxYJO8Zs1kdDzZtmKP/lrb1aIgZyTrlT3nYC hAu+5QaYEBI31Jsz23tx3Q96hwqnzFsRP5Zv9aV5L1oT3FWJ+FGgctgPUbH0Xwrg8DkAj7R9CqH s5HNBrguOo+I7AQlI5nj6nZtMZHK5pIx5cWyeNOoCpoGnTyZyhijvLsxpF4oq9uSVpOKhBLvo9v mFf0GKZJ+82GQ5SUxDY3G/QOh7rMnLyxf1yKk2DX6Q4s= X-Google-Smtp-Source: AGHT+IHT2eZsuDBZCCUdTjmcq+oBn1mLiCVw4dyHErxCXf+keS4Z7nRDbUIXhcGV5W24pJMpa8mJ3g== X-Received: by 2002:a05:6602:484:b0:887:56f5:26a2 with SMTP id ca18e2360f4ac-93b96a9952cmr1753381039f.14.1759791763505; Mon, 06 Oct 2025 16:02:43 -0700 (PDT) Received: from chromium.org ([50.235.115.130]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-57b5ec55bc7sm5422226173.69.2025.10.06.16.02.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Oct 2025 16:02:42 -0700 (PDT) From: Simon Glass To: linux-arm-kernel@lists.infradead.org Cc: Chen-Yu Tsai , Ahmad Fatoum , Masahiro Yamada , =?UTF-8?q?J=20=2E=20Neusch=C3=A4fer?= , Nicolas Schier , Tom Rini , Simon Glass , linux-kernel@vger.kernel.org Subject: [PATCH v4 1/7] scripts/make_fit: Speed up operation Date: Mon, 6 Oct 2025 17:01:52 -0600 Message-ID: <20251006230205.521341-2-sjg@chromium.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251006230205.521341-1-sjg@chromium.org> References: <20251006230205.521341-1-sjg@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The kernel is likely at least 16MB so we may as well use that as a step size when reallocating space for the FIT in memory. Pack the FIT at the end, so there is no wasted space. This reduces the time to pack by an order of magnitude, or so. Signed-off-by: Simon Glass Reviewed-by: Nicolas Schier --- (no changes since v3) Changes in v3: - Move the ramdisk chunk into the correct patch scripts/make_fit.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/scripts/make_fit.py b/scripts/make_fit.py index 1683e5ec6e67..0f5e7c4b8aed 100755 --- a/scripts/make_fit.py +++ b/scripts/make_fit.py @@ -98,7 +98,7 @@ def setup_fit(fsw, name): fsw (libfdt.FdtSw): Object to use for writing name (str): Name of kernel image """ - fsw.INC_SIZE =3D 65536 + fsw.INC_SIZE =3D 16 << 20 fsw.finish_reservemap() fsw.begin_node('') fsw.property_string('description', f'{name} with devicetree set') @@ -299,7 +299,9 @@ def build_fit(args): finish_fit(fsw, entries) =20 # Include the kernel itself in the returned file count - return fsw.as_fdt().as_bytearray(), seq + 1, size + fdt =3D fsw.as_fdt() + fdt.pack() + return fdt.as_bytearray(), seq + 1, size =20 =20 def run_make_fit(): --=20 2.43.0 From nobody Wed Dec 17 21:25:30 2025 Received: from mail-io1-f66.google.com (mail-io1-f66.google.com [209.85.166.66]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 65A2A1DBB13 for ; Mon, 6 Oct 2025 23:02:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.66 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759791768; cv=none; b=O5mf9dgJP2UmDodsBFPe4xLD7M37b3PQL+OC4L1CQlc0qXsC5SlS6QlO4+CkCGnaebN7iVY/sCOuGC7k+ASWIMbVUyq4HGiq4Y1YZP8SVcey7RGAgF6rqF/8QAe9BBxOGUKHuQfrM3kEjdcRYaX7lyhRF+8EKbiErqU9eV1pvQ8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759791768; c=relaxed/simple; bh=v1IOMxuClhNtVj5VjeVsxIX0foETUgX8qoVEvIYumzA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=YpHm6kOPS5gM7kX1leJhGkpVoSeHYCtLHAXnRrIfCEDO9RZWNbCgAlKaaKk+zMCoXz046nb8YH/Iz1/60ibDtbfOlfIuwJenm5rgdWJO6Hfq+cRImXYAdA9vqukA1yRfB6BhBxS7menhAom/USLoR/HnA+j2MhHvnhpic9KYmc8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=bosDa8xB; arc=none smtp.client-ip=209.85.166.66 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="bosDa8xB" Received: by mail-io1-f66.google.com with SMTP id ca18e2360f4ac-91122373653so233701839f.1 for ; Mon, 06 Oct 2025 16:02:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1759791765; x=1760396565; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=D/GhgyUTJVSqf6jn50N7qbJFCmaZJonK+/mIvqtdKlw=; b=bosDa8xBRm5I87c7h+Jc+AuJDXOMR1yfg2vIGZp+axmNLBz4XMiMzeqjnIjQ9HWAPs ms2l5K1W1PVXoebKxUPo1SrdRt5f6tBcbV6vZgj26qi1wYjmO1AZgPOkCrXA9eYGEdkt YXLhrmhKJ2y7eATtfiep/OV4rwmzIHwqobGgM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759791765; x=1760396565; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=D/GhgyUTJVSqf6jn50N7qbJFCmaZJonK+/mIvqtdKlw=; b=i4hXHZyyEEBjoFBJYBaaHCGCfLvkI04dNmQ5AYTnDYAfdW2qNSib82nTA1xZ7wsnyr 2lZuQd2fxdCbVAv7y2+BjTyQbAEsm9xJEX1r7Dwn9R0BySoTJVxtnShhE47IWG5Ix6l9 tWuLOrb23vr141axdFtlDQMi3pPP8pqzHJeOs+TUpbyi+cCcuR3rqYsX7HaSfTfglyx8 RLRMOAbdEe3ULDAcCyy1H862VntE5dXXNAY25SBTb5+YL604whBNN5+99cQJggr9yLrd EBaq/9uuzPgjl+Lxv9dVasNNIt84Po17ef2gi0B0lz/Nh0fbceCUhx+EkeEbZFCx9nkd UuIQ== X-Forwarded-Encrypted: i=1; AJvYcCUQRHNY+yvx5mmZeq5z/Rk+NmaGYVIp0kW7n2BDI1cdMMdW8HhVfDSl6t+CpW7YUooZbWJy668+94MuK68=@vger.kernel.org X-Gm-Message-State: AOJu0YybtYSZSxruCR7d7S+RwdkP93/i7202ifkMQ5yF8F9KonOcRley hvk9Ee9umX03kldooTs1nTKqCca4wKkvv+EIx7poesHc9aLkDcCpnUOtA6tAfQcDLg== X-Gm-Gg: ASbGncu8xf0Pws8Qx9QaHPnNGb9PAkFHJat4AzGMePgYYFG01lvRoBzo543iUIRyvOo 3Gx7BgrVO+K2QHOFj5/s4nitjvK8DWi0pjv8i8SIRi14nY8+uThMe7/KLn8q/5WxnztjIw43yhK lh96je4SwDjqnx8ztYPR1Uu7w2IFUT2pRH2ULB+5MgYSCdbUUUUsWXLl4ggy8PjQPUZfTi94qTL 076GnoT4E25DhpXv7qHp7JKScTKAS4f73V6prPsxt4eReIFBWcXRwPs9h8HjukQIZYhRdC1AFh1 3Bc7jJBQAH2WkZT2a/y7jakGGZS2LVjsXUHdUaog1a4VURSoiDT84LuHLLnja6DGef4UGrvXheh LXE5BE5jIAx3F6iY7YYy3Ob8UwQmWKMmEHZfY0MtdZIyxjZi360MlGg== X-Google-Smtp-Source: AGHT+IEbnOK5MEZLn3qI/u/o7BEClgUIMh+Q50zXFLmPT5dUkz6tZ9h7Rqa5VCJPtPqD4jFv0BZZjg== X-Received: by 2002:a05:6e02:2581:b0:425:6f0b:a1db with SMTP id e9e14a558f8ab-42e7ad16a3emr170806305ab.9.1759791765261; Mon, 06 Oct 2025 16:02:45 -0700 (PDT) Received: from chromium.org ([50.235.115.130]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-57b5ec55bc7sm5422226173.69.2025.10.06.16.02.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Oct 2025 16:02:44 -0700 (PDT) From: Simon Glass To: linux-arm-kernel@lists.infradead.org Cc: Chen-Yu Tsai , Ahmad Fatoum , Masahiro Yamada , =?UTF-8?q?J=20=2E=20Neusch=C3=A4fer?= , Nicolas Schier , Tom Rini , Simon Glass , linux-kernel@vger.kernel.org Subject: [PATCH v4 2/7] scripts/make_fit: Support an initial ramdisk Date: Mon, 6 Oct 2025 17:01:53 -0600 Message-ID: <20251006230205.521341-3-sjg@chromium.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251006230205.521341-1-sjg@chromium.org> References: <20251006230205.521341-1-sjg@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" FIT (Flat Image Tree) allows a ramdisk to be included in each configuration. Add support for this to the script. This feature is not available via 'make image.fit' since the ramdisk likely needs to be built separately anyway, e.g. using modules from the kernel build. A later patch in this series provides support for doing that. Note that the uncompressed size is not correct when a ramdisk is use, since it is too expensive to decompress the ramdisk. Signed-off-by: Simon Glass Reviewed-by: Nicolas Schier --- (no changes since v3) Changes in v3: - Add a comment at the top of the file about the -r option - Count the ramdisk in the total files - Update the commit message - Update the commit message Changes in v2: - Don't compress the ramdisk as it is already compressed scripts/make_fit.py | 52 ++++++++++++++++++++++++++++++++++++++------- 1 file changed, 44 insertions(+), 8 deletions(-) diff --git a/scripts/make_fit.py b/scripts/make_fit.py index 0f5e7c4b8aed..984371f505bc 100755 --- a/scripts/make_fit.py +++ b/scripts/make_fit.py @@ -10,10 +10,14 @@ Usage: make_fit.py -A arm64 -n 'Linux-6.6' -O linux -o arch/arm64/boot/image.fit -k /tmp/kern/arch/arm64/boot/image.itk - @arch/arm64/boot/dts/dtbs-list -E -c gzip + -r /boot/initrd.img-6.14.0-27-generic @arch/arm64/boot/dts/dtbs-li= st + -E -c gzip =20 -Creates a FIT containing the supplied kernel and a set of devicetree files, -either specified individually or listed in a file (with an '@' prefix). +Creates a FIT containing the supplied kernel, an optional ramdisk, and a s= et of +devicetree files, either specified individually or listed in a file (with = an +'@' prefix). + +Use -r to specify an existing ramdisk/initrd file. =20 Use -E to generate an external FIT (where the data is placed after the FIT data structure). This allows parsing of the data without loading @@ -29,8 +33,6 @@ looks at the .cmd files produced by the kernel build. =20 The resulting FIT can be booted by bootloaders which support FIT, such as U-Boot, Linuxboot, Tianocore, etc. - -Note that this tool does not yet support adding a ramdisk / initrd. """ =20 import argparse @@ -81,6 +83,8 @@ def parse_args(): help=3D'Specifies the operating system') parser.add_argument('-k', '--kernel', type=3Dstr, required=3DTrue, help=3D'Specifies the (uncompressed) kernel input file (.itk)') + parser.add_argument('-r', '--ramdisk', type=3Dstr, + help=3D'Specifies the ramdisk/initrd input file') parser.add_argument('-v', '--verbose', action=3D'store_true', help=3D'Enable verbose output') parser.add_argument('dtbs', type=3Dstr, nargs=3D'*', @@ -133,7 +137,28 @@ def write_kernel(fsw, data, args): fsw.property_u32('entry', 0) =20 =20 -def finish_fit(fsw, entries): +def write_ramdisk(fsw, data, args): + """Write out the ramdisk image + + Writes a ramdisk node along with the required properties + + Args: + fsw (libfdt.FdtSw): Object to use for writing + data (bytes): Data to write (possibly compressed) + args (Namespace): Contains necessary strings: + arch: FIT architecture, e.g. 'arm64' + fit_os: Operating Systems, e.g. 'linux' + """ + with fsw.add_node('ramdisk'): + fsw.property_string('description', 'Ramdisk') + fsw.property_string('type', 'ramdisk') + fsw.property_string('arch', args.arch) + fsw.property_string('os', args.os) + fsw.property('data', data) + fsw.property_u32('load', 0) + + +def finish_fit(fsw, entries, has_ramdisk=3DFalse): """Finish the FIT ready for use =20 Writes the /configurations node and subnodes @@ -143,6 +168,7 @@ def finish_fit(fsw, entries): entries (list of tuple): List of configurations: str: Description of model str: Compatible stringlist + has_ramdisk (bool): True if a ramdisk is included in the FIT """ fsw.end_node() seq =3D 0 @@ -154,6 +180,8 @@ def finish_fit(fsw, entries): fsw.property_string('description', model) fsw.property('fdt', bytes(''.join(f'fdt-{x}\x00' for x in = files), "ascii")) fsw.property_string('kernel', 'kernel') + if has_ramdisk: + fsw.property_string('ramdisk', 'ramdisk') fsw.end_node() =20 =20 @@ -274,6 +302,14 @@ def build_fit(args): size +=3D os.path.getsize(args.kernel) write_kernel(fsw, comp_data, args) =20 + # Handle the ramdisk if provided. Compression is not supported as it is + # already compressed. + if args.ramdisk: + with open(args.ramdisk, 'rb') as inf: + data =3D inf.read() + size +=3D len(data) + write_ramdisk(fsw, data, args) + for fname in args.dtbs: # Ignore non-DTB (*.dtb) files if os.path.splitext(fname)[1] !=3D '.dtb': @@ -296,12 +332,12 @@ def build_fit(args): =20 entries.append([model, compat, files_seq]) =20 - finish_fit(fsw, entries) + finish_fit(fsw, entries, bool(args.ramdisk)) =20 # Include the kernel itself in the returned file count fdt =3D fsw.as_fdt() fdt.pack() - return fdt.as_bytearray(), seq + 1, size + return fdt.as_bytearray(), seq + 1 + bool(args.ramdisk), size =20 =20 def run_make_fit(): --=20 2.43.0 From nobody Wed Dec 17 21:25:30 2025 Received: from mail-io1-f65.google.com (mail-io1-f65.google.com [209.85.166.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AAF0F1FFC48 for ; Mon, 6 Oct 2025 23:02:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.65 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759791771; cv=none; b=rHT+AxLNV3sonkJbCoiAGz57M2nB6sIEVhYCmBaoQqAfYwa0BWoq9mplkGW07xsNJXHkVqasepEX8l18Zn/8lzXbbSBpPbfDa4I2aRkAmFLvT8gchZpys3mvlxlj1ZWlwfgnjuwqNnWHTIkFVVzF3gcSE7oj7D87xWcaJ/o9SII= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759791771; c=relaxed/simple; bh=s4qO35QXAyXHS7iu4deoUTEjbP+L9WOhAVecOqeS5go=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Y64LKWnVN2gYowbpDUcDJ0UmTXgVzg6z3JKwZMrHZPYIw+tLc527zxKoYP9ak0GRRJNc+4NxHwpcwLEbGG+kPd8J55tfeZjCSpllw7eSwhSgt+WDD5ROSg8ba1LfnvYJfDhvYdMgk8enT+1nDHgDP3rKI19BWNDl894qSS1xi6M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=n+hJKmd/; arc=none smtp.client-ip=209.85.166.65 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="n+hJKmd/" Received: by mail-io1-f65.google.com with SMTP id ca18e2360f4ac-93607aec358so131916739f.0 for ; Mon, 06 Oct 2025 16:02:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1759791769; x=1760396569; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ebeC0cVyiB9zakr1pU6jmsWgTyhsnBRYHpTkJMDPkjw=; b=n+hJKmd/mQcXvX/TLw6KjnFj2VDlajHtsxvKIPDJEJzK7eat9qP3Tc3DqPxkKOd68I DEijL88H/cfv+9/lMah43o4DKJ0/6Thyv0E2KB2/nG19cEPJRM3Hm5ESJiGfvMwlATqr Lf9NSc9kmHFdERMEy5ApWmemHreYwoMSZgr1A= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759791769; x=1760396569; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ebeC0cVyiB9zakr1pU6jmsWgTyhsnBRYHpTkJMDPkjw=; b=o5KgsUHsKFb24tnGc6pIw/nvLi+lE/+bzpT1GTMCVmfWmRhJessPmPpBZ2J5t3Yi00 fCcrA0vxINYnzY/GBvM1yyhQsQMjLMmr/lG5Rqpai2RYrec8Cuwu7uoCYee8qCnNJ6PC 2oRB9NmrHCO+3A+uqAmtUQaZaWc488E1CVIErCjLfTFSajiDmo7Cf3WD9Ly0G5KwCCwn hmYOuWrHY/O7e8F7M/1VQrIpqeFsW+S5EnVYrFKBKB3HPlpxA2MS4T3ofrnQT1nRWV9E hdZ/S36mXsZkNizE9Itv/9J+TU9Rcfg5fgNYXUEP1lQ5IYBH0i6H13JbzWBDugmbdyRQ GGRQ== X-Forwarded-Encrypted: i=1; AJvYcCWNj8dZ8/6THSWkCPXDBnoj2gTzPes1eU9WgSMeO/TiX355ND9UnXqWkZKZTkPKwcpb6olxCBwTfUjmNE8=@vger.kernel.org X-Gm-Message-State: AOJu0YyRVdCHpagNMMAMIpO0Yroe1DNRfHY1DZYtWwTR4mZ1RLwcPfUy +nHtWNXNhRE3BK6UuBl6neggSNnUqpro/Zl0QKBeH9O+ODk2R8UzAdfsLOzAKetkNA== X-Gm-Gg: ASbGncvt1bwv0LjR10pYYIQl9h6jGM8YKDKR5ltiyVHkw28IXt2PGEByQEo195bV7Me CCSKj5roXJui8XQy4Iy+XHVJhmOL3ItNaWp7U5M53qYAe5pnzGDZuDoHyMX/wNzVn/ooB8qsmPj rGKtRTeflQkkJ65cAfEcRBVff5FXGoY64zzstXSclDlFLFsrGgsN82G57QpA89J/+Ver1d8Ds0M lQ8ppHNAiJ517r/fluIR0TvEjZ12J8YZN32ZDHJ6lnOh7F47+KU3CyujqbU/GR6V6BuIad/iiHw SGp7DaYpFXGgX6nIGVtjtNHaqY7aY1xzGZ1CrIgn9+QyMa/vnniN4ryJmTXTg1fO54IrmrFDBbq h95eF3RBekhOFRHPVKN/lPRRoNOL8dw6C8bfrABdmmPdWdlmVpcp2aA== X-Google-Smtp-Source: AGHT+IE9mdrAm5cg7BGAOe6wYDIg3l4Up+MccpLj79jTe3IHNt2jz5i4agbBHCXH73i/NIp8NWNZ1A== X-Received: by 2002:a05:6602:15c5:b0:93b:ba4a:3b64 with SMTP id ca18e2360f4ac-93bba4a441fmr740270939f.17.1759791768428; Mon, 06 Oct 2025 16:02:48 -0700 (PDT) Received: from chromium.org ([50.235.115.130]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-57b5ec55bc7sm5422226173.69.2025.10.06.16.02.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Oct 2025 16:02:46 -0700 (PDT) From: Simon Glass To: linux-arm-kernel@lists.infradead.org Cc: Chen-Yu Tsai , Ahmad Fatoum , Masahiro Yamada , =?UTF-8?q?J=20=2E=20Neusch=C3=A4fer?= , Nicolas Schier , Tom Rini , Simon Glass , linux-kernel@vger.kernel.org Subject: [PATCH v4 3/7] scripts/make_fit: Move dtb processing into a function Date: Mon, 6 Oct 2025 17:01:54 -0600 Message-ID: <20251006230205.521341-4-sjg@chromium.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251006230205.521341-1-sjg@chromium.org> References: <20251006230205.521341-1-sjg@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Since build_fit() is getting quite long, move the dtb processing into a separate function. Signed-off-by: Simon Glass Reviewed-by: Nicolas Schier --- (no changes since v1) scripts/make_fit.py | 67 +++++++++++++++++++++++++++++---------------- 1 file changed, 44 insertions(+), 23 deletions(-) diff --git a/scripts/make_fit.py b/scripts/make_fit.py index 984371f505bc..1a74a9dcd85e 100755 --- a/scripts/make_fit.py +++ b/scripts/make_fit.py @@ -277,6 +277,47 @@ def process_dtb(fname, args): =20 return (model, compat, files) =20 + +def _process_dtbs(args, fsw, entries, fdts): + """Process all DTB files and add them to the FIT + + Args: + args: Program arguments + fsw: FIT writer object + entries: List to append entries to + fdts: Dictionary of processed DTBs + + Returns: + tuple: + Number of files processed + Total size of files processed + """ + seq =3D 0 + size =3D 0 + for fname in args.dtbs: + # Ignore non-DTB (*.dtb) files + if os.path.splitext(fname)[1] !=3D '.dtb': + continue + + try: + (model, compat, files) =3D process_dtb(fname, args) + except Exception as e: + sys.stderr.write(f'Error processing {fname}:\n') + raise e + + for fn in files: + if fn not in fdts: + seq +=3D 1 + size +=3D os.path.getsize(fn) + output_dtb(fsw, seq, fn, args.arch, args.compress) + fdts[fn] =3D seq + + files_seq =3D [fdts[fn] for fn in files] + entries.append([model, compat, files_seq]) + + return seq, size + + def build_fit(args): """Build the FIT from the provided files and arguments =20 @@ -289,7 +330,6 @@ def build_fit(args): int: Number of configurations generated size: Total uncompressed size of data """ - seq =3D 0 size =3D 0 fsw =3D libfdt.FdtSw() setup_fit(fsw, args.name) @@ -310,34 +350,15 @@ def build_fit(args): size +=3D len(data) write_ramdisk(fsw, data, args) =20 - for fname in args.dtbs: - # Ignore non-DTB (*.dtb) files - if os.path.splitext(fname)[1] !=3D '.dtb': - continue - - try: - (model, compat, files) =3D process_dtb(fname, args) - except Exception as e: - sys.stderr.write(f"Error processing {fname}:\n") - raise e - - for fn in files: - if fn not in fdts: - seq +=3D 1 - size +=3D os.path.getsize(fn) - output_dtb(fsw, seq, fn, args.arch, args.compress) - fdts[fn] =3D seq - - files_seq =3D [fdts[fn] for fn in files] - - entries.append([model, compat, files_seq]) + count, fdt_size =3D _process_dtbs(args, fsw, entries, fdts) + size +=3D fdt_size =20 finish_fit(fsw, entries, bool(args.ramdisk)) =20 # Include the kernel itself in the returned file count fdt =3D fsw.as_fdt() fdt.pack() - return fdt.as_bytearray(), seq + 1 + bool(args.ramdisk), size + return fdt.as_bytearray(), count + 1 + bool(args.ramdisk), size =20 =20 def run_make_fit(): --=20 2.43.0 From nobody Wed Dec 17 21:25:30 2025 Received: from mail-io1-f68.google.com (mail-io1-f68.google.com [209.85.166.68]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A31572153EA for ; Mon, 6 Oct 2025 23:02:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.68 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759791773; cv=none; b=f7b+u2cjsrhRgy4hZ+R1hwYaYiK10sbPlD5iQ+G4qnWEKQI6YnE8DtGg7MlXGkAC5+jNRg2IO3o9eH7JN/wV8qUe8ppDdswq1f/n96Y0VbqdF72l66tx920QxsDcnd5WmpQr5XZMbTO4rPjCivb/yMKdznf+Zk5kUnBSaaKeeJs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759791773; c=relaxed/simple; bh=lHndcnCbZuEHl3jF9yXprme2yRHTjd0c8S9RLPZOZyU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fBh5WYalMJOxGeS56R+xlybaGaEj0pdpzO25kLTGKxVIBooXULeFcgaYAiJ314K9V/IdOqh7iWB8br8gPNedeSvhAZ6f2B4v7g029xnQkgsafDOmxZ6BlRGlEucn3LbBdy4/Fv/2tVMwqTmvV8AINECko64yfEN341qGSS+ZIKk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=Rm94yIup; arc=none smtp.client-ip=209.85.166.68 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="Rm94yIup" Received: by mail-io1-f68.google.com with SMTP id ca18e2360f4ac-92c4adc8bfeso430492839f.0 for ; Mon, 06 Oct 2025 16:02:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1759791771; x=1760396571; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ReTIzD0lyRGNQa5K8IAUq+EP/qION+p7p5sDaqfXvB4=; b=Rm94yIupD2tBPaD7ML8D5YQLBCBktgXLMw+dvUFdrPNU7kYo/5Z+JJ205Rbc8uZNkT lzihs/YkvPIJdC3PvaorkyXvyanO5bCDfBPUlEUSxzyAZxdPOuoc2QCqvgUSFvoAw9Se EAW7VlWFRTbWfNY86hMuOlaeXk7X3YIydVz+A= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759791771; x=1760396571; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ReTIzD0lyRGNQa5K8IAUq+EP/qION+p7p5sDaqfXvB4=; b=GtXOEPQPxkCEx7Ro9NlwrukftcGZC4mVN94i+c8Y/gXPGFrlI/yb8iC9qMrLWGx3La MR3RWlDqQCxZqELPQet+tKS153S2IWJfbfu19tKkBzEE7NJF3Qs1QtYVugd1HtLbixGM gOVHwr2UT7WQvbAYZxrQoYBgW6uXc5oP61lKlkL3eH9KoLpBz3MArFS/zEuPppqd9qFt dQP99A0WY1qA9cN/7hURbJ8rKqbsO3S/Zgdi2yvwkL1w2C4Mq9qyeM/FR6f6jDkyzqss JRJwWpdchatJ/T4VgA6fInajhnFh3PyIY6Kju7RT5/uAkbjA3Md1djkz2I/w3rHrgWtu pWAQ== X-Forwarded-Encrypted: i=1; AJvYcCUzFSWfS71PIrFF/F2wlOWe6WhFJ0Dq5mQ9T+R9mobFdUspHoWMOzdYruJUt0qvqNHfOH04VhIZLt0o5fc=@vger.kernel.org X-Gm-Message-State: AOJu0Yx6+2SrMK08zzK2QaCeq+BoYG17J+O4OLlje+uGuyqdy05KqNFo i4BhedKZHAo7/OQ/JEU4sNnN6LnDG0iWmZ42YGQjuckNQZuJODYTb/CcZmenLYW4EA== X-Gm-Gg: ASbGncu2oBJMzUUx8/qLhxnWup1iXKbticFyjax+ulnxG75RMW7DIsjS0OHiAP+5rhg s9OkOdGZzIc+j52BDDWhhmAXnkG0JrSvEsP2etkGm0vDMdd+BwHg8q9QLkPYg9qJ80wBu4ltqgT OA1o5Wz80vQHqlMWOOD8McubeHpluT+k+fNapya+zGyn70H/GybMeN+AmusSiaGD22hzF9XBkHY vhG+Op5nsR2KwbdgVby48gQdRkSazmmns1KAkdYA7fm9UvPeDJgCzbm6VEAm+Y71TUlkc4PyuyA KkN/iZoWUxLIxI48PpF4Fblv3JRFj5TFRAfufhmqB5u6xntmwzI66lfkGEcdgugv7Ao1uTflAsX mJ9l6At8EKxPoc4JoHzHIzwZJoB/j0EnFtsGQ6T6OG1U+xlKSWku0iQ== X-Google-Smtp-Source: AGHT+IGzPBkps6pgxo+xkRUOWRdcHJniXJOS0XDhoCKSiaRXOXMx/0guokH0JJv8zRRgcQP/nPr8eQ== X-Received: by 2002:a05:6602:2c06:b0:93b:b891:fdb2 with SMTP id ca18e2360f4ac-93bb89203f5mr871324539f.11.1759791770626; Mon, 06 Oct 2025 16:02:50 -0700 (PDT) Received: from chromium.org ([50.235.115.130]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-57b5ec55bc7sm5422226173.69.2025.10.06.16.02.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Oct 2025 16:02:49 -0700 (PDT) From: Simon Glass To: linux-arm-kernel@lists.infradead.org Cc: Chen-Yu Tsai , Ahmad Fatoum , Masahiro Yamada , =?UTF-8?q?J=20=2E=20Neusch=C3=A4fer?= , Nicolas Schier , Tom Rini , Simon Glass , linux-kernel@vger.kernel.org Subject: [PATCH v4 4/7] scripts/make_fit: Provide a way to add built modules Date: Mon, 6 Oct 2025 17:01:55 -0600 Message-ID: <20251006230205.521341-5-sjg@chromium.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251006230205.521341-1-sjg@chromium.org> References: <20251006230205.521341-1-sjg@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Provide arguments to support building a ramdisk from a directory tree of modules. This is a convenient way to try out a kernel with its modules. This makes use of the cpio tool rather than attempting to use a python module or our own code. The list of modules is provided in a file. Signed-off-by: Simon Glass Suggested-by: Ahmad Fatoum Reviewed-by: Nicolas Schier --- Changes in v4: - Provide the list of modules from the Makefile - Reduce verbosity (don't print every module filename) Changes in v3: - Add a way to add built modules into the FIT scripts/make_fit.py | 98 +++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 90 insertions(+), 8 deletions(-) diff --git a/scripts/make_fit.py b/scripts/make_fit.py index 1a74a9dcd85e..3db129f40b20 100755 --- a/scripts/make_fit.py +++ b/scripts/make_fit.py @@ -13,11 +13,17 @@ Usage: -r /boot/initrd.img-6.14.0-27-generic @arch/arm64/boot/dts/dtbs-li= st -E -c gzip =20 + # Build with modules ramdisk instead of external ramdisk: + make_fit.py -A arm64 -n 'Linux-6.17' -O linux + -o arch/arm64/boot/image.fit -k /tmp/kern/arch/arm64/boot/image.itk + -m module1.ko module2.ko module3.ko @arch/arm64/boot/dts/dtbs-list + Creates a FIT containing the supplied kernel, an optional ramdisk, and a s= et of devicetree files, either specified individually or listed in a file (with = an '@' prefix). =20 Use -r to specify an existing ramdisk/initrd file. +Use -m to build a ramdisk from specified kernel module files. =20 Use -E to generate an external FIT (where the data is placed after the FIT data structure). This allows parsing of the data without loading @@ -38,6 +44,7 @@ as U-Boot, Linuxboot, Tianocore, etc. import argparse import collections import os +import shutil import subprocess import sys import tempfile @@ -83,8 +90,14 @@ def parse_args(): help=3D'Specifies the operating system') parser.add_argument('-k', '--kernel', type=3Dstr, required=3DTrue, help=3D'Specifies the (uncompressed) kernel input file (.itk)') - parser.add_argument('-r', '--ramdisk', type=3Dstr, + + # Create mutually exclusive group for ramdisk options + rd_group =3D parser.add_mutually_exclusive_group() + rd_group.add_argument('-r', '--ramdisk', type=3Dstr, help=3D'Specifies the ramdisk/initrd input file') + rd_group.add_argument('-m', '--modules', type=3Dstr, nargs=3D'+', + help=3D'List of module filenames to include in ramdisk') + parser.add_argument('-v', '--verbose', action=3D'store_true', help=3D'Enable verbose output') parser.add_argument('dtbs', type=3Dstr, nargs=3D'*', @@ -240,6 +253,60 @@ def output_dtb(fsw, seq, fname, arch, compress): fsw.property('data', compressed) =20 =20 +def build_ramdisk(args, tmpdir): + """Build a cpio ramdisk containing kernel modules + + Similar to mkinitramfs, this creates a compressed cpio-archive contain= ing + the kernel modules for the current kernel version. + + Args: + args (Namespace): Program arguments + tmpdir (str): Temporary directory to use for modules installation + + Returns: + tuple: + bytes: Compressed cpio data containing modules + int: total uncompressed size + """ + suppress =3D None if args.verbose else subprocess.DEVNULL + + if args.verbose: + print(f'Copying {len(args.modules)} modules to ramdisk') + + # Create output-directory structure + outdir =3D os.path.join(tmpdir, 'initramfs') + modules_dir =3D os.path.join(outdir, 'usr', 'lib', 'modules') + os.makedirs(modules_dir, exist_ok=3DTrue) + + # Copy in the specified modules + for module in args.modules: + dest_path =3D os.path.join(modules_dir, os.path.basename(module)) + shutil.copy2(module, dest_path) + + if args.verbose: + print(f'Creating cpio archive from {outdir}') + + with tempfile.NamedTemporaryFile() as cpio_file: + # Change to initramfs directory and create cpio archive + with subprocess.Popen(['find', '.', '-print0'], cwd=3Doutdir, + stdout=3Dsubprocess.PIPE) as find: + with subprocess.Popen(['cpio', '-o', '-0', '-H', 'newc'], + stdin=3Dfind.stdout, stdout=3Dcpio_file, + stderr=3Dsuppress, cwd=3Doutdir) as cpio: + find.stdout.close() + cpio.wait() + find.wait() + + if cpio.returncode !=3D 0: + raise RuntimeError('Failed to create cpio archive') + + cpio_file.seek(0) # Reset to beginning for reading + if args.verbose: + print('Reading ramdisk...' if args.compress =3D=3D 'none' else + f'Compressing ramdisk with {args.compress}...') + return compress_data(cpio_file, args.compress), cpio_file.tell() + + def process_dtb(fname, args): """Process an input DTB, decomposing it if requested and is possible =20 @@ -318,11 +385,12 @@ def _process_dtbs(args, fsw, entries, fdts): return seq, size =20 =20 -def build_fit(args): +def build_fit(args, tmpdir): """Build the FIT from the provided files and arguments =20 Args: args (Namespace): Program arguments + tmpdir (str): Temporary directory for any temporary files =20 Returns: tuple: @@ -344,20 +412,29 @@ def build_fit(args): =20 # Handle the ramdisk if provided. Compression is not supported as it is # already compressed. + ramdisk_data =3D None if args.ramdisk: with open(args.ramdisk, 'rb') as inf: - data =3D inf.read() - size +=3D len(data) - write_ramdisk(fsw, data, args) + ramdisk_data =3D inf.read() + size +=3D len(ramdisk_data) + elif args.modules: + if args.verbose: + print('Building modules ramdisk...') + ramdisk_data, uncomp_size =3D build_ramdisk(args, tmpdir) + size +=3D uncomp_size + + if ramdisk_data: + write_ramdisk(fsw, ramdisk_data, args) =20 count, fdt_size =3D _process_dtbs(args, fsw, entries, fdts) size +=3D fdt_size =20 - finish_fit(fsw, entries, bool(args.ramdisk)) + finish_fit(fsw, entries, has_ramdisk=3Dbool(ramdisk_data)) =20 - # Include the kernel itself in the returned file count fdt =3D fsw.as_fdt() fdt.pack() + + # Count FDT files, kernel, plus ramdisk if present return fdt.as_bytearray(), count + 1 + bool(args.ramdisk), size =20 =20 @@ -365,7 +442,12 @@ def run_make_fit(): """Run the tool's main logic""" args =3D parse_args() =20 - out_data, count, size =3D build_fit(args) + tmpdir =3D tempfile.mkdtemp(prefix=3D'make_fit_') + try: + out_data, count, size =3D build_fit(args, tmpdir) + finally: + shutil.rmtree(tmpdir) + with open(args.output, 'wb') as outf: outf.write(out_data) =20 --=20 2.43.0 From nobody Wed Dec 17 21:25:30 2025 Received: from mail-il1-f172.google.com (mail-il1-f172.google.com [209.85.166.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 08B7721ABDC for ; Mon, 6 Oct 2025 23:02:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759791775; cv=none; b=CUWM7vsUZLLrDDu0DFxv8hI1xZX1ea8KdgovubaL+TU5lY1chhXJX09Kcpr4fCxRhFEE52IYw+xjCunblxC7mtrJB2m43UNS2ZmlZ6mrt+raFgWopi7VBleVB7K+itdByR/Et/Mw5LoLhP9luMD1Fzmca7mrxOW5ZNUPAvOLGR8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759791775; c=relaxed/simple; bh=J5q0jWsK5BjH042bjuBuYYFQ87rhLFCo1nf6x44DvAY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=RFuXsb65gs6FtnXmlFldDjvXMprQMGwGEcnzNn6/yeOezIiKyOv7LrKd/+xuIECiya3+bOLNBFmy4+V5/zRFmqkaEbpJPUldvwJfVpOf0EUBR6ILvLKvEPP6Jsvw4o/6N404kRV2vIEEd+o/h9W758gG6QnCvWgHIgA7DE/9sHU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=FpyspUHA; arc=none smtp.client-ip=209.85.166.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="FpyspUHA" Received: by mail-il1-f172.google.com with SMTP id e9e14a558f8ab-42e2c336adcso19139385ab.1 for ; Mon, 06 Oct 2025 16:02:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1759791773; x=1760396573; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VYASDCPfiPbCUV1KYUTYbx4wHolb9ntBgheoD945Sac=; b=FpyspUHAbqwZXazaT9iNZdMk6sRLYYyRHaevXs3MQRqkRMU5Kl25+Jf8X5vCfjAoDp NQ1FfYXbTtrFhD0i9DnyGBHuLYGMcM7/4ChLQqs4WXCkVFTYRkh7uzzgsHwAlupaNe/q 5ITTL3VRhb4jQufIfwgLYt6YbIxVqlyFEhARU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759791773; x=1760396573; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VYASDCPfiPbCUV1KYUTYbx4wHolb9ntBgheoD945Sac=; b=vXwKZDPjZXYn+9/YhLNfPHgZobCEHWEQbXbSH3IliOUFTnOVCqG/Ts6NPkmb7erinD CG2Haf42Sn8/G6ToBHAzkSo+mdgzGdkBC6NPAzafBv3Tv3X0SLKIwDQhwBXN7uYJZAbw hs8Dw+UnNXiU4Nk6kxKgbfGH9Gq+fE22J+UmaY5kJrxbNrwia4p1D7Af1z0IFv7lA5f/ pyyWOKq4Ti2Z43+UwzrU3gKJNS9aneVWeO9WJIz1X0epFOmsekXGOG1M85xXHjL4oIYG JNtU1HaUEfprH6OYPR+wdAgVV7G9qkMDMiIiGyc7qhZrka3LF4FI7LFM1lZBiX+QJ2AJ Totg== X-Forwarded-Encrypted: i=1; AJvYcCXphfDpKCZsqmXUz7qTBDlr/SE5qqbTyCHxWm2FJd34vtxctYax2jM8SgjjuoTFOyOQ3vrWkOZhLMmKKjw=@vger.kernel.org X-Gm-Message-State: AOJu0YzDM+CD1vFOe7vl1mwzwgubcpaT1d6PbIqFIs09QEH3nj929cfe h4eOc8QPLGWQd9TPdRzGvIh7lEy5p3+0A6mJvxeqB7k2IoqYM5wQ96efzpz86VLOcg== X-Gm-Gg: ASbGncvCdM4oyrmbkLFRhL1Z7KzI0xhLhoKP6c7buMtbIdwmW0uS2NQiaCjlyt20jYr DCdMVUq4ov3mwbOICqqpTfz73sSi1UYCrv8GOm/jB+Xze8Kl0UBC8hj3aBVN7c1cGvAyQhAY9PZ qOUk6JbWw/FVE3uKgcx7Y0wUZb1qCPqcOfU6gtSyqMY65/rkDb1zEMrXN05B9mm37UL4JoQ6pI0 UIRjI03KQ3x5jTwm+7Biaahntde1asGci2FVPPXib6SuT8ZNnXEIEy+723mJj/4BkquU8nmaPJS OxdEG4P78OkLPROipLxyU6Rmiw8V7CDb7LBfMXrk7iB/oPqnmVlPiGaQ7GmN2vXA1Y9wBIV5Kaz wCPjoegauKZzRjgakg3VF631Zc/VbGoD34xvMvBR7oss= X-Google-Smtp-Source: AGHT+IFWMMJp+NFmsOx6pq2dwd7Hy4kI7ynB4UcS1GCGMob/F9UqCyjwJZxXEHBZiF/C7t22Bn7ozg== X-Received: by 2002:a05:6e02:174e:b0:428:c370:d972 with SMTP id e9e14a558f8ab-42e7acedb7dmr164223205ab.7.1759791772886; Mon, 06 Oct 2025 16:02:52 -0700 (PDT) Received: from chromium.org ([50.235.115.130]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-57b5ec55bc7sm5422226173.69.2025.10.06.16.02.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Oct 2025 16:02:51 -0700 (PDT) From: Simon Glass To: linux-arm-kernel@lists.infradead.org Cc: Chen-Yu Tsai , Ahmad Fatoum , Masahiro Yamada , =?UTF-8?q?J=20=2E=20Neusch=C3=A4fer?= , Nicolas Schier , Tom Rini , Simon Glass , Nathan Chancellor , Ard Biesheuvel , Catalin Marinas , Josh Poimboeuf , Kees Cook , Nicolas Schier , Rong Xu , Will Deacon , linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 5/7] kbuild: Allow adding modules into the FIT ramdisk Date: Mon, 6 Oct 2025 17:01:56 -0600 Message-ID: <20251006230205.521341-6-sjg@chromium.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251006230205.521341-1-sjg@chromium.org> References: <20251006230205.521341-1-sjg@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Support 'make image.fit FIT_MODULES=3D1' to put all the modules into a ramdisk image within the FIT. Signed-off-by: Simon Glass Suggested-by: Ahmad Fatoum Acked-by: Nathan Chancellor --- Changes in v4: - Rename the Makefile variable from 'EXTRA' to 'MAKE_FIT_FLAGS' - Use an empty FIT_MODULES to disable the feature, instead of '0' - Make use of the 'modules' dependency to ensure modules are built - Pass the list of modules to the script arch/arm64/boot/Makefile | 4 ++++ scripts/Makefile.lib | 10 ++++++++-- 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/arch/arm64/boot/Makefile b/arch/arm64/boot/Makefile index b5a08333bc57..d10c85f96aea 100644 --- a/arch/arm64/boot/Makefile +++ b/arch/arm64/boot/Makefile @@ -43,6 +43,10 @@ $(obj)/Image.zst: $(obj)/Image FORCE $(obj)/Image.xz: $(obj)/Image FORCE $(call if_changed,xzkern) =20 +ifeq ($(FIT_MODULES),1) +.PHONY: modules +$(obj)/image.fit: modules +endif $(obj)/image.fit: $(obj)/Image $(obj)/dts/dtbs-list FORCE $(call if_changed,fit) =20 diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib index 1d581ba5df66..c6a3aa653035 100644 --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -398,11 +398,17 @@ MAKE_FIT :=3D $(srctree)/scripts/make_fit.py # Use this to override the compression algorithm FIT_COMPRESSION ?=3D gzip =20 +# Set this to 1 to include an initrd with all the kernel modules +FIT_MODULES ?=3D + quiet_cmd_fit =3D FIT $@ - cmd_fit =3D $(MAKE_FIT) -o $@ --arch $(UIMAGE_ARCH) --os linux \ - --name '$(UIMAGE_NAME)' \ + cmd_fit =3D $(if $(FIT_MODULES), \ + find $(objtree) -name '*.ko' > $(objtree)/.modules-list 2>/dev/null &&) \ + $(MAKE_FIT) -o $@ --arch $(UIMAGE_ARCH) --os linux \ + --name '$(UIMAGE_NAME)' $(MAKE_FIT_FLAGS) \ $(if $(findstring 1,$(KBUILD_VERBOSE)),-v) \ $(if $(FIT_DECOMPOSE_DTBS),--decompose-dtbs) \ + $(if $(FIT_MODULES),--modules @$(objtree)/.modules-list) \ --compress $(FIT_COMPRESSION) -k $< @$(word 2,$^) =20 # XZ --=20 2.43.0 From nobody Wed Dec 17 21:25:30 2025 Received: from mail-il1-f170.google.com (mail-il1-f170.google.com [209.85.166.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2924822156C for ; Mon, 6 Oct 2025 23:02:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759791777; cv=none; b=q4oFey5NYx778cQF7QtuaHR0BH2QgHivHKgNUecrV0jxh//cPcdNVZP2+W3M6Ec3u+xZ7EdDZ9qNxR5O0CjVMilNJySoH6dZAgfPsqRrKSw0u3hXdsBJPhEKljKD5eAs9zpdqRcKZC38tSib76XaHTfy7X7bfN6881q0KzyfwfY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759791777; c=relaxed/simple; bh=8ZGGs3PQhqVTmNk+d9LVkL248C8uJ8HMmqu52Ln87QA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=l0CuOqCy7jeGJXGLGYyz7XmrCTAUg0/he0+N7rCGxz+gT5GejAnxMdAj5We3gzfDjDrqZ4XfLKjVTgStFNB2asxpNeeoFjOPCnhzDinj1NUrvA5KdBBJDjd26COGmGk5KWAENWQzNqau5Ggboogs2/A1DvlIMfiYorgG8Po/Yt8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=RDwkfA7k; arc=none smtp.client-ip=209.85.166.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="RDwkfA7k" Received: by mail-il1-f170.google.com with SMTP id e9e14a558f8ab-42f67e91ae7so27959075ab.0 for ; Mon, 06 Oct 2025 16:02:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1759791775; x=1760396575; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MuN93K9Dcy3JOdIK4h07lOL7a8R3BcxISOfJinfzFRA=; b=RDwkfA7kO8rkaJi4BCk8yLzklLnDQm7rs+mEHKUQ3oakCArL1+vkcukRpuuX5V8wWa UukqT06Tg3Q46lZGpfWRT5lSaL5+JR1/6DbLuw33t1eLO27WcF3OmEHPjZnIaqAl16fY oXjVF4HUYFiec4oVGN4chNqVHif+c6HEe4vrs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759791775; x=1760396575; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MuN93K9Dcy3JOdIK4h07lOL7a8R3BcxISOfJinfzFRA=; b=ngJEMWoqgYoq46AYJnb8VZTmYZxEt1EJeSbgWF8nCSzHgKUeSbhswDMkpA5RT4wxdI 6O8hTD6CrpgDqqN9pANjiYIYiOk5U55FZzt5pOD4tBZNk36JrItYjTjAatO3ViWHFydr uDhKCDDyLFq9MZbIXBCq7A+pWesVH2gKqfyjmnk+VWtnSg/ghzqBzl5QRgaJaevug1mD AVA/LZpyueRRoGgzMUMz2IAmsKczZIHCoAB0coHMs0N2pCqi4hEdNEUC6pW0dZ+uAWIC +Oeso4330YHLYXJ7LZ78jz20/d4LbCsWC228KW0m74iWcIz4/1kujjYlrwL1eD3C1SHB 9Eog== X-Forwarded-Encrypted: i=1; AJvYcCWomF7hpGsxXNlemfqVIgDlaZMZ1/t2sZz/Sc/r0nc8InR9GFwLBNK2hsPFldzCqR/dGpFqtmKo6NqyMm4=@vger.kernel.org X-Gm-Message-State: AOJu0YxxKUjWEF3EneGKqEOYontjgkvzmJ85OUNhuzeoFCMvduxWY9Vq dKoPl4OQokToqcx15Tm8o+FL8Ddm7KdkidLhfC8w3Dk7iHiMuVGekbZaEDIEX2sjfFOZUVhQ6Mw bKXktwA== X-Gm-Gg: ASbGncuOu0GnIjgD3MAl055FGPpxvsKQIU/T9phGz5hV9/2EAv/4xAYSnvdSHCVPOpH 9Z22VE06rQiFAh/baS4o7hiXqWRP/ZzG9Tk/XH0gnGbJZ5F7Z8XDA3aDwYYA3Nm0fxaL9BVWgpm bhPvieu3ZoedTRq5B7cY6XgKKHPiFJgQT8BflxiJPEr2ygKDXe12k0MDRGqPRchcaKrWEJOabmJ ikhvWju72BeZDnbUwYuXHIRrwZ4Dxy2Lnmgo1DeGSfN6agVZlm0pRtfrh0yHGihhtVpcb2MwKu3 n1y3fkd7DP2kgMYaSxK1aZU07uJVnSe5akJVJvgvfdnSQd0IqoN0RRM7QEKD66krLWGRTo1c9bm aMoXX9f9ZkyV6OoBUKOJ9bTtc+rASBnjUgT0sbGJ5b5s= X-Google-Smtp-Source: AGHT+IHSrmzWbWKM4UXhJ9zCemfI3AXTkSgm4z+CxyCTVL+PvyrUOX4Z4dhsZa1vuM2vjJ4J/tIMig== X-Received: by 2002:a05:6e02:1fec:b0:42e:7598:73eb with SMTP id e9e14a558f8ab-42e7ad9cbc8mr205043605ab.23.1759791775171; Mon, 06 Oct 2025 16:02:55 -0700 (PDT) Received: from chromium.org ([50.235.115.130]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-57b5ec55bc7sm5422226173.69.2025.10.06.16.02.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Oct 2025 16:02:54 -0700 (PDT) From: Simon Glass To: linux-arm-kernel@lists.infradead.org Cc: Chen-Yu Tsai , Ahmad Fatoum , Masahiro Yamada , =?UTF-8?q?J=20=2E=20Neusch=C3=A4fer?= , Nicolas Schier , Tom Rini , Simon Glass , David Sterba , Nick Terrell , linux-kernel@vger.kernel.org Subject: [PATCH v4 6/7] scripts/make_fit: Support a few more parallel compressors Date: Mon, 6 Oct 2025 17:01:57 -0600 Message-ID: <20251006230205.521341-7-sjg@chromium.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251006230205.521341-1-sjg@chromium.org> References: <20251006230205.521341-1-sjg@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add support for pbzip2 and plzip which can compress in parallel. This speeds up the ramdisk compression. Signed-off-by: Simon Glass --- (no changes since v1) scripts/make_fit.py | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/scripts/make_fit.py b/scripts/make_fit.py index 3db129f40b20..10a040f4eb83 100755 --- a/scripts/make_fit.py +++ b/scripts/make_fit.py @@ -57,10 +57,10 @@ import libfdt CompTool =3D collections.namedtuple('CompTool', 'ext,tools') =20 COMP_TOOLS =3D { - 'bzip2': CompTool('.bz2', 'bzip2'), + 'bzip2': CompTool('.bz2', 'pbzip2,bzip2'), 'gzip': CompTool('.gz', 'pigz,gzip'), 'lz4': CompTool('.lz4', 'lz4'), - 'lzma': CompTool('.lzma', 'lzma'), + 'lzma': CompTool('.lzma', 'plzip,lzma'), 'lzo': CompTool('.lzo', 'lzop'), 'zstd': CompTool('.zstd', 'zstd'), } @@ -220,7 +220,12 @@ def compress_data(inf, compress): done =3D False for tool in comp.tools.split(','): try: - subprocess.call([tool, '-c'], stdin=3Dinf, stdout=3Dou= tf) + # Add parallel flags for tools that support them + cmd =3D [tool] + if tool in ('zstd', 'xz'): + cmd.extend(['-T0']) # Use all available cores + cmd.append('-c') + subprocess.call(cmd, stdin=3Dinf, stdout=3Doutf) done =3D True break except FileNotFoundError: --=20 2.43.0 From nobody Wed Dec 17 21:25:30 2025 Received: from mail-il1-f172.google.com (mail-il1-f172.google.com [209.85.166.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 593182153EA for ; Mon, 6 Oct 2025 23:02:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759791780; cv=none; b=Pfvdv9kUfbxXS9pmjmPQTS8jf7GFVZfDfZJjfsApRL0q1pYf6CFRBBYmA6KVLcQOaiBgs+IB4FRvdNIyJj9FWnCXDBRwTQV4ZmBORd+LvmZmdlFblC2g3Y9TEl5kquakgP2K8U42ZaTjQ8s4rfCjveH6HvYPp/NFXab6Q+T9ai4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759791780; c=relaxed/simple; bh=iRwr4aWqYl4kIwimtlljRUSeu5ezG3oP5PG69wjW44g=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=gDFP2whg1XhHzjTUiqotT0slT183Y0wgMtR4CBmsV7AvxPOmti4vr+4BBF4QV4UR3wsF2S0NaALqzH65o6L2ypeGSul3eae36fBjZNOeFXcQqvAJAVoeUqIIHjC8CYSXoSoiCOBd2dCZaUgGlbFRtKMmFnNQEDCmzyLKaTJrJCE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=N0a+XvsU; arc=none smtp.client-ip=209.85.166.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="N0a+XvsU" Received: by mail-il1-f172.google.com with SMTP id e9e14a558f8ab-42f05cbc3d1so17609485ab.0 for ; Mon, 06 Oct 2025 16:02:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1759791777; x=1760396577; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=b318BeGWA3r29XTQ0vNTU6rijD9/dIK2iMOcNHHPjBE=; b=N0a+XvsUkqx7r+lxqvMehweZPQ02EQGTxsWU9qLOUyhvvIfDAwJQwQwo7xDPODHels GFB5e/Amr8+JkETntJ6MK0Y3IeFJpYkmcc8I5l3vvv9Z7HpuVKe1quXgzbLNfbyuo/bJ LQET3qr/PfskCg5hIUhFUtB3C8t/P8RjTlSXU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759791777; x=1760396577; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=b318BeGWA3r29XTQ0vNTU6rijD9/dIK2iMOcNHHPjBE=; b=YwjCgb5bCBfoSwVnOGgg2R3XU4sQYIDXyuvWxnAyhFeGpK5u8GhLmF9k0uXTCkP58r LDarytLrXYJ6ytzdIz/p0aJC0pZqt/5qHLPDAhbdTq6VK5NMqw/y/k16DRjPS3GjrVCV ldAunOS7pWxKmw9WRnYmD3MCXuHn5GTaRLy84bNZoAGHANZdLiCIueX9QFihrdMurjl6 cV3yDzkZ2ScaKFh+EOuqi9itTiv83yislJ4Wf2h6Ca7Gk/1+hRgJ6+cWoFHUjOGTsPar HFFvwxa1P4pJmYtJmimaakcIKLK5y3FQtgSEEQo5arnULN1T8mlkssz+YHrwR4H8pYGi +nYw== X-Forwarded-Encrypted: i=1; AJvYcCUFl0BS7bSvD5re2KaLkpfw5Zwbb7MUZBqw78XXkj/TIbVBOlJGcpSRHKvesdNqCSqIq7l4NZPL72pra7k=@vger.kernel.org X-Gm-Message-State: AOJu0Yyr6aY1W79ZFyxsjf577X55uc/Bag96LRkLAT6nvE/ABd4RyI0r wLZ5xIHyfz5Uyx3q02tN/UjuwvLZoqvUgw6mo6db2DSB1IpFlSxJFTmI167/kObPyQ== X-Gm-Gg: ASbGnct4YEBbegmoKuogyx6bOjXGxh+kXR6MxF5enEDVTPI00WU1j5u4S0EGI7W+gqs EbWjsNEMmVN6yg6q/BxXDvNLTyoUjRaMBumZq+c6QU9xiwlqhlZ4KNbuerKcoTLPKG66e8SvMca Kdw+3zPsGcog2KsFX2nCunqNUqASUXQHUKgHgfgM8F1V8gYBNj0byOeLVPXqG7fpSmf4zVeeoUN kzHGdJAa6T3wKVMgx1sU65zKlinJjArPPsuSf52vbm1J6FQncquiDnZK/KuO/uUSOd3jKEs+wBb 1YTewmMz1i6iXgTTT/mpPsEFgr9Z3qBHOJ9m0vSo9FqDbuSB1RYMqhoV3UbX8hO3FofgHszj8zG HqAue3YLqNTho6Bq1cvEiohzT37yLcJHRcTxTsQ4q1ew= X-Google-Smtp-Source: AGHT+IG7h6VEIrYxcywgIQo3xi3mnzcDBjSio3qPq2k9Vpojaaa/mhuStt80mORSz4TB5A7diQ4H6A== X-Received: by 2002:a05:6e02:12c1:b0:427:215a:93fa with SMTP id e9e14a558f8ab-42f7c349cd6mr17245875ab.2.1759791777236; Mon, 06 Oct 2025 16:02:57 -0700 (PDT) Received: from chromium.org ([50.235.115.130]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-57b5ec55bc7sm5422226173.69.2025.10.06.16.02.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Oct 2025 16:02:55 -0700 (PDT) From: Simon Glass To: linux-arm-kernel@lists.infradead.org Cc: Chen-Yu Tsai , Ahmad Fatoum , Masahiro Yamada , =?UTF-8?q?J=20=2E=20Neusch=C3=A4fer?= , Nicolas Schier , Tom Rini , Simon Glass , linux-kernel@vger.kernel.org Subject: [PATCH v4 7/7] scripts/make_fit: Compress dtbs in parallel Date: Mon, 6 Oct 2025 17:01:58 -0600 Message-ID: <20251006230205.521341-8-sjg@chromium.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251006230205.521341-1-sjg@chromium.org> References: <20251006230205.521341-1-sjg@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When there are 1500 device tree files it takes quite a while to compress them. Do it in parallel. Signed-off-by: Simon Glass --- (no changes since v1) scripts/make_fit.py | 56 ++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 50 insertions(+), 6 deletions(-) diff --git a/scripts/make_fit.py b/scripts/make_fit.py index 10a040f4eb83..d1af496f3a57 100755 --- a/scripts/make_fit.py +++ b/scripts/make_fit.py @@ -43,6 +43,7 @@ as U-Boot, Linuxboot, Tianocore, etc. =20 import argparse import collections +import multiprocessing import os import shutil import subprocess @@ -237,15 +238,31 @@ def compress_data(inf, compress): return comp_data =20 =20 -def output_dtb(fsw, seq, fname, arch, compress): +def compress_dtb(fname, compress): + """Compress a single DTB file + + Args: + fname (str): Filename containing the DTB + compress (str): Compression algorithm, e.g. 'gzip' + + Returns: + tuple: (str: fname, bytes: compressed_data) + """ + with open(fname, 'rb') as inf: + compressed =3D compress_data(inf, compress) + return fname, compressed + + +def output_dtb(fsw, seq, fname, arch, compress, data=3DNone): """Write out a single devicetree to the FIT =20 Args: fsw (libfdt.FdtSw): Object to use for writing seq (int): Sequence number (1 for first) fname (str): Filename containing the DTB - arch: FIT architecture, e.g. 'arm64' + arch (str): FIT architecture, e.g. 'arm64' compress (str): Compressed algorithm, e.g. 'gzip' + data (bytes): Pre-compressed data (optional) """ with fsw.add_node(f'fdt-{seq}'): fsw.property_string('description', os.path.basename(fname)) @@ -253,9 +270,10 @@ def output_dtb(fsw, seq, fname, arch, compress): fsw.property_string('arch', arch) fsw.property_string('compression', compress) =20 - with open(fname, 'rb') as inf: - compressed =3D compress_data(inf, compress) - fsw.property('data', compressed) + if data is None: + with open(fname, 'rb') as inf: + data =3D compress_data(inf, compress) + fsw.property('data', data) =20 =20 def build_ramdisk(args, tmpdir): @@ -366,6 +384,11 @@ def _process_dtbs(args, fsw, entries, fdts): """ seq =3D 0 size =3D 0 + + # First figure out the unique DTB files that need compression + todo =3D [] + file_info =3D [] # List of (fname, model, compat, files) tuples + for fname in args.dtbs: # Ignore non-DTB (*.dtb) files if os.path.splitext(fname)[1] !=3D '.dtb': @@ -377,11 +400,32 @@ def _process_dtbs(args, fsw, entries, fdts): sys.stderr.write(f'Error processing {fname}:\n') raise e =20 + file_info.append((fname, model, compat, files)) + for fn in files: + if fn not in fdts and fn not in todo: + todo.append(fn) + + # Compress all DTBs in parallel + cache =3D {} + if todo and args.compress !=3D 'none': + if args.verbose: + print(f'Compressing {len(todo)} DTBs...') + + with multiprocessing.Pool() as pool: + compress_args =3D [(fn, args.compress) for fn in todo] + # unpacks each tuple, calls compress_dtb(fn, compress) in para= llel + results =3D pool.starmap(compress_dtb, compress_args) + + cache =3D dict(results) + + # Now write all DTBs to the FIT using pre-compressed data + for fname, model, compat, files in file_info: for fn in files: if fn not in fdts: seq +=3D 1 size +=3D os.path.getsize(fn) - output_dtb(fsw, seq, fn, args.arch, args.compress) + output_dtb(fsw, seq, fn, args.arch, args.compress, + cache.get(fn)) fdts[fn] =3D seq =20 files_seq =3D [fdts[fn] for fn in files] --=20 2.43.0