From nobody Sat Jun 13 14:13:45 2026 Received: from cstnet.cn (smtp81.cstnet.cn [159.226.251.81]) (using TLSv1.2 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8BF0E287510 for ; Thu, 7 May 2026 05:09:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=159.226.251.81 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778130597; cv=none; b=GLo9siI45Zswv2B46sD+EMtdv+ca/pjPA5YSLBPhcAmPrKyGxwXOoTrxOD5tMmx+rJDO3+PU8dbCJuA6wOGf2bgh2XJqwkmNf3v2vjeCFZE89WfZ0LQPO9OX5Fc1GmdzgeeKJd7r36kgYUy3/+Xl4ScT9ZR4uPb46O76Jt8P0yM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778130597; c=relaxed/simple; bh=zfEdAemJqT7QhDQBXPC0foBACGuXC/dFBMEvUSVp+Do=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=WgQSc+Ao8DPDoH3J2W95PZJAYc1uXS+RfXIKilZ+GZ1xS7dHoNx7B43eO7bi6lyNH14CL/KwZDPFWIqmcQl9kvyPtBb65rMOFpS0aqWeiDjDhAA/h4P86FAUt9f/44krwkV1TzSWw8e6UCe+yPZvytazPYroSHBHI1Upee8InzI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=mails.ucas.ac.cn; spf=pass smtp.mailfrom=mails.ucas.ac.cn; arc=none smtp.client-ip=159.226.251.81 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=mails.ucas.ac.cn Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=mails.ucas.ac.cn Received: from localhost.localdomain (unknown [14.22.11.162]) by APP-03 (Coremail) with SMTP id rQCowAD3AsqSHvxpMmhLEA--.42925S2; Thu, 07 May 2026 13:09:40 +0800 (CST) From: Mengdie Yan To: linux-erofs@lists.ozlabs.org Cc: hudsonzhu@tencent.com, xiang@kernel.org, chao@kernel.org, jefflexu@linux.alibaba.com, linux-kernel@vger.kernel.org, Mengdie Yan Subject: [PATCH] erofs-utils: mkfs: support hot-file-list for tar and OCI full sources Date: Thu, 7 May 2026 13:09:38 +0800 Message-ID: <20260507050938.2388894-1-yanmengdie24@mails.ucas.ac.cn> X-Mailer: git-send-email 2.43.7 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: rQCowAD3AsqSHvxpMmhLEA--.42925S2 X-Coremail-Antispam: 1UD129KBjvJXoW3KF1UGrWxWrW8ZFWkCF4kZwb_yoWkWrW5pr 4akr4rX3y8KFy7uw4IqF1a9r1ag3Wktr47KaySgrs5JFn5JrsrtF4kArWjqay5Wrs5XrWY qr429ay3ur1DJF7anT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUvC14x267AKxVW8JVW5JwAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2ocxC64kIII0Yj41l84x0c7CEw4AK67xGY2AK02 1l84ACjcxK6xIIjxv20xvE14v26ryj6F1UM28EF7xvwVC0I7IYx2IY6xkF7I0E14v26r4j 6F4UM28EF7xvwVC2z280aVAFwI0_Cr1j6rxdM28EF7xvwVC2z280aVCY1x0267AKxVW0oV Cq3wAac4AC62xK8xCEY4vEwIxC4wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC 0VAKzVAqx4xG6I80ewAv7VC0I7IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr 1lOx8S6xCaFVCjc4AY6r1j6r4UM4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IE rcIFxwCY1x0262kKe7AKxVWUAVWUtwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbV WUJVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF 67kF1VAFwI0_JF0_Jw1lIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUJVWUCwCI42 IY6xIIjxv20xvEc7CjxVAFwI0_Jr0_Gr1lIxAIcVCF04k26cxKx2IYs7xG6r1j6r1xMIIF 0xvEx4A2jsIE14v26r1j6r4UMIIF0xvEx4A2jsIEc7CjxVAFwI0_Jr0_GrUvcSsGvfC2Kf nxnUUI43ZEXa7VUbsYFJUUUUU== X-CM-SenderInfo: 51dqzv5qjgxvysu6ztxlovh3xfdvhtffof0/ Content-Type: text/plain; charset="utf-8" Extract hot-file marking helpers so non-local import paths can reuse the existing ranking logic. This extends --hot-file-list support from local directories to tar full mode and OCI full mode while keeping index, rvsp and zerofill modes rejected. For tar/OCI imports, hot ranks now follow regular entries, hardlink aliases and auto-created parent directories so hot files remain in the front layout region even when the source stream does not come from a local rootfs walk. Add regression tests for tar full mode, hardlink aliases, unsupported non-full tar modes, and compressed tar full builds so hot layout remains effective with zstd compression as well. --- include/erofs/inode.h | 2 + lib/inode.c | 48 ++++++++++++++++-------- lib/rebuild.c | 1 + lib/tar.c | 2 + mkfs/main.c | 26 +++++++++++-- tests/hotfile-layout.sh | 82 +++++++++++++++++++++++++++++++++++++++++ 6 files changed, 142 insertions(+), 19 deletions(-) diff --git a/include/erofs/inode.h b/include/erofs/inode.h index bf089e8..9aef660 100644 --- a/include/erofs/inode.h +++ b/include/erofs/inode.h @@ -43,6 +43,8 @@ int erofs_allocate_inode_bh_data(struct erofs_inode *inod= e, erofs_blk_t nblocks, bool erofs_dentry_is_wht(struct erofs_sb_info *sbi, struct erofs_dentry *d= ); int __erofs_fill_inode(struct erofs_importer *im, struct erofs_inode *inod= e, struct stat *st, const char *path); +void erofs_inode_set_hot(struct erofs_inode *inode, const char *path); +void erofs_inode_update_hot_file(struct erofs_inode *inode, const char *pa= th); struct erofs_inode *erofs_new_inode(struct erofs_sb_info *sbi); int erofs_importer_load_tree(struct erofs_importer *im, bool rebuild, bool incremental); diff --git a/lib/inode.c b/lib/inode.c index cabe085..1fae8c2 100644 --- a/lib/inode.c +++ b/lib/inode.c @@ -1326,6 +1326,37 @@ int __erofs_fill_inode(struct erofs_importer *im, st= ruct erofs_inode *inode, return 0; } =20 +void erofs_inode_set_hot(struct erofs_inode *inode, const char *path) +{ + inode->hot_rank =3D EROFS_HOT_RANK_NONE; + inode->hotfile =3D false; + inode->hotdir =3D false; + + if (erofs_is_special_identifier(path)) + return; + + inode->hot_rank =3D erofs_get_hot_file_rank(path); + inode->hotfile =3D inode->hot_rank !=3D EROFS_HOT_RANK_NONE; + if (!inode->hotfile && S_ISDIR(inode->i_mode)) { + inode->hot_rank =3D erofs_get_hot_dir_rank(path); + inode->hotdir =3D inode->hot_rank !=3D EROFS_HOT_RANK_NONE; + } +} + +void erofs_inode_update_hot_file(struct erofs_inode *inode, const char *pa= th) +{ + unsigned int rank; + + if (erofs_is_special_identifier(path)) + return; + + rank =3D erofs_get_hot_file_rank(path); + if (rank !=3D EROFS_HOT_RANK_NONE && rank < inode->hot_rank) { + inode->hot_rank =3D rank; + inode->hotfile =3D true; + } +} + static int erofs_fill_inode(struct erofs_importer *im, struct erofs_inode = *inode, struct stat *st, const char *path) { @@ -1363,15 +1394,7 @@ static int erofs_fill_inode(struct erofs_importer *i= m, struct erofs_inode *inode if (!inode->i_srcpath) return -ENOMEM; } - inode->hot_rank =3D EROFS_HOT_RANK_NONE; - if (!erofs_is_special_identifier(path)) { - inode->hot_rank =3D erofs_get_hot_file_rank(path); - inode->hotfile =3D inode->hot_rank !=3D EROFS_HOT_RANK_NONE; - if (!inode->hotfile && S_ISDIR(st->st_mode)) { - inode->hot_rank =3D erofs_get_hot_dir_rank(path); - inode->hotdir =3D inode->hot_rank !=3D EROFS_HOT_RANK_NONE; - } - } + erofs_inode_set_hot(inode, path); =20 if (erofs_should_use_inode_extended(im, inode, path)) { if (params->force_inodeversion =3D=3D EROFS_FORCE_INODE_COMPACT) { @@ -1445,12 +1468,7 @@ static struct erofs_inode *erofs_iget_from_local(str= uct erofs_importer *im, if (!S_ISDIR(st.st_mode) && !params->hard_dereference) { inode =3D erofs_iget(st.st_dev, st.st_ino); if (inode) { - u32 rank =3D erofs_get_hot_file_rank(path); - - if (rank !=3D EROFS_HOT_RANK_NONE && rank < inode->hot_rank) { - inode->hot_rank =3D rank; - inode->hotfile =3D true; - } + erofs_inode_update_hot_file(inode, path); return inode; } } diff --git a/lib/rebuild.c b/lib/rebuild.c index 74bbeda..371a542 100644 --- a/lib/rebuild.c +++ b/lib/rebuild.c @@ -61,6 +61,7 @@ static struct erofs_dentry *erofs_rebuild_mkdir(struct er= ofs_inode *dir, inode->i_mtime_nsec =3D dir->i_mtime_nsec; inode->dev =3D dir->dev; inode->i_nlink =3D 2; + erofs_inode_set_hot(inode, inode->i_srcpath); =20 d =3D erofs_d_alloc(dir, s); if (IS_ERR(d)) { diff --git a/lib/tar.c b/lib/tar.c index d2dc141..8052249 100644 --- a/lib/tar.c +++ b/lib/tar.c @@ -1039,6 +1039,7 @@ out_eot: =20 inode =3D erofs_igrab(d2->inode); ++inode->i_nlink; + erofs_inode_update_hot_file(inode, eh.path); if (d->type !=3D EROFS_FT_UNKNOWN) { tarerofs_remove_inode(d->inode); erofs_iput(d->inode); @@ -1096,6 +1097,7 @@ new_inode: ret =3D __erofs_fill_inode(im, inode, &st, eh.path); if (ret) goto out; + erofs_inode_set_hot(inode, eh.path); inode->i_size =3D st.st_size; =20 if (!S_ISDIR(inode->i_mode)) { diff --git a/mkfs/main.c b/mkfs/main.c index c154247..7ed7844 100644 --- a/mkfs/main.c +++ b/mkfs/main.c @@ -213,8 +213,8 @@ static void usage(int argc, char **argv) " --gid-offset=3D# add offset # to all file gids (# =3D id offse= t)\n" " --hard-dereference dereference hardlinks, add links as separate in= odes\n" " --hot-file-list=3DX specify newline-separated hot file paths for\= n" - " local directory sources; local rootfs builds\n" - " resolve symlink aliases\n" + " local directory, tar full, or OCI full sources;= \n" + " local rootfs builds resolve symlink aliases\n" " (e.g. /lib/... -> /usr/lib/...) and prioritize\= n" " ancestor directories as well\n" " --ignore-mtime use build time instead of strict per-file modif= ication time\n" @@ -331,6 +331,24 @@ static enum { EROFS_MKFS_SOURCE_REBUILD, } source_mode; =20 +static bool mkfs_hotfile_supported_source(void) +{ + switch (source_mode) { + case EROFS_MKFS_SOURCE_LOCALDIR: + return true; + case EROFS_MKFS_SOURCE_TAR: + return !erofstar.index_mode && + dataimport_mode !=3D EROFS_MKFS_DATA_IMPORT_RVSP && + dataimport_mode !=3D EROFS_MKFS_DATA_IMPORT_ZEROFILL; + case EROFS_MKFS_SOURCE_OCI: + return !mkfs_oci_tarindex_mode && + dataimport_mode !=3D EROFS_MKFS_DATA_IMPORT_RVSP && + dataimport_mode !=3D EROFS_MKFS_DATA_IMPORT_ZEROFILL; + default: + return false; + } +} + static unsigned int rebuild_src_count; static LIST_HEAD(rebuild_src_list); static u8 fixeduuid[16]; @@ -1557,8 +1575,8 @@ static int mkfs_parse_options_cfg(struct erofs_import= er_params *params, return err; } =20 - if (hotfile_list_path && source_mode !=3D EROFS_MKFS_SOURCE_LOCALDIR) { - erofs_err("--hot-file-list is only supported for local directory sources= "); + if (hotfile_list_path && !mkfs_hotfile_supported_source()) { + erofs_err("--hot-file-list is only supported for local directories, tar = full mode, and OCI full mode"); return -EOPNOTSUPP; } =20 diff --git a/tests/hotfile-layout.sh b/tests/hotfile-layout.sh index 6af3f41..659972a 100755 --- a/tests/hotfile-layout.sh +++ b/tests/hotfile-layout.sh @@ -179,6 +179,53 @@ assert_root_dirdata_precedes_hot_file() { fi } =20 +assert_tar_hot_file_precedes_cold_file() { + img=3D"$1" + + hot=3D$(extent_start /a/hot "$img") + cold=3D$(extent_start /b/cold "$img") + + if [ "$hot" -ge "$cold" ]; then + echo "tar hot file was not placed before cold file: hot=3D$hot cold=3D$c= old" >&2 + exit 1 + fi +} + +assert_tar_hot_hardlink_alias_precedes_cold_file() { + img=3D"$1" + + hot=3D$(extent_start /b/hot-alias "$img") + cold=3D$(extent_start /a/cold "$img") + links=3D$(inode_links /b/hot-alias "$img") + + if [ "$hot" -ge "$cold" ]; then + echo "tar hot hardlink alias was not placed before cold file: hot=3D$hot= cold=3D$cold" >&2 + exit 1 + fi + + if [ "$links" -ne 2 ]; then + echo "tar hot hardlink alias changed link count: links=3D$links" >&2 + exit 1 + fi +} + +assert_hotfile_mode_rejected() { + log=3D"$1" + shift + + if "$@" >"$log" 2>&1; then + echo "unsupported hot-file-list mode unexpectedly succeeded" >&2 + cat "$log" >&2 + exit 1 + fi + + if ! grep -q -- '--hot-file-list is only supported' "$log"; then + echo "unsupported hot-file-list mode failed with an unexpected error" >&2 + cat "$log" >&2 + exit 1 + fi +} + root=3D"$tmpdir/root" mkdir -p "$root/a" "$root/c" printf hot > "$root/a/hot" @@ -319,3 +366,38 @@ printf '/zz/hot\n' > "$tmpdir/hotlist.wide-root" "$MKFS" -d9 --hot-file-list=3D"$tmpdir/hotlist.wide-root" \ "$tmpdir/img.wide-root.erofs" "$root_wide" >"$tmpdir/mkfs.wide-root.log" = 2>&1 assert_root_dirdata_precedes_hot_file "$tmpdir/img.wide-root.erofs" + +root_tar=3D"$tmpdir/root-tar" +mkdir -p "$root_tar/a" "$root_tar/b" +dd if=3D/dev/zero bs=3D4096 count=3D64 of=3D"$root_tar/b/cold" status=3Dno= ne +dd if=3D/dev/zero bs=3D4096 count=3D64 of=3D"$root_tar/a/hot" status=3Dnone +(cd "$root_tar" && tar cf "$tmpdir/root.tar" b/cold a/hot) +printf '/a/hot\n' > "$tmpdir/hotlist.tar" +"$MKFS" -d9 --tar=3Df --hot-file-list=3D"$tmpdir/hotlist.tar" \ + "$tmpdir/img.tar.erofs" "$tmpdir/root.tar" >"$tmpdir/mkfs.tar.log" 2>&1 +assert_tar_hot_file_precedes_cold_file "$tmpdir/img.tar.erofs" +"$MKFS" -d9 -zzstd,level=3D9 --workers=3D1 --tar=3Df \ + --hot-file-list=3D"$tmpdir/hotlist.tar" \ + "$tmpdir/img.tar.zstd.erofs" "$tmpdir/root.tar" >"$tmpdir/mkfs.tar.zstd.l= og" 2>&1 +assert_tar_hot_file_precedes_cold_file "$tmpdir/img.tar.zstd.erofs" +assert_hotfile_mode_rejected "$tmpdir/mkfs.tar-index.log" \ + "$MKFS" --tar=3Di --hot-file-list=3D"$tmpdir/hotlist.tar" \ + "$tmpdir/img.tar-index.erofs" "$tmpdir/root.tar" +assert_hotfile_mode_rejected "$tmpdir/mkfs.tar-zerofill.log" \ + "$MKFS" --tar=3Df --clean=3D0 --hot-file-list=3D"$tmpdir/hotlist.tar" \ + "$tmpdir/img.tar-zerofill.erofs" "$tmpdir/root.tar" + +root_tar_hardlink=3D"$tmpdir/root-tar-hardlink" +mkdir -p "$root_tar_hardlink/a" "$root_tar_hardlink/b" "$root_tar_hardlink= /z" +dd if=3D/dev/zero bs=3D4096 count=3D64 of=3D"$root_tar_hardlink/a/cold" st= atus=3Dnone +dd if=3D/dev/zero bs=3D4096 count=3D64 of=3D"$root_tar_hardlink/z/original= " status=3Dnone +ln "$root_tar_hardlink/z/original" "$root_tar_hardlink/b/hot-alias" +(cd "$root_tar_hardlink" && tar cf "$tmpdir/root-hardlink.tar" a/cold z/or= iginal b/hot-alias) +printf '/b/hot-alias\n' > "$tmpdir/hotlist.tar-hardlink" +"$MKFS" -d9 --tar=3Df --hot-file-list=3D"$tmpdir/hotlist.tar-hardlink" \ + "$tmpdir/img.tar-hardlink.erofs" "$tmpdir/root-hardlink.tar" >"$tmpdir/mk= fs.tar-hardlink.log" 2>&1 +assert_tar_hot_hardlink_alias_precedes_cold_file "$tmpdir/img.tar-hardlink= .erofs" +"$MKFS" -d9 -zzstd,level=3D9 --workers=3D1 --tar=3Df \ + --hot-file-list=3D"$tmpdir/hotlist.tar-hardlink" \ + "$tmpdir/img.tar-hardlink.zstd.erofs" "$tmpdir/root-hardlink.tar" >"$tmpd= ir/mkfs.tar-hardlink.zstd.log" 2>&1 +assert_tar_hot_hardlink_alias_precedes_cold_file "$tmpdir/img.tar-hardlink= .zstd.erofs" --=20 2.43.7