scripts: add origin commit identification based on specific patterns

[PATCH] scripts: add origin commit identification based on specific patterns

Posted by Zhiyu Zhang 7 months ago

This patch adds the functionability to smartly identify origin commit
of the translation by matching the following patterns in commit log:
1) update to commit HASH
2) Update the translation through commit HASH
If no such pattern is found, script will obey the original workflow.

Signed-off-by: Zhiyu Zhang <zhiyuzhang999@gmail.com>
---
 scripts/checktransupdate.py | 38 ++++++++++++++++++++++++++++++++++++-
 1 file changed, 37 insertions(+), 1 deletion(-)

diff --git a/scripts/checktransupdate.py b/scripts/checktransupdate.py
index 578c3fecfdfd..e39529e46c3d 100755
--- a/scripts/checktransupdate.py
+++ b/scripts/checktransupdate.py
@@ -24,6 +24,7 @@ commit 42fb9cfd5b18 ("Documentation: dev-tools: Add link to RV docs")
 """
 
 import os
+import re
 import time
 import logging
 from argparse import ArgumentParser, ArgumentTypeError, BooleanOptionalAction
@@ -69,6 +70,38 @@ def get_origin_from_trans(origin_path, t_from_head):
     return o_from_t
 
 
+def get_origin_from_trans_smartly(origin_path, t_from_head):
+    """Get the latest origin commit from the formatted translation commit:
+    (1) update to commit HASH (TITLE)
+    (2) Update the translation through commit HASH (TITLE)
+    """
+    # catch flag for 12-bit commit hash
+    HASH = r'([0-9a-f]{12})'
+    # pattern 1: contains "update to commit HASH"
+    pat_update_to = re.compile(rf'update to commit {HASH}')
+    # pattern 2: contains "Update the translation through commit HASH"
+    pat_update_translation = re.compile(rf'Update the translation through commit {HASH}')
+
+    origin_commit_hash = None
+    for line in t_from_head["message"]:
+        # check if the line matches the first pattern
+        match = pat_update_to.search(line)
+        if match:
+            origin_commit_hash = match.group(1)
+            break
+        # check if the line matches the second pattern
+        match = pat_update_translation.search(line)
+        if match:
+            origin_commit_hash = match.group(1)
+            break
+    if origin_commit_hash is None:
+        return None
+    o_from_t = get_latest_commit_from(origin_path, origin_commit_hash)
+    if o_from_t is not None:
+        logging.debug("tracked origin commit id: %s", o_from_t["hash"])
+    return o_from_t
+
+
 def get_commits_count_between(opath, commit1, commit2):
     """Get the commits count between two commits for the specified file"""
     command = f"git log --pretty=format:%H {commit1}...{commit2} -- {opath}"
@@ -108,7 +141,10 @@ def check_per_file(file_path):
         logging.error("Cannot find the latest commit for %s", file_path)
         return
 
-    o_from_t = get_origin_from_trans(opath, t_from_head)
+    o_from_t = get_origin_from_trans_smartly(opath, t_from_head)
+    # notice, o_from_t from get_*_smartly() is always more accurate than from get_*()
+    if o_from_t is None:
+        o_from_t = get_origin_from_trans(opath, t_from_head)
 
     if o_from_t is None:
         logging.error("Error: Cannot find the latest origin commit for %s", file_path)
-- 
2.34.1

Re: [PATCH] scripts: add origin commit identification based on specific patterns

Posted by Dongliang Mu 6 months, 3 weeks ago

On Mon, Jul 14, 2025 at 12:38 AM Zhiyu Zhang <zhiyuzhang999@gmail.com> wrote:
>
> This patch adds the functionability to smartly identify origin commit
> of the translation by matching the following patterns in commit log:
> 1) update to commit HASH
> 2) Update the translation through commit HASH
> If no such pattern is found, script will obey the original workflow.
>
> Signed-off-by: Zhiyu Zhang <zhiyuzhang999@gmail.com>

Reviewed-by: Dongliang Mu <dzm91@hust.edu.cn>


> ---
>  scripts/checktransupdate.py | 38 ++++++++++++++++++++++++++++++++++++-
>  1 file changed, 37 insertions(+), 1 deletion(-)
>
> diff --git a/scripts/checktransupdate.py b/scripts/checktransupdate.py
> index 578c3fecfdfd..e39529e46c3d 100755
> --- a/scripts/checktransupdate.py
> +++ b/scripts/checktransupdate.py
> @@ -24,6 +24,7 @@ commit 42fb9cfd5b18 ("Documentation: dev-tools: Add link to RV docs")
>  """
>
>  import os
> +import re
>  import time
>  import logging
>  from argparse import ArgumentParser, ArgumentTypeError, BooleanOptionalAction
> @@ -69,6 +70,38 @@ def get_origin_from_trans(origin_path, t_from_head):
>      return o_from_t
>
>
> +def get_origin_from_trans_smartly(origin_path, t_from_head):
> +    """Get the latest origin commit from the formatted translation commit:
> +    (1) update to commit HASH (TITLE)
> +    (2) Update the translation through commit HASH (TITLE)
> +    """
> +    # catch flag for 12-bit commit hash
> +    HASH = r'([0-9a-f]{12})'
> +    # pattern 1: contains "update to commit HASH"
> +    pat_update_to = re.compile(rf'update to commit {HASH}')
> +    # pattern 2: contains "Update the translation through commit HASH"
> +    pat_update_translation = re.compile(rf'Update the translation through commit {HASH}')
> +
> +    origin_commit_hash = None
> +    for line in t_from_head["message"]:
> +        # check if the line matches the first pattern
> +        match = pat_update_to.search(line)
> +        if match:
> +            origin_commit_hash = match.group(1)
> +            break
> +        # check if the line matches the second pattern
> +        match = pat_update_translation.search(line)
> +        if match:
> +            origin_commit_hash = match.group(1)
> +            break
> +    if origin_commit_hash is None:
> +        return None
> +    o_from_t = get_latest_commit_from(origin_path, origin_commit_hash)
> +    if o_from_t is not None:
> +        logging.debug("tracked origin commit id: %s", o_from_t["hash"])
> +    return o_from_t
> +
> +
>  def get_commits_count_between(opath, commit1, commit2):
>      """Get the commits count between two commits for the specified file"""
>      command = f"git log --pretty=format:%H {commit1}...{commit2} -- {opath}"
> @@ -108,7 +141,10 @@ def check_per_file(file_path):
>          logging.error("Cannot find the latest commit for %s", file_path)
>          return
>
> -    o_from_t = get_origin_from_trans(opath, t_from_head)
> +    o_from_t = get_origin_from_trans_smartly(opath, t_from_head)
> +    # notice, o_from_t from get_*_smartly() is always more accurate than from get_*()
> +    if o_from_t is None:
> +        o_from_t = get_origin_from_trans(opath, t_from_head)
>
>      if o_from_t is None:
>          logging.error("Error: Cannot find the latest origin commit for %s", file_path)
> --
> 2.34.1
>
>

Re: [PATCH] scripts: add origin commit identification based on specific patterns

Posted by Jonathan Corbet 6 months, 3 weeks ago

Zhiyu Zhang <zhiyuzhang999@gmail.com> writes:

> This patch adds the functionability to smartly identify origin commit
> of the translation by matching the following patterns in commit log:
> 1) update to commit HASH
> 2) Update the translation through commit HASH
> If no such pattern is found, script will obey the original workflow.
>
> Signed-off-by: Zhiyu Zhang <zhiyuzhang999@gmail.com>
> ---
>  scripts/checktransupdate.py | 38 ++++++++++++++++++++++++++++++++++++-
>  1 file changed, 37 insertions(+), 1 deletion(-)

So I don't have any objection to this, but wouldn't mind hearing from
folks who actually use this script - has anybody else tested it out?

Thanks,

jon

Re: [PATCH] scripts: add origin commit identification based on specific patterns

Posted by Dongliang Mu 6 months, 3 weeks ago

On Fri, Jul 18, 2025 at 4:09 AM Jonathan Corbet <corbet@lwn.net> wrote:
>
> Zhiyu Zhang <zhiyuzhang999@gmail.com> writes:
>
> > This patch adds the functionability to smartly identify origin commit
> > of the translation by matching the following patterns in commit log:
> > 1) update to commit HASH
> > 2) Update the translation through commit HASH
> > If no such pattern is found, script will obey the original workflow.
> >
> > Signed-off-by: Zhiyu Zhang <zhiyuzhang999@gmail.com>
> > ---
> >  scripts/checktransupdate.py | 38 ++++++++++++++++++++++++++++++++++++-
> >  1 file changed, 37 insertions(+), 1 deletion(-)
>
> So I don't have any objection to this, but wouldn't mind hearing from
> folks who actually use this script - has anybody else tested it out?

I’ve tested this script, and the new origin commit tracking
functionality is working effectively—it can reduce the number of
commits requiring updates.

Dongliang Mu

>
> Thanks,
>
> jon
>

Re: [PATCH] scripts: add origin commit identification based on specific patterns

Posted by Jonathan Corbet 6 months, 2 weeks ago

Dongliang Mu <mudongliangabcd@gmail.com> writes:

> On Fri, Jul 18, 2025 at 4:09 AM Jonathan Corbet <corbet@lwn.net> wrote:
>>
>> Zhiyu Zhang <zhiyuzhang999@gmail.com> writes:
>>
>> > This patch adds the functionability to smartly identify origin commit
>> > of the translation by matching the following patterns in commit log:
>> > 1) update to commit HASH
>> > 2) Update the translation through commit HASH
>> > If no such pattern is found, script will obey the original workflow.
>> >
>> > Signed-off-by: Zhiyu Zhang <zhiyuzhang999@gmail.com>
>> > ---
>> >  scripts/checktransupdate.py | 38 ++++++++++++++++++++++++++++++++++++-
>> >  1 file changed, 37 insertions(+), 1 deletion(-)
>>
>> So I don't have any objection to this, but wouldn't mind hearing from
>> folks who actually use this script - has anybody else tested it out?
>
> I’ve tested this script, and the new origin commit tracking
> functionality is working effectively—it can reduce the number of
> commits requiring updates.

Great, thanks, I've applied it.

jon