[PATCH] tools/build: make in-target rule robust against too long argument error

changqing.li@windriver.com posted 1 patch 6 months, 2 weeks ago
tools/build/Makefile.build | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
[PATCH] tools/build: make in-target rule robust against too long argument error
Posted by changqing.li@windriver.com 6 months, 2 weeks ago
From: Changqing Li <changqing.li@windriver.com>

The command length of in-target scales with the depth of the directory
times the number of objects in the Makefile. When there are many
objects, and O=[absolute_path] is set, and the absolute_path is
relatively long. It is possile that this line "$(call
if_changed,$(host)ld_multi)" will report error:
"make[4]: /bin/sh: Argument list too long"

For example, build perf tools with O=/long/output/path

Like built-in.a and *.mod rules in scripts/Makefile.build, add
$(objpredix)/ by the shell command instead of by Make's builtin
function.

Signed-off-by: Changqing Li <changqing.li@windriver.com>
---
 tools/build/Makefile.build | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/tools/build/Makefile.build b/tools/build/Makefile.build
index 3584ff308607..39066a3ef2fc 100644
--- a/tools/build/Makefile.build
+++ b/tools/build/Makefile.build
@@ -70,11 +70,13 @@ quiet_cmd_gen = GEN     $@
 # If there's nothing to link, create empty $@ object.
 quiet_cmd_ld_multi = LD      $@
       cmd_ld_multi = $(if $(strip $(obj-y)),\
-                     $(LD) -r -o $@  $(filter $(obj-y),$^),rm -f $@; $(AR) rcs $@)
+                     printf "$(objprefix)%s " $(patsubst $(objprefix)%,%,$(filter $(obj-y),$^)) | \
+                     xargs $(LD) -r -o $@,rm -f $@; $(AR) rcs $@)
 
 quiet_cmd_host_ld_multi = HOSTLD  $@
       cmd_host_ld_multi = $(if $(strip $(obj-y)),\
-                          $(HOSTLD) -r -o $@  $(filter $(obj-y),$^),rm -f $@; $(HOSTAR) rcs $@)
+                          printf "$(objprefix)%s " $(patsubst $(objprefix)%,%,$(filter $(obj-y),$^)) | \
+                          xargs $(HOSTLD) -r -o $@,rm -f $@; $(HOSTAR) rcs $@)
 
 ifneq ($(filter $(obj),$(hostprogs)),)
   host = host_
-- 
2.34.1
Re: [PATCH] tools/build: make in-target rule robust against too long argument error
Posted by Ian Rogers 5 months, 3 weeks ago
On Mon, Jul 28, 2025 at 2:31 AM <changqing.li@windriver.com> wrote:
>
> From: Changqing Li <changqing.li@windriver.com>
>
> The command length of in-target scales with the depth of the directory
> times the number of objects in the Makefile. When there are many
> objects, and O=[absolute_path] is set, and the absolute_path is
> relatively long. It is possile that this line "$(call
> if_changed,$(host)ld_multi)" will report error:
> "make[4]: /bin/sh: Argument list too long"
>
> For example, build perf tools with O=/long/output/path
>
> Like built-in.a and *.mod rules in scripts/Makefile.build, add
> $(objpredix)/ by the shell command instead of by Make's builtin
> function.
>
> Signed-off-by: Changqing Li <changqing.li@windriver.com>

Thanks Changqing, the change makes sense to me. The printf is pushing
the values into xargs rather than using $^ with ld. I've tried
reproducing the error to test your fix by creating long directory
names in /tmp and then passing them to O=. I've not been able to do
this. Could you send a reproduction for me to test?

Thanks,
Ian

> ---
>  tools/build/Makefile.build | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/tools/build/Makefile.build b/tools/build/Makefile.build
> index 3584ff308607..39066a3ef2fc 100644
> --- a/tools/build/Makefile.build
> +++ b/tools/build/Makefile.build
> @@ -70,11 +70,13 @@ quiet_cmd_gen = GEN     $@
>  # If there's nothing to link, create empty $@ object.
>  quiet_cmd_ld_multi = LD      $@
>        cmd_ld_multi = $(if $(strip $(obj-y)),\
> -                     $(LD) -r -o $@  $(filter $(obj-y),$^),rm -f $@; $(AR) rcs $@)
> +                     printf "$(objprefix)%s " $(patsubst $(objprefix)%,%,$(filter $(obj-y),$^)) | \
> +                     xargs $(LD) -r -o $@,rm -f $@; $(AR) rcs $@)
>
>  quiet_cmd_host_ld_multi = HOSTLD  $@
>        cmd_host_ld_multi = $(if $(strip $(obj-y)),\
> -                          $(HOSTLD) -r -o $@  $(filter $(obj-y),$^),rm -f $@; $(HOSTAR) rcs $@)
> +                          printf "$(objprefix)%s " $(patsubst $(objprefix)%,%,$(filter $(obj-y),$^)) | \
> +                          xargs $(HOSTLD) -r -o $@,rm -f $@; $(HOSTAR) rcs $@)
>
>  ifneq ($(filter $(obj),$(hostprogs)),)
>    host = host_
> --
> 2.34.1
>
Re: [PATCH] tools/build: make in-target rule robust against too long argument error
Posted by Changqing Li 5 months, 3 weeks ago
On 8/19/25 04:38, Ian Rogers wrote:
> CAUTION: This email comes from a non Wind River email account!
> Do not click links or open attachments unless you recognize the sender and know the content is safe.
>
> On Mon, Jul 28, 2025 at 2:31 AM <changqing.li@windriver.com> wrote:
>> From: Changqing Li <changqing.li@windriver.com>
>>
>> The command length of in-target scales with the depth of the directory
>> times the number of objects in the Makefile. When there are many
>> objects, and O=[absolute_path] is set, and the absolute_path is
>> relatively long. It is possile that this line "$(call
>> if_changed,$(host)ld_multi)" will report error:
>> "make[4]: /bin/sh: Argument list too long"
>>
>> For example, build perf tools with O=/long/output/path
>>
>> Like built-in.a and *.mod rules in scripts/Makefile.build, add
>> $(objpredix)/ by the shell command instead of by Make's builtin
>> function.
>>
>> Signed-off-by: Changqing Li <changqing.li@windriver.com>
> Thanks Changqing, the change makes sense to me. The printf is pushing
> the values into xargs rather than using $^ with ld. I've tried
> reproducing the error to test your fix by creating long directory
> names in /tmp and then passing them to O=. I've not been able to do
> this. Could you send a reproduction for me to test?
>
> Thanks,
> Ian

Hi, Ian

Thanks. Here is my steps:

1. git clone https://github.com/torvalds/linux.git

2. cd linux/tools/perf/

3. mkdir 
/tmp/01234567890123456789/0123456789012345678901234567890123456789/0123456789012345678901234567890123456789/012345678901234567890123456789/01234567890123456789/012345678901234567890123456789/01234567890123456789012345678/9012345678901234567890123456789/012345678901234567890123456789/012345678901234567890123456789/01234567890123456789012345678901234567890123456789012345678901234567890123/perf-1.0

4.  make NO_LIBTRACEEVENT=1 
O=/tmp/01234567890123456789/0123456789012345678901234567890123456789/0123456789012345678901234567890123456789/012345678901234567890123456789/01234567890123456789/012345678901234567890123456789/01234567890123456789012345678/9012345678901234567890123456789/012345678901234567890123456789/012345678901234567890123456789/01234567890123456789012345678901234567890123456789012345678901234567890123/perf-1.0 


5. Above step failed with error:

make[4]: /bin/sh: Argument list too long

make[4]: *** [/buildarea2/cli10/tmp/linux/tools/build/Makefile.build 
<https://urldefense.com/v3/__http://Makefile.build__;!!AjveYdw8EvQ!c2n7zPX4Vrhwd7a9UxsjCXBGO5V8Llmon1egvupz4kHL4869ilesQgRgQZO8nj8AWwQJUlY9DOoWPD688quA73jNugR6IZRxy09Eig$>:148: 
/tmp/01234567890123456789/0123456789012345678901234567890123456789/0123456789012345678901234567890123456789/012345678901234567890123456789/01234567890123456789/012345678901234567890123456789/01234567890123456789012345678/9012345678901234567890123456789/012345678901234567890123456789/012345678901234567890123456789/01234567890123456789012345678901234567890123456789012345678901234567890123/perf-1.0/util/perf-util-in.o] 
Error 127

make[3]: *** [/buildarea2/cli10/tmp/linux/tools/build/Makefile.build 
<https://urldefense.com/v3/__http://Makefile.build__;!!AjveYdw8EvQ!c2n7zPX4Vrhwd7a9UxsjCXBGO5V8Llmon1egvupz4kHL4869ilesQgRgQZO8nj8AWwQJUlY9DOoWPD688quA73jNugR6IZRxy09Eig$>:142: 
util] Error 2

make[2]: *** [Makefile.perf:797: 
/tmp/01234567890123456789/0123456789012345678901234567890123456789/0123456789012345678901234567890123456789/012345678901234567890123456789/01234567890123456789/012345678901234567890123456789/01234567890123456789012345678/9012345678901234567890123456789/012345678901234567890123456789/012345678901234567890123456789/01234567890123456789012345678901234567890123456789012345678901234567890123/perf-1.0/perf-util-in.o] 
Error 2


Regards

Changqing
>> ---
>>   tools/build/Makefile.build | 6 ++++--
>>   1 file changed, 4 insertions(+), 2 deletions(-)
>>
>> diff --git a/tools/build/Makefile.build b/tools/build/Makefile.build
>> index 3584ff308607..39066a3ef2fc 100644
>> --- a/tools/build/Makefile.build
>> +++ b/tools/build/Makefile.build
>> @@ -70,11 +70,13 @@ quiet_cmd_gen = GEN     $@
>>   # If there's nothing to link, create empty $@ object.
>>   quiet_cmd_ld_multi = LD      $@
>>         cmd_ld_multi = $(if $(strip $(obj-y)),\
>> -                     $(LD) -r -o $@  $(filter $(obj-y),$^),rm -f $@; $(AR) rcs $@)
>> +                     printf "$(objprefix)%s " $(patsubst $(objprefix)%,%,$(filter $(obj-y),$^)) | \
>> +                     xargs $(LD) -r -o $@,rm -f $@; $(AR) rcs $@)
>>
>>   quiet_cmd_host_ld_multi = HOSTLD  $@
>>         cmd_host_ld_multi = $(if $(strip $(obj-y)),\
>> -                          $(HOSTLD) -r -o $@  $(filter $(obj-y),$^),rm -f $@; $(HOSTAR) rcs $@)
>> +                          printf "$(objprefix)%s " $(patsubst $(objprefix)%,%,$(filter $(obj-y),$^)) | \
>> +                          xargs $(HOSTLD) -r -o $@,rm -f $@; $(HOSTAR) rcs $@)
>>
>>   ifneq ($(filter $(obj),$(hostprogs)),)
>>     host = host_
>> --
>> 2.34.1
>>
Re: [PATCH] tools/build: make in-target rule robust against too long argument error
Posted by Changqing Li 3 months, 1 week ago
Hi, Ian

Kindly ping.  Could you reproduce it?  Any comments?

Regards

Changqing

On 8/19/25 11:01, Changqing Li wrote:
>
> On 8/19/25 04:38, Ian Rogers wrote:
>> CAUTION: This email comes from a non Wind River email account!
>> Do not click links or open attachments unless you recognize the 
>> sender and know the content is safe.
>>
>> On Mon, Jul 28, 2025 at 2:31 AM <changqing.li@windriver.com> wrote:
>>> From: Changqing Li <changqing.li@windriver.com>
>>>
>>> The command length of in-target scales with the depth of the directory
>>> times the number of objects in the Makefile. When there are many
>>> objects, and O=[absolute_path] is set, and the absolute_path is
>>> relatively long. It is possile that this line "$(call
>>> if_changed,$(host)ld_multi)" will report error:
>>> "make[4]: /bin/sh: Argument list too long"
>>>
>>> For example, build perf tools with O=/long/output/path
>>>
>>> Like built-in.a and *.mod rules in scripts/Makefile.build, add
>>> $(objpredix)/ by the shell command instead of by Make's builtin
>>> function.
>>>
>>> Signed-off-by: Changqing Li <changqing.li@windriver.com>
>> Thanks Changqing, the change makes sense to me. The printf is pushing
>> the values into xargs rather than using $^ with ld. I've tried
>> reproducing the error to test your fix by creating long directory
>> names in /tmp and then passing them to O=. I've not been able to do
>> this. Could you send a reproduction for me to test?
>>
>> Thanks,
>> Ian
>
> Hi, Ian
>
> Thanks. Here is my steps:
>
> 1. git clone https://github.com/torvalds/linux.git
>
> 2. cd linux/tools/perf/
>
> 3. mkdir 
> /tmp/01234567890123456789/0123456789012345678901234567890123456789/0123456789012345678901234567890123456789/012345678901234567890123456789/01234567890123456789/012345678901234567890123456789/01234567890123456789012345678/9012345678901234567890123456789/012345678901234567890123456789/012345678901234567890123456789/01234567890123456789012345678901234567890123456789012345678901234567890123/perf-1.0
>
> 4.  make NO_LIBTRACEEVENT=1 
> O=/tmp/01234567890123456789/0123456789012345678901234567890123456789/0123456789012345678901234567890123456789/012345678901234567890123456789/01234567890123456789/012345678901234567890123456789/01234567890123456789012345678/9012345678901234567890123456789/012345678901234567890123456789/012345678901234567890123456789/01234567890123456789012345678901234567890123456789012345678901234567890123/perf-1.0 
>
>
> 5. Above step failed with error:
>
> make[4]: /bin/sh: Argument list too long
>
> make[4]: *** [/buildarea2/cli10/tmp/linux/tools/build/Makefile.build 
> <https://urldefense.com/v3/__http://Makefile.build__;!!AjveYdw8EvQ!c2n7zPX4Vrhwd7a9UxsjCXBGO5V8Llmon1egvupz4kHL4869ilesQgRgQZO8nj8AWwQJUlY9DOoWPD688quA73jNugR6IZRxy09Eig$>:148: 
> /tmp/01234567890123456789/0123456789012345678901234567890123456789/0123456789012345678901234567890123456789/012345678901234567890123456789/01234567890123456789/012345678901234567890123456789/01234567890123456789012345678/9012345678901234567890123456789/012345678901234567890123456789/012345678901234567890123456789/01234567890123456789012345678901234567890123456789012345678901234567890123/perf-1.0/util/perf-util-in.o] 
> Error 127
>
> make[3]: *** [/buildarea2/cli10/tmp/linux/tools/build/Makefile.build 
> <https://urldefense.com/v3/__http://Makefile.build__;!!AjveYdw8EvQ!c2n7zPX4Vrhwd7a9UxsjCXBGO5V8Llmon1egvupz4kHL4869ilesQgRgQZO8nj8AWwQJUlY9DOoWPD688quA73jNugR6IZRxy09Eig$>:142: 
> util] Error 2
>
> make[2]: *** [Makefile.perf:797: 
> /tmp/01234567890123456789/0123456789012345678901234567890123456789/0123456789012345678901234567890123456789/012345678901234567890123456789/01234567890123456789/012345678901234567890123456789/01234567890123456789012345678/9012345678901234567890123456789/012345678901234567890123456789/012345678901234567890123456789/01234567890123456789012345678901234567890123456789012345678901234567890123/perf-1.0/perf-util-in.o] 
> Error 2
>
>
> Regards
>
> Changqing
>>> ---
>>>   tools/build/Makefile.build | 6 ++++--
>>>   1 file changed, 4 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/tools/build/Makefile.build b/tools/build/Makefile.build
>>> index 3584ff308607..39066a3ef2fc 100644
>>> --- a/tools/build/Makefile.build
>>> +++ b/tools/build/Makefile.build
>>> @@ -70,11 +70,13 @@ quiet_cmd_gen = GEN     $@
>>>   # If there's nothing to link, create empty $@ object.
>>>   quiet_cmd_ld_multi = LD      $@
>>>         cmd_ld_multi = $(if $(strip $(obj-y)),\
>>> -                     $(LD) -r -o $@  $(filter $(obj-y),$^),rm -f 
>>> $@; $(AR) rcs $@)
>>> +                     printf "$(objprefix)%s " $(patsubst 
>>> $(objprefix)%,%,$(filter $(obj-y),$^)) | \
>>> +                     xargs $(LD) -r -o $@,rm -f $@; $(AR) rcs $@)
>>>
>>>   quiet_cmd_host_ld_multi = HOSTLD  $@
>>>         cmd_host_ld_multi = $(if $(strip $(obj-y)),\
>>> -                          $(HOSTLD) -r -o $@  $(filter 
>>> $(obj-y),$^),rm -f $@; $(HOSTAR) rcs $@)
>>> +                          printf "$(objprefix)%s " $(patsubst 
>>> $(objprefix)%,%,$(filter $(obj-y),$^)) | \
>>> +                          xargs $(HOSTLD) -r -o $@,rm -f $@; 
>>> $(HOSTAR) rcs $@)
>>>
>>>   ifneq ($(filter $(obj),$(hostprogs)),)
>>>     host = host_
>>> -- 
>>> 2.34.1
>>>