x86/vmlinux: Fix linker fill bytes for ld.lld

[PATCH] x86/vmlinux: Fix linker fill bytes for ld.lld

Posted by Song Liu 2 years, 3 months ago

With ":text =0xcccc", ld.lld fills unused text area with 0xcccc0000.
Example objdump -D output:

ffffffff82b04203:       00 00                   add    %al,(%rax)
ffffffff82b04205:       cc                      int3
ffffffff82b04206:       cc                      int3
ffffffff82b04207:       00 00                   add    %al,(%rax)
ffffffff82b04209:       cc                      int3
ffffffff82b0420a:       cc                      int3

Replace it with ":text =0xcccccccc", so we get the following instead:

ffffffff82b04203:       cc                      int3
ffffffff82b04204:       cc                      int3
ffffffff82b04205:       cc                      int3
ffffffff82b04206:       cc                      int3
ffffffff82b04207:       cc                      int3
ffffffff82b04208:       cc                      int3

gcc/ld doesn't seem to have the same issue. The generated code stays the
same for gcc/ld.

Cc: Kees Cook <keescook@chromium.org>
Cc: x86@kernel.org
Signed-off-by: Song Liu <song@kernel.org>
---
 arch/x86/kernel/vmlinux.lds.S | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S
index 83d41c2601d7..41d56fb9bf92 100644
--- a/arch/x86/kernel/vmlinux.lds.S
+++ b/arch/x86/kernel/vmlinux.lds.S
@@ -156,7 +156,7 @@ SECTIONS
 		ALIGN_ENTRY_TEXT_END
 		*(.gnu.warning)
 
-	} :text =0xcccc
+	} :text =0xcccccccc
 
 	/* End of text section, which should occupy whole number of pages */
 	_etext = .;
-- 
2.34.1

Re: [PATCH] x86/vmlinux: Fix linker fill bytes for ld.lld

Posted by Peter Zijlstra 2 years, 3 months ago

On Wed, Sep 06, 2023 at 10:52:15AM -0700, Song Liu wrote:
> With ":text =0xcccc", ld.lld fills unused text area with 0xcccc0000.
> Example objdump -D output:
> 
> ffffffff82b04203:       00 00                   add    %al,(%rax)
> ffffffff82b04205:       cc                      int3
> ffffffff82b04206:       cc                      int3
> ffffffff82b04207:       00 00                   add    %al,(%rax)
> ffffffff82b04209:       cc                      int3
> ffffffff82b0420a:       cc                      int3
> 
> Replace it with ":text =0xcccccccc", so we get the following instead:
> 
> ffffffff82b04203:       cc                      int3
> ffffffff82b04204:       cc                      int3
> ffffffff82b04205:       cc                      int3
> ffffffff82b04206:       cc                      int3
> ffffffff82b04207:       cc                      int3
> ffffffff82b04208:       cc                      int3
> 
> gcc/ld doesn't seem to have the same issue. The generated code stays the
> same for gcc/ld.
> 
> Cc: Kees Cook <keescook@chromium.org>
> Cc: x86@kernel.org
> Signed-off-by: Song Liu <song@kernel.org>

Please provide a Fixes tag, I'm thinking this (otherwise trivial commit)
wants to be backported for sanity.

Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>

> ---
>  arch/x86/kernel/vmlinux.lds.S | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S
> index 83d41c2601d7..41d56fb9bf92 100644
> --- a/arch/x86/kernel/vmlinux.lds.S
> +++ b/arch/x86/kernel/vmlinux.lds.S
> @@ -156,7 +156,7 @@ SECTIONS
>  		ALIGN_ENTRY_TEXT_END
>  		*(.gnu.warning)
>  
> -	} :text =0xcccc
> +	} :text =0xcccccccc
>  
>  	/* End of text section, which should occupy whole number of pages */
>  	_etext = .;
> -- 
> 2.34.1
>

Re: [PATCH] x86/vmlinux: Fix linker fill bytes for ld.lld

Posted by Song Liu 2 years, 3 months ago

On Wed, Sep 6, 2023 at 12:58 PM Peter Zijlstra <peterz@infradead.org> wrote:
>
> On Wed, Sep 06, 2023 at 10:52:15AM -0700, Song Liu wrote:
> > With ":text =0xcccc", ld.lld fills unused text area with 0xcccc0000.
> > Example objdump -D output:
> >
> > ffffffff82b04203:       00 00                   add    %al,(%rax)
> > ffffffff82b04205:       cc                      int3
> > ffffffff82b04206:       cc                      int3
> > ffffffff82b04207:       00 00                   add    %al,(%rax)
> > ffffffff82b04209:       cc                      int3
> > ffffffff82b0420a:       cc                      int3
> >
> > Replace it with ":text =0xcccccccc", so we get the following instead:
> >
> > ffffffff82b04203:       cc                      int3
> > ffffffff82b04204:       cc                      int3
> > ffffffff82b04205:       cc                      int3
> > ffffffff82b04206:       cc                      int3
> > ffffffff82b04207:       cc                      int3
> > ffffffff82b04208:       cc                      int3
> >
> > gcc/ld doesn't seem to have the same issue. The generated code stays the
> > same for gcc/ld.
> >
> > Cc: Kees Cook <keescook@chromium.org>
> > Cc: x86@kernel.org
> > Signed-off-by: Song Liu <song@kernel.org>
>
> Please provide a Fixes tag, I'm thinking this (otherwise trivial commit)
> wants to be backported for sanity.

I guess we need:

Fixes: 7705dc855797 ("x86/vmlinux: Use INT3 instead of NOP for linker
fill bytes")

Thanks,
Song

>
> Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>

Re: [PATCH] x86/vmlinux: Fix linker fill bytes for ld.lld

Posted by Kees Cook 2 years, 3 months ago

On Wed, Sep 06, 2023 at 10:52:15AM -0700, Song Liu wrote:
> With ":text =0xcccc", ld.lld fills unused text area with 0xcccc0000.
> Example objdump -D output:
> 
> ffffffff82b04203:       00 00                   add    %al,(%rax)
> ffffffff82b04205:       cc                      int3
> ffffffff82b04206:       cc                      int3
> ffffffff82b04207:       00 00                   add    %al,(%rax)
> ffffffff82b04209:       cc                      int3
> ffffffff82b0420a:       cc                      int3
> 
> Replace it with ":text =0xcccccccc", so we get the following instead:
> 
> ffffffff82b04203:       cc                      int3
> ffffffff82b04204:       cc                      int3
> ffffffff82b04205:       cc                      int3
> ffffffff82b04206:       cc                      int3
> ffffffff82b04207:       cc                      int3
> ffffffff82b04208:       cc                      int3
> 
> gcc/ld doesn't seem to have the same issue. The generated code stays the
> same for gcc/ld.
> 
> Cc: Kees Cook <keescook@chromium.org>
> Cc: x86@kernel.org
> Signed-off-by: Song Liu <song@kernel.org>

Ah! Thanks for the catch... I wonder if ld.lld should be fixed too? My
understanding was that ":text =...." was defined as being explicitly
u16?

Reviewed-by: Kees Cook <keescook@chromium.org>

-- 
Kees Cook

Re: [PATCH] x86/vmlinux: Fix linker fill bytes for ld.lld

Posted by Song Liu 2 years, 3 months ago


> On Sep 6, 2023, at 11:06 AM, Kees Cook <keescook@chromium.org> wrote:
> 
> On Wed, Sep 06, 2023 at 10:52:15AM -0700, Song Liu wrote:
>> With ":text =0xcccc", ld.lld fills unused text area with 0xcccc0000.
>> Example objdump -D output:
>> 
>> ffffffff82b04203:       00 00                   add    %al,(%rax)
>> ffffffff82b04205:       cc                      int3
>> ffffffff82b04206:       cc                      int3
>> ffffffff82b04207:       00 00                   add    %al,(%rax)
>> ffffffff82b04209:       cc                      int3
>> ffffffff82b0420a:       cc                      int3
>> 
>> Replace it with ":text =0xcccccccc", so we get the following instead:
>> 
>> ffffffff82b04203:       cc                      int3
>> ffffffff82b04204:       cc                      int3
>> ffffffff82b04205:       cc                      int3
>> ffffffff82b04206:       cc                      int3
>> ffffffff82b04207:       cc                      int3
>> ffffffff82b04208:       cc                      int3
>> 
>> gcc/ld doesn't seem to have the same issue. The generated code stays the
>> same for gcc/ld.
>> 
>> Cc: Kees Cook <keescook@chromium.org>
>> Cc: x86@kernel.org
>> Signed-off-by: Song Liu <song@kernel.org>
> 
> Ah! Thanks for the catch... I wonder if ld.lld should be fixed too? My
> understanding was that ":text =...." was defined as being explicitly
> u16?

Per my experiment, gcc/ld gives same output for :text =0xcc, :text =0xcccc,
and :text =0xcccccccc; while ld.lld handles :text = as u32, so :text =0xcc
with ld.lld gives:

ffffffff82b042a1:       00 cc                   add    %cl,%ah
ffffffff82b042a3:       00 00                   add    %al,(%rax)
ffffffff82b042a5:       00 cc                   add    %cl,%ah
ffffffff82b042a7:       00 00                   add    %al,(%rax)
ffffffff82b042a9:       00 cc                   add    %cl,%ah
ffffffff82b042ab:       00 00                   add    %al,(%rax)

I am not sure what the right behavior is per specification. 

Thanks,
Song

Re: [PATCH] x86/vmlinux: Fix linker fill bytes for ld.lld

Posted by Fangrui Song 2 years, 3 months ago

On Wed, Sep 6, 2023 at 11:16 AM Song Liu <songliubraving@meta.com> wrote:
>
>
>
> > On Sep 6, 2023, at 11:06 AM, Kees Cook <keescook@chromium.org> wrote:
> >
> > On Wed, Sep 06, 2023 at 10:52:15AM -0700, Song Liu wrote:
> >> With ":text =0xcccc", ld.lld fills unused text area with 0xcccc0000.
> >> Example objdump -D output:
> >>
> >> ffffffff82b04203:       00 00                   add    %al,(%rax)
> >> ffffffff82b04205:       cc                      int3
> >> ffffffff82b04206:       cc                      int3
> >> ffffffff82b04207:       00 00                   add    %al,(%rax)
> >> ffffffff82b04209:       cc                      int3
> >> ffffffff82b0420a:       cc                      int3
> >>
> >> Replace it with ":text =0xcccccccc", so we get the following instead:
> >>
> >> ffffffff82b04203:       cc                      int3
> >> ffffffff82b04204:       cc                      int3
> >> ffffffff82b04205:       cc                      int3
> >> ffffffff82b04206:       cc                      int3
> >> ffffffff82b04207:       cc                      int3
> >> ffffffff82b04208:       cc                      int3
> >>
> >> gcc/ld doesn't seem to have the same issue. The generated code stays the
> >> same for gcc/ld.
> >>
> >> Cc: Kees Cook <keescook@chromium.org>
> >> Cc: x86@kernel.org
> >> Signed-off-by: Song Liu <song@kernel.org>
> >
> > Ah! Thanks for the catch... I wonder if ld.lld should be fixed too? My
> > understanding was that ":text =...." was defined as being explicitly
> > u16?
>
> Per my experiment, gcc/ld gives same output for :text =0xcc, :text =0xcccc,
> and :text =0xcccccccc; while ld.lld handles :text = as u32, so :text =0xcc
> with ld.lld gives:
>
> ffffffff82b042a1:       00 cc                   add    %cl,%ah
> ffffffff82b042a3:       00 00                   add    %al,(%rax)
> ffffffff82b042a5:       00 cc                   add    %cl,%ah
> ffffffff82b042a7:       00 00                   add    %al,(%rax)
> ffffffff82b042a9:       00 cc                   add    %cl,%ah
> ffffffff82b042ab:       00 00                   add    %al,(%rax)
>
> I am not sure what the right behavior is per specification.
>
> Thanks,
> Song

AFAIK GNU ld's behavior is not documented here
https://sourceware.org/binutils/docs/ld/Output-Section-Fill.html

The Output Section Fill syntax allows an expression. It seems that if
you use =0xcc+0, GNU ld will use a 4-byte filler as well, similar to
gold and ld.lld.

Frankly, I feel that GNU ld behavior should not be relied upon. lld's
comment states that it is a deliberate choice to follow gold instead
of GNU ld here.

I'll elaborate a bit and add this discrepancy to my
https://maskray.me/blog/2020-12-19-lld-and-gnu-linker-incompatibilities
:)



-- 
宋方睿

[tip: x86/urgent] x86/build: Fix linker fill bytes quirk/incompatibility for ld.lld

Posted by tip-bot2 for Song Liu 2 years, 3 months ago

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     65e710899fd19f435f40268f3a92dfaa11f14470
Gitweb:        https://git.kernel.org/tip/65e710899fd19f435f40268f3a92dfaa11f14470
Author:        Song Liu <song@kernel.org>
AuthorDate:    Wed, 06 Sep 2023 10:52:15 -07:00
Committer:     Ingo Molnar <mingo@kernel.org>
CommitterDate: Wed, 06 Sep 2023 23:49:12 +02:00

x86/build: Fix linker fill bytes quirk/incompatibility for ld.lld

With ":text =0xcccc", ld.lld fills unused text area with 0xcccc0000.
Example objdump -D output:

	ffffffff82b04203:       00 00                   add    %al,(%rax)
	ffffffff82b04205:       cc                      int3
	ffffffff82b04206:       cc                      int3
	ffffffff82b04207:       00 00                   add    %al,(%rax)
	ffffffff82b04209:       cc                      int3
	ffffffff82b0420a:       cc                      int3

Replace it with ":text =0xcccccccc", so we get the following instead:

	ffffffff82b04203:       cc                      int3
	ffffffff82b04204:       cc                      int3
	ffffffff82b04205:       cc                      int3
	ffffffff82b04206:       cc                      int3
	ffffffff82b04207:       cc                      int3
	ffffffff82b04208:       cc                      int3

gcc/ld doesn't seem to have the same issue. The generated code stays the
same for gcc/ld.

Signed-off-by: Song Liu <song@kernel.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Kees Cook <keescook@chromium.org>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Fixes: 7705dc855797 ("x86/vmlinux: Use INT3 instead of NOP for linker fill bytes")
Link: https://lore.kernel.org/r/20230906175215.2236033-1-song@kernel.org
---
 arch/x86/kernel/vmlinux.lds.S | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S
index 83d41c2..f15fb71 100644
--- a/arch/x86/kernel/vmlinux.lds.S
+++ b/arch/x86/kernel/vmlinux.lds.S
@@ -156,7 +156,7 @@ SECTIONS
 		ALIGN_ENTRY_TEXT_END
 		*(.gnu.warning)
 
-	} :text =0xcccc
+	} :text = 0xcccccccc
 
 	/* End of text section, which should occupy whole number of pages */
 	_etext = .;