[PATCH v3 0/5] x86/xen: Drop absolute references from startup code

Ard Biesheuvel posted 5 patches 1 month, 2 weeks ago
arch/x86/kernel/vmlinux.lds.S | 19 ++++++++
arch/x86/platform/pvh/head.S  | 50 +++++++++++---------
arch/x86/tools/relocs.c       |  1 +
arch/x86/xen/xen-head.S       |  6 ++-
4 files changed, 52 insertions(+), 24 deletions(-)
[PATCH v3 0/5] x86/xen: Drop absolute references from startup code
Posted by Ard Biesheuvel 1 month, 2 weeks ago
From: Ard Biesheuvel <ardb@kernel.org>

This series was broken out of the series I sent recently [0], after
Jason pointed out that my Xen startup code changes conflict with his
changes to make the PVH startup code position independent.

Jason's work reduces the delta of my changes, and given that my other
series will likely advance at a much slower pace, the Xen changes are
presented here so they can be merged independently.

The end result after applying this series (see below) is that there are
no longer any Xen-related absolute relocations that need to be applied
to .head.text, a section which carries code that may be invoked from the
1:1 mapping of memory before the kernel virtual mapping is up.  The use
of absolute references in this code section has resulted in a few boot
issues that were very hard to track down (Clang built kernels running
under SEV-SNP in particular, which does not provide the best debug
experience).

Even though the occurrences in the Xen startup code were fine, there is
now a lot of C code emitted into .head.text as well, and so it would be
helpful to teach objtool to reject absolute references entirely in this
section (or rely on the linker for that). Therefore, not relying on them
in the first place is a step towards that goal.

Changes since v2 [2]:
- add Jason's Tested-by to patch #4
- use a better name for the linker defined symbols used in the ELF notes
  (patch #4)
- add a comment in the linker script explaining why the symbol values
  are constructed in the way they are
- rebase onto v6.12-rc2

Changes since v1 [1]:
- add Jason's Rb to patches #2, #3 and #5
- drop the use of a 32-bit field for the ELF note- QEMU reads a u64 and
  so the top word needs to remain 0x0
- tweak #ifdefs in patch #4 so the hypercall_page linker symbol does not
  depend on CONFIG_XEN_PV
- rebase onto v6.12-rc1

Changes wrt [0]:
- add Jason's Rb to patch #1
- rebase onto xen/tip's linux-next branch
- split out fix for GDT descriptor size field
- add patch to remove the zeroing of phys_base, which is no longer
  needed
- use a 32-bit field for XEN_ELFNOTE_PHYS32_ENTRY, and use its contents
  to obtain the build time physical address of pvh_startup_xen()

[0] https://lore.kernel.org/all/20240925150059.3955569-30-ardb+git@google.com
[1] https://lore.kernel.org/all/20240926104113.80146-7-ardb+git@google.com/
[2] https://lore.kernel.org/all/20240930071513.909462-7-ardb+git@google.com/

Relocation section '.rela.head.text' at offset 0xb428 contains 15 entries:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
000000000018  000800000002 R_X86_64_PC32     0000000000000000 .init.data + 18
00000000002f  000e00000002 R_X86_64_PC32     0000000000000000 pvh_start_info + 2f
000000000037  000f00000002 R_X86_64_PC32     0000000000000000 pvh_start_info_sz + 37
000000000042  000800000002 R_X86_64_PC32     0000000000000000 .init.data + 4092
000000000060  001000000002 R_X86_64_PC32     000000000000002c xen_elfnote_phys3[...] + 60
000000000068  001100000002 R_X86_64_PC32     0000000000000000 phys_base + 68
00000000006e  001200000002 R_X86_64_PC32     0000000000005000 pvh_init_top_pgt + 6e
000000000089  001300000002 R_X86_64_PC32     0000000000006000 pvh_level3_ident_pgt + 89
000000000091  001400000002 R_X86_64_PC32     0000000000008000 pvh_level3_kernel_pgt + 91
0000000000a3  001500000002 R_X86_64_PC32     0000000000009000 pvh_level2_kernel_pgt + a3
0000000000be  001200000002 R_X86_64_PC32     0000000000005000 pvh_init_top_pgt + be
0000000000de  000800000002 R_X86_64_PC32     0000000000000000 .init.data + 1c
0000000000e9  001600000002 R_X86_64_PC32     0000000000000000 xen_prepare_pvh - 4
0000000000f8  001700000002 R_X86_64_PC32     0000000000000000 pvh_bootparams - 4
0000000000fd  001800000004 R_X86_64_PLT32    0000000000000000 startup_64 - 4

Relocation section '.rela.note.Xen' at offset 0xb668 contains 1 entry:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
00000000002c  001a00000002 R_X86_64_PC32     0000000000000000 xen_elfnote_phys3[...] + 0

Cc: Jason Andryuk <jason.andryuk@amd.com>
Cc: Juergen Gross <jgross@suse.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: x86@kernel.org
Cc: xen-devel@lists.xenproject.org

Ard Biesheuvel (5):
  x86/pvh: Call C code via the kernel virtual mapping
  x86/pvh: Use correct size value in GDT descriptor
  x86/pvh: Omit needless clearing of phys_base
  x86/xen: Avoid relocatable quantities in Xen ELF notes
  x86/pvh: Avoid absolute symbol references in .head.text

 arch/x86/kernel/vmlinux.lds.S | 19 ++++++++
 arch/x86/platform/pvh/head.S  | 50 +++++++++++---------
 arch/x86/tools/relocs.c       |  1 +
 arch/x86/xen/xen-head.S       |  6 ++-
 4 files changed, 52 insertions(+), 24 deletions(-)

-- 
2.47.0.rc0.187.ge670bccf7e-goog
Re: [PATCH v3 0/5] x86/xen: Drop absolute references from startup code
Posted by Ard Biesheuvel 4 weeks, 1 day ago
On Wed, 9 Oct 2024 at 18:09, Ard Biesheuvel <ardb+git@google.com> wrote:
>
> From: Ard Biesheuvel <ardb@kernel.org>
>
> This series was broken out of the series I sent recently [0], after
> Jason pointed out that my Xen startup code changes conflict with his
> changes to make the PVH startup code position independent.
>
> Jason's work reduces the delta of my changes, and given that my other
> series will likely advance at a much slower pace, the Xen changes are
> presented here so they can be merged independently.
>
> The end result after applying this series (see below) is that there are
> no longer any Xen-related absolute relocations that need to be applied
> to .head.text, a section which carries code that may be invoked from the
> 1:1 mapping of memory before the kernel virtual mapping is up.  The use
> of absolute references in this code section has resulted in a few boot
> issues that were very hard to track down (Clang built kernels running
> under SEV-SNP in particular, which does not provide the best debug
> experience).
>
> Even though the occurrences in the Xen startup code were fine, there is
> now a lot of C code emitted into .head.text as well, and so it would be
> helpful to teach objtool to reject absolute references entirely in this
> section (or rely on the linker for that). Therefore, not relying on them
> in the first place is a step towards that goal.
>
> Changes since v2 [2]:
> - add Jason's Tested-by to patch #4
> - use a better name for the linker defined symbols used in the ELF notes
>   (patch #4)
> - add a comment in the linker script explaining why the symbol values
>   are constructed in the way they are
> - rebase onto v6.12-rc2
>
> Changes since v1 [1]:
> - add Jason's Rb to patches #2, #3 and #5
> - drop the use of a 32-bit field for the ELF note- QEMU reads a u64 and
>   so the top word needs to remain 0x0
> - tweak #ifdefs in patch #4 so the hypercall_page linker symbol does not
>   depend on CONFIG_XEN_PV
> - rebase onto v6.12-rc1
>
> Changes wrt [0]:
> - add Jason's Rb to patch #1
> - rebase onto xen/tip's linux-next branch
> - split out fix for GDT descriptor size field
> - add patch to remove the zeroing of phys_base, which is no longer
>   needed
> - use a 32-bit field for XEN_ELFNOTE_PHYS32_ENTRY, and use its contents
>   to obtain the build time physical address of pvh_startup_xen()
>
> [0] https://lore.kernel.org/all/20240925150059.3955569-30-ardb+git@google.com
> [1] https://lore.kernel.org/all/20240926104113.80146-7-ardb+git@google.com/
> [2] https://lore.kernel.org/all/20240930071513.909462-7-ardb+git@google.com/
>
> Relocation section '.rela.head.text' at offset 0xb428 contains 15 entries:
>   Offset          Info           Type           Sym. Value    Sym. Name + Addend
> 000000000018  000800000002 R_X86_64_PC32     0000000000000000 .init.data + 18
> 00000000002f  000e00000002 R_X86_64_PC32     0000000000000000 pvh_start_info + 2f
> 000000000037  000f00000002 R_X86_64_PC32     0000000000000000 pvh_start_info_sz + 37
> 000000000042  000800000002 R_X86_64_PC32     0000000000000000 .init.data + 4092
> 000000000060  001000000002 R_X86_64_PC32     000000000000002c xen_elfnote_phys3[...] + 60
> 000000000068  001100000002 R_X86_64_PC32     0000000000000000 phys_base + 68
> 00000000006e  001200000002 R_X86_64_PC32     0000000000005000 pvh_init_top_pgt + 6e
> 000000000089  001300000002 R_X86_64_PC32     0000000000006000 pvh_level3_ident_pgt + 89
> 000000000091  001400000002 R_X86_64_PC32     0000000000008000 pvh_level3_kernel_pgt + 91
> 0000000000a3  001500000002 R_X86_64_PC32     0000000000009000 pvh_level2_kernel_pgt + a3
> 0000000000be  001200000002 R_X86_64_PC32     0000000000005000 pvh_init_top_pgt + be
> 0000000000de  000800000002 R_X86_64_PC32     0000000000000000 .init.data + 1c
> 0000000000e9  001600000002 R_X86_64_PC32     0000000000000000 xen_prepare_pvh - 4
> 0000000000f8  001700000002 R_X86_64_PC32     0000000000000000 pvh_bootparams - 4
> 0000000000fd  001800000004 R_X86_64_PLT32    0000000000000000 startup_64 - 4
>
> Relocation section '.rela.note.Xen' at offset 0xb668 contains 1 entry:
>   Offset          Info           Type           Sym. Value    Sym. Name + Addend
> 00000000002c  001a00000002 R_X86_64_PC32     0000000000000000 xen_elfnote_phys3[...] + 0
>
> Cc: Jason Andryuk <jason.andryuk@amd.com>
> Cc: Juergen Gross <jgross@suse.com>
> Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
> Cc: x86@kernel.org
> Cc: xen-devel@lists.xenproject.org
>

Ping?
Re: [PATCH v3 0/5] x86/xen: Drop absolute references from startup code
Posted by Jürgen Groß 4 weeks, 1 day ago
On 29.10.24 13:50, Ard Biesheuvel wrote:
> On Wed, 9 Oct 2024 at 18:09, Ard Biesheuvel <ardb+git@google.com> wrote:
>>
>> From: Ard Biesheuvel <ardb@kernel.org>
>>
>> This series was broken out of the series I sent recently [0], after
>> Jason pointed out that my Xen startup code changes conflict with his
>> changes to make the PVH startup code position independent.
>>
>> Jason's work reduces the delta of my changes, and given that my other
>> series will likely advance at a much slower pace, the Xen changes are
>> presented here so they can be merged independently.
>>
>> The end result after applying this series (see below) is that there are
>> no longer any Xen-related absolute relocations that need to be applied
>> to .head.text, a section which carries code that may be invoked from the
>> 1:1 mapping of memory before the kernel virtual mapping is up.  The use
>> of absolute references in this code section has resulted in a few boot
>> issues that were very hard to track down (Clang built kernels running
>> under SEV-SNP in particular, which does not provide the best debug
>> experience).
>>
>> Even though the occurrences in the Xen startup code were fine, there is
>> now a lot of C code emitted into .head.text as well, and so it would be
>> helpful to teach objtool to reject absolute references entirely in this
>> section (or rely on the linker for that). Therefore, not relying on them
>> in the first place is a step towards that goal.
>>
>> Changes since v2 [2]:
>> - add Jason's Tested-by to patch #4
>> - use a better name for the linker defined symbols used in the ELF notes
>>    (patch #4)
>> - add a comment in the linker script explaining why the symbol values
>>    are constructed in the way they are
>> - rebase onto v6.12-rc2
>>
>> Changes since v1 [1]:
>> - add Jason's Rb to patches #2, #3 and #5
>> - drop the use of a 32-bit field for the ELF note- QEMU reads a u64 and
>>    so the top word needs to remain 0x0
>> - tweak #ifdefs in patch #4 so the hypercall_page linker symbol does not
>>    depend on CONFIG_XEN_PV
>> - rebase onto v6.12-rc1
>>
>> Changes wrt [0]:
>> - add Jason's Rb to patch #1
>> - rebase onto xen/tip's linux-next branch
>> - split out fix for GDT descriptor size field
>> - add patch to remove the zeroing of phys_base, which is no longer
>>    needed
>> - use a 32-bit field for XEN_ELFNOTE_PHYS32_ENTRY, and use its contents
>>    to obtain the build time physical address of pvh_startup_xen()
>>
>> [0] https://lore.kernel.org/all/20240925150059.3955569-30-ardb+git@google.com
>> [1] https://lore.kernel.org/all/20240926104113.80146-7-ardb+git@google.com/
>> [2] https://lore.kernel.org/all/20240930071513.909462-7-ardb+git@google.com/
>>
>> Relocation section '.rela.head.text' at offset 0xb428 contains 15 entries:
>>    Offset          Info           Type           Sym. Value    Sym. Name + Addend
>> 000000000018  000800000002 R_X86_64_PC32     0000000000000000 .init.data + 18
>> 00000000002f  000e00000002 R_X86_64_PC32     0000000000000000 pvh_start_info + 2f
>> 000000000037  000f00000002 R_X86_64_PC32     0000000000000000 pvh_start_info_sz + 37
>> 000000000042  000800000002 R_X86_64_PC32     0000000000000000 .init.data + 4092
>> 000000000060  001000000002 R_X86_64_PC32     000000000000002c xen_elfnote_phys3[...] + 60
>> 000000000068  001100000002 R_X86_64_PC32     0000000000000000 phys_base + 68
>> 00000000006e  001200000002 R_X86_64_PC32     0000000000005000 pvh_init_top_pgt + 6e
>> 000000000089  001300000002 R_X86_64_PC32     0000000000006000 pvh_level3_ident_pgt + 89
>> 000000000091  001400000002 R_X86_64_PC32     0000000000008000 pvh_level3_kernel_pgt + 91
>> 0000000000a3  001500000002 R_X86_64_PC32     0000000000009000 pvh_level2_kernel_pgt + a3
>> 0000000000be  001200000002 R_X86_64_PC32     0000000000005000 pvh_init_top_pgt + be
>> 0000000000de  000800000002 R_X86_64_PC32     0000000000000000 .init.data + 1c
>> 0000000000e9  001600000002 R_X86_64_PC32     0000000000000000 xen_prepare_pvh - 4
>> 0000000000f8  001700000002 R_X86_64_PC32     0000000000000000 pvh_bootparams - 4
>> 0000000000fd  001800000004 R_X86_64_PLT32    0000000000000000 startup_64 - 4
>>
>> Relocation section '.rela.note.Xen' at offset 0xb668 contains 1 entry:
>>    Offset          Info           Type           Sym. Value    Sym. Name + Addend
>> 00000000002c  001a00000002 R_X86_64_PC32     0000000000000000 xen_elfnote_phys3[...] + 0
>>
>> Cc: Jason Andryuk <jason.andryuk@amd.com>
>> Cc: Juergen Gross <jgross@suse.com>
>> Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
>> Cc: x86@kernel.org
>> Cc: xen-devel@lists.xenproject.org
>>
> 
> Ping?

I have queued this series for 6.13.


Juergen
Re: [PATCH v3 0/5] x86/xen: Drop absolute references from startup code
Posted by Ard Biesheuvel 3 weeks, 6 days ago
On Tue, 29 Oct 2024 at 13:54, Jürgen Groß <jgross@suse.com> wrote:
>
> On 29.10.24 13:50, Ard Biesheuvel wrote:
> > On Wed, 9 Oct 2024 at 18:09, Ard Biesheuvel <ardb+git@google.com> wrote:
> >>
..
> >
> > Ping?
>
> I have queued this series for 6.13.
>

Thanks!