From: Frediano Ziglio <frediano.ziglio@cloud.com>

Using EFI Secure Boot all kernel level code should be signed and
there should be no way to run unchecked code.
For this reason the Kexec interface needs to be changed in order
to allows signature checking.

The purgatory code is included in Xen itself as passing this code
from userspace it's not secure (see patches 2/4 and 3/4).

Changes since v1:
- update copyright lines;
- better sha2 declarations.

Ross Lagerwall (4):
  xen/lib: Export additional sha256 functions
  kexec: Include purgatory in Xen
  kexec: Implement new EFI load types
  kexec: Support non-page-aligned kexec segments

 xen/arch/arm/Makefile                 |   1 +
 xen/arch/arm/kexec.c                  |  27 +
 xen/arch/x86/Makefile                 |   2 +
 xen/arch/x86/bzimage.c                |  40 +-
 xen/arch/x86/kexec.c                  | 125 +++++
 xen/arch/x86/purgatory/.gitignore     |   3 +
 xen/arch/x86/purgatory/Makefile       |  64 +++
 xen/arch/x86/purgatory/config.h       |  37 ++
 xen/arch/x86/purgatory/entry64.S      | 108 ++++
 xen/arch/x86/purgatory/purgatory.c    |  59 +++
 xen/arch/x86/purgatory/setup-x86_64.S |  63 +++
 xen/arch/x86/purgatory/stack.S        |  21 +
 xen/common/Kconfig                    |   1 +
 xen/common/kexec.c                    |  33 +-
 xen/common/kimage.c                   | 703 ++++++++++++++++++++++++--
 xen/include/public/kexec.h            |  23 +-
 xen/include/xen/kimage.h              |  57 ++-
 xen/include/xen/sha2.h                |  12 +
 xen/include/xen/x86-linux.h           |  62 +++
 xen/lib/sha2-256.c                    |  19 +-
 20 files changed, 1348 insertions(+), 112 deletions(-)
 create mode 100644 xen/arch/arm/kexec.c
 create mode 100644 xen/arch/x86/kexec.c
 create mode 100644 xen/arch/x86/purgatory/.gitignore
 create mode 100644 xen/arch/x86/purgatory/Makefile
 create mode 100644 xen/arch/x86/purgatory/config.h
 create mode 100644 xen/arch/x86/purgatory/entry64.S
 create mode 100644 xen/arch/x86/purgatory/purgatory.c
 create mode 100644 xen/arch/x86/purgatory/setup-x86_64.S
 create mode 100644 xen/arch/x86/purgatory/stack.S
 create mode 100644 xen/include/xen/x86-linux.h

-- 
2.43.0

On 07.05.2025 11:42, Frediano Ziglio wrote:
> From: Frediano Ziglio <frediano.ziglio@cloud.com>
> 
> Using EFI Secure Boot all kernel level code should be signed and
> there should be no way to run unchecked code.
> For this reason the Kexec interface needs to be changed in order
> to allows signature checking.
> 
> The purgatory code is included in Xen itself as passing this code
> from userspace it's not secure (see patches 2/4 and 3/4).
> 
> Changes since v1:
> - update copyright lines;
> - better sha2 declarations.
> 
> Ross Lagerwall (4):
>   xen/lib: Export additional sha256 functions
>   kexec: Include purgatory in Xen
>   kexec: Implement new EFI load types
>   kexec: Support non-page-aligned kexec segments

As a general remark: You're sending all of these patches on Ross' behalf, yet
none of them has your own S-o-b. It is my understanding that strictly speaking
we wouldn't be permitted to take such patches.

Jan

On 07/05/2025 10:42 am, Frediano Ziglio wrote:
> Ross Lagerwall (4):
>   xen/lib: Export additional sha256 functions
>   kexec: Include purgatory in Xen
>   kexec: Implement new EFI load types
>   kexec: Support non-page-aligned kexec segments

I realise a lot of this is coming from kexec-tools and/or Linux, but it
looks very very mad.

From patch 1, we're embedding this in Xen:

xen.git/xen/arch/x86/purgatory$ ls -lah purgatory.ro
-rw-r--r-- 1 andrew andrew 30K May  9 15:24 purgatory.ro

yet -Wa,--strip-local-absolute alone halves the size:

xen.git/xen/arch/x86/purgatory$ ls -lah purgatory.ro
-rw-r--r-- 1 andrew andrew 17K May  9 15:25 purgatory.ro

Looking at purgatory itself, we enter at purgatory_start, load a local
GDT, set up a local stack, call into C for the hashing (and nothing
else), then jmp to entry64...

... which loads a (different) local GDT, (different) local stack, loads
the GPRs and then jumps into the new kernel.

Combined with kexec_reloc(), that's 3x we change GDT and stack in
several hundred instructions.


Looking further at patch 2, we only set up 3 GPRs; %rip, %rsp and %rdi
pointing the parameter block.

Patch 2 also contains an a large amount of EFI-editing logic (all
vulnerable to XSA-25), which AFAICT exists only because purgatory is
built non-PIC and wants relocating.  I can't see any external
references, or anything that couldn't be resolved at link time for a PIC
build.


There are two things which purgatory does which Xen doesn't currently
cater for:

1) Setting up the GPRs in that manner
2) The digest checks

#1 is very easy to fix and can probably even be done on the current ABI
(older Kexecs using purgatory won't care), and #2 ought to be easy too
by extending machine_kexec().  We can do the digest checks
unconditionally (it's a sensible check irrespective).

I think that removes the majority of this series, with no loss in
functionality?

Given that we're leaving the signature check to the dom0 kernel (which
is TCB and therefore can in the UEFI-SB model), we just might be able to
get away without any hypercall changes at all?

Thoughts?

~Andrew

On Fri, May 9, 2025 at 4:04 PM Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>
> On 07/05/2025 10:42 am, Frediano Ziglio wrote:
> > Ross Lagerwall (4):
> >   xen/lib: Export additional sha256 functions
> >   kexec: Include purgatory in Xen
> >   kexec: Implement new EFI load types
> >   kexec: Support non-page-aligned kexec segments
>
> I realise a lot of this is coming from kexec-tools and/or Linux, but it
> looks very very mad.
>
> From patch 1, we're embedding this in Xen:
>
> xen.git/xen/arch/x86/purgatory$ ls -lah purgatory.ro
> -rw-r--r-- 1 andrew andrew 30K May  9 15:24 purgatory.ro
>
> yet -Wa,--strip-local-absolute alone halves the size:
>
> xen.git/xen/arch/x86/purgatory$ ls -lah purgatory.ro
> -rw-r--r-- 1 andrew andrew 17K May  9 15:25 purgatory.ro
>
> Looking at purgatory itself, we enter at purgatory_start, load a local
> GDT, set up a local stack, call into C for the hashing (and nothing
> else), then jmp to entry64...
>
> ... which loads a (different) local GDT, (different) local stack, loads
> the GPRs and then jumps into the new kernel.
>
> Combined with kexec_reloc(), that's 3x we change GDT and stack in
> several hundred instructions.
>
>
> Looking further at patch 2, we only set up 3 GPRs; %rip, %rsp and %rdi
> pointing the parameter block.
>
> Patch 2 also contains an a large amount of EFI-editing logic (all
> vulnerable to XSA-25), which AFAICT exists only because purgatory is
> built non-PIC and wants relocating.  I can't see any external
> references, or anything that couldn't be resolved at link time for a PIC
> build.
>
>
> There are two things which purgatory does which Xen doesn't currently
> cater for:
>
> 1) Setting up the GPRs in that manner
> 2) The digest checks
>
> #1 is very easy to fix and can probably even be done on the current ABI
> (older Kexecs using purgatory won't care), and #2 ought to be easy too
> by extending machine_kexec().  We can do the digest checks
> unconditionally (it's a sensible check irrespective).
>

I think the problem of #2 is that doing in the purgatory avoids
problems like possible memory corruptions. For instance if the host is
crashing due to some corruption it could not always be possible to
boot the saved kernel.

> I think that removes the majority of this series, with no loss in
> functionality?
>
> Given that we're leaving the signature check to the dom0 kernel (which
> is TCB and therefore can in the UEFI-SB model), we just might be able to
> get away without any hypercall changes at all?
>

Yes and no. The user space could not provide the purgatory. But if the
kernel is providing it, preventing the user space to send it, I
suppose it can be done. At this point however the question is how to
change the interface provided to userspace for doing it. It could make
sense to have the changes in xen/include/public/kexec.h and let the
kernel do the rest.

> Thoughts?
>
> ~Andrew

Frediano

On 09/05/2025 4:34 pm, Frediano Ziglio wrote:
> On Fri, May 9, 2025 at 4:04 PM Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>> On 07/05/2025 10:42 am, Frediano Ziglio wrote:
>>> Ross Lagerwall (4):
>>>   xen/lib: Export additional sha256 functions
>>>   kexec: Include purgatory in Xen
>>>   kexec: Implement new EFI load types
>>>   kexec: Support non-page-aligned kexec segments
>> I realise a lot of this is coming from kexec-tools and/or Linux, but it
>> looks very very mad.
>>
>> From patch 1, we're embedding this in Xen:
>>
>> xen.git/xen/arch/x86/purgatory$ ls -lah purgatory.ro
>> -rw-r--r-- 1 andrew andrew 30K May  9 15:24 purgatory.ro
>>
>> yet -Wa,--strip-local-absolute alone halves the size:
>>
>> xen.git/xen/arch/x86/purgatory$ ls -lah purgatory.ro
>> -rw-r--r-- 1 andrew andrew 17K May  9 15:25 purgatory.ro
>>
>> Looking at purgatory itself, we enter at purgatory_start, load a local
>> GDT, set up a local stack, call into C for the hashing (and nothing
>> else), then jmp to entry64...
>>
>> ... which loads a (different) local GDT, (different) local stack, loads
>> the GPRs and then jumps into the new kernel.
>>
>> Combined with kexec_reloc(), that's 3x we change GDT and stack in
>> several hundred instructions.
>>
>>
>> Looking further at patch 2, we only set up 3 GPRs; %rip, %rsp and %rdi
>> pointing the parameter block.
>>
>> Patch 2 also contains an a large amount of EFI-editing logic (all
>> vulnerable to XSA-25), which AFAICT exists only because purgatory is
>> built non-PIC and wants relocating.  I can't see any external
>> references, or anything that couldn't be resolved at link time for a PIC
>> build.
>>
>>
>> There are two things which purgatory does which Xen doesn't currently
>> cater for:
>>
>> 1) Setting up the GPRs in that manner
>> 2) The digest checks
>>
>> #1 is very easy to fix and can probably even be done on the current ABI
>> (older Kexecs using purgatory won't care), and #2 ought to be easy too
>> by extending machine_kexec().  We can do the digest checks
>> unconditionally (it's a sensible check irrespective).
>>
> I think the problem of #2 is that doing in the purgatory avoids
> problems like possible memory corruptions. For instance if the host is
> crashing due to some corruption it could not always be possible to
> boot the saved kernel.

It doesn't really matter if Xen does the digest check right at the point
of exiting, or purgatory does it moments later.

If there's memory corruption anywhere on this path, we're not making it
into the crash kernel whomever does the digest check.

Crashing is a best-effort exercise; it's never guaranteed to be successful.
>> I think that removes the majority of this series, with no loss in
>> functionality?
>>
>> Given that we're leaving the signature check to the dom0 kernel (which
>> is TCB and therefore can in the UEFI-SB model), we just might be able to
>> get away without any hypercall changes at all?
>>
> Yes and no. The user space could not provide the purgatory. But if the
> kernel is providing it, preventing the user space to send it, I
> suppose it can be done. At this point however the question is how to
> change the interface provided to userspace for doing it. It could make
> sense to have the changes in xen/include/public/kexec.h and let the
> kernel do the rest.

I'm not really suggesting any change in userpsace/dom0 kernel.  I'm
suggesting "we don't need a purgatory blob at all given two simple
changes in Xen."

This in turn means (I think) we can drop all the ELF relocation logic.

I'm unsure whether we need new hypercall subopts or not.  Even if we do,
I think the result can still be more simple than currently presented.

~Andrew

On Fri, May 9, 2025 at 6:40 PM Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>
> On 09/05/2025 4:34 pm, Frediano Ziglio wrote:
> > On Fri, May 9, 2025 at 4:04 PM Andrew Cooper <andrew.cooper3@citrix.com> wrote:
> >> On 07/05/2025 10:42 am, Frediano Ziglio wrote:
> >>> Ross Lagerwall (4):
> >>>   xen/lib: Export additional sha256 functions
> >>>   kexec: Include purgatory in Xen
> >>>   kexec: Implement new EFI load types
> >>>   kexec: Support non-page-aligned kexec segments
> >> I realise a lot of this is coming from kexec-tools and/or Linux, but it
> >> looks very very mad.
> >>
> >> From patch 1, we're embedding this in Xen:
> >>
> >> xen.git/xen/arch/x86/purgatory$ ls -lah purgatory.ro
> >> -rw-r--r-- 1 andrew andrew 30K May  9 15:24 purgatory.ro
> >>
> >> yet -Wa,--strip-local-absolute alone halves the size:
> >>
> >> xen.git/xen/arch/x86/purgatory$ ls -lah purgatory.ro
> >> -rw-r--r-- 1 andrew andrew 17K May  9 15:25 purgatory.ro
> >>
> >> Looking at purgatory itself, we enter at purgatory_start, load a local
> >> GDT, set up a local stack, call into C for the hashing (and nothing
> >> else), then jmp to entry64...
> >>
> >> ... which loads a (different) local GDT, (different) local stack, loads
> >> the GPRs and then jumps into the new kernel.
> >>
> >> Combined with kexec_reloc(), that's 3x we change GDT and stack in
> >> several hundred instructions.
> >>
> >>
> >> Looking further at patch 2, we only set up 3 GPRs; %rip, %rsp and %rdi
> >> pointing the parameter block.
> >>
> >> Patch 2 also contains an a large amount of EFI-editing logic (all
> >> vulnerable to XSA-25), which AFAICT exists only because purgatory is
> >> built non-PIC and wants relocating.  I can't see any external
> >> references, or anything that couldn't be resolved at link time for a PIC
> >> build.
> >>
> >>
> >> There are two things which purgatory does which Xen doesn't currently
> >> cater for:
> >>
> >> 1) Setting up the GPRs in that manner
> >> 2) The digest checks
> >>
> >> #1 is very easy to fix and can probably even be done on the current ABI
> >> (older Kexecs using purgatory won't care), and #2 ought to be easy too
> >> by extending machine_kexec().  We can do the digest checks
> >> unconditionally (it's a sensible check irrespective).
> >>
> > I think the problem of #2 is that doing in the purgatory avoids
> > problems like possible memory corruptions. For instance if the host is
> > crashing due to some corruption it could not always be possible to
> > boot the saved kernel.
>
> It doesn't really matter if Xen does the digest check right at the point
> of exiting, or purgatory does it moments later.
>
> If there's memory corruption anywhere on this path, we're not making it
> into the crash kernel whomever does the digest check.
>
> Crashing is a best-effort exercise; it's never guaranteed to be successful.
> >> I think that removes the majority of this series, with no loss in
> >> functionality?
> >>
> >> Given that we're leaving the signature check to the dom0 kernel (which
> >> is TCB and therefore can in the UEFI-SB model), we just might be able to
> >> get away without any hypercall changes at all?
> >>
> > Yes and no. The user space could not provide the purgatory. But if the
> > kernel is providing it, preventing the user space to send it, I
> > suppose it can be done. At this point however the question is how to
> > change the interface provided to userspace for doing it. It could make
> > sense to have the changes in xen/include/public/kexec.h and let the
> > kernel do the rest.
>
> I'm not really suggesting any change in userpsace/dom0 kernel.  I'm
> suggesting "we don't need a purgatory blob at all given two simple
> changes in Xen."
>

Maybe it was not clear from my previous comment. A change to the
userspace interface is needed. One reason is the mentioned purgatory,
it cannot be provided by userspace. The other reason is checking the
signature, the kernel should be passed verbatim in order to check it.

Which specific 2 changes are you referring to?

Given that the kernel already does the signature, having the kernel
providing the purgatory too would be sensible.

> This in turn means (I think) we can drop all the ELF relocation logic.
>
> I'm unsure whether we need new hypercall subopts or not.  Even if we do,
> I think the result can still be more simple than currently presented.
>
> ~Andrew

Possibly the Xen interface could stay the same. Still, as explained
above, a userspace change is needed.

Frediano

On 09/05/2025 4:04 pm, Andrew Cooper wrote:
> On 07/05/2025 10:42 am, Frediano Ziglio wrote:
>> Ross Lagerwall (4):
>>   xen/lib: Export additional sha256 functions
>>   kexec: Include purgatory in Xen
>>   kexec: Implement new EFI load types
>>   kexec: Support non-page-aligned kexec segments
> I realise a lot of this is coming from kexec-tools and/or Linux, but it
> looks very very mad.
>
> From patch 1, we're embedding this in Xen:
>
> xen.git/xen/arch/x86/purgatory$ ls -lah purgatory.ro
> -rw-r--r-- 1 andrew andrew 30K May  9 15:24 purgatory.ro
>
> yet -Wa,--strip-local-absolute alone halves the size:
>
> xen.git/xen/arch/x86/purgatory$ ls -lah purgatory.ro
> -rw-r--r-- 1 andrew andrew 17K May  9 15:25 purgatory.ro
>
> Looking at purgatory itself, we enter at purgatory_start, load a local
> GDT, set up a local stack, call into C for the hashing (and nothing
> else), then jmp to entry64...
>
> ... which loads a (different) local GDT, (different) local stack, loads
> the GPRs and then jumps into the new kernel.
>
> Combined with kexec_reloc(), that's 3x we change GDT and stack in
> several hundred instructions.
>
>
> Looking further at patch 2, we only set up 3 GPRs; %rip, %rsp and %rdi
> pointing the parameter block.
>
> Patch 2 also contains an a large amount of EFI-editing logic (all
> vulnerable to XSA-25), which AFAICT exists only because purgatory is
> built non-PIC and wants relocating.  I can't see any external
> references, or anything that couldn't be resolved at link time for a PIC
> build.
>
>
> There are two things which purgatory does which Xen doesn't currently
> cater for:
>
> 1) Setting up the GPRs in that manner
> 2) The digest checks
>
> #1 is very easy to fix and can probably even be done on the current ABI
> (older Kexecs using purgatory won't care), and #2 ought to be easy too
> by extending machine_kexec().  We can do the digest checks
> unconditionally (it's a sensible check irrespective).
>
> I think that removes the majority of this series, with no loss in
> functionality?
>
> Given that we're leaving the signature check to the dom0 kernel (which
> is TCB and therefore can in the UEFI-SB model), we just might be able to
> get away without any hypercall changes at all?
>
> Thoughts?

Sorry, one extra thing.  By doing the digest check in machine_kexec(),
we can also inform the user that the kexec kernel looks corrupt, and we
won't be entering it.  (And probably try to reboot rather than hanging too.)

~Andrew