xen: Support continuations from tasklets

[Xen-devel] [PATCH 0/6] xen: Support continuations from tasklets

Posted by Andrew Cooper 4 years, 4 months ago

This is the Post "x86/smt: Support for enabling/disabling SMT at runtime" work
which stumbled into discovering XSA-296, for the purpose of making
continuations more efficient.

From testing this series, I have re-confirmed the previous reported
observation that:

  # while :; do xen-hptool smt-enable; xen-hptool smt-disable; done

in dom0 eventually causes the serial console to cease working (wedge midway
through printing a line).

There are sporadic "Broke affinity for IRQ26, new: ffff" messages, but the
serial always seems to break shortly after the first "Broke affinity for
IRQ30, new: ffff".  Both IRQs are non-descript PCI-MSI/-X interrupts bound to
dom0.

Andrew Cooper (6):
  xen/tasklet: Fix return value truncation on arm64
  xen/tasklet: Switch data parameter from unsigned long to void *.
  xen/domctl: Consolidate hypercall continuation handling at the top level
  xen/hypercall: Cope with -ERESTART on more hypercall paths
  xen/tasklet: Return -ERESTART from continue_hypercall_on_cpu()
  x86/smt: Don't use -EBUSY for smt_up_down_helper() continuations

 xen/arch/arm/traps.c                      |  8 +++++++
 xen/arch/x86/domctl.c                     |  5 ++++-
 xen/arch/x86/hvm/hvm.c                    |  6 ++---
 xen/arch/x86/hvm/vlapic.c                 |  8 +++----
 xen/arch/x86/hypercall.c                  | 14 ++++++++++++
 xen/arch/x86/mm/hap/hap.c                 |  3 +--
 xen/arch/x86/mm/shadow/common.c           |  7 +++---
 xen/arch/x86/platform_hypercall.c         | 14 ++++++++++--
 xen/arch/x86/sysctl.c                     |  9 ++++++--
 xen/common/compat/domain.c                |  9 ++++----
 xen/common/domain.c                       | 37 +++++++++++++++++--------------
 xen/common/domctl.c                       | 19 +++++-----------
 xen/common/kexec.c                        | 20 +++++++++++++----
 xen/common/keyhandler.c                   | 19 ++++++++--------
 xen/common/stop_machine.c                 |  5 +++--
 xen/common/sysctl.c                       | 13 +++++++++--
 xen/common/tasklet.c                      |  6 ++---
 xen/common/trace.c                        |  4 ++--
 xen/drivers/char/console.c                |  4 ++--
 xen/drivers/passthrough/amd/iommu_guest.c |  7 +++---
 xen/drivers/passthrough/amd/iommu_init.c  |  6 ++---
 xen/drivers/passthrough/iommu.c           |  4 ++--
 xen/drivers/passthrough/vtd/iommu.c       |  4 ++--
 xen/include/asm-arm/regs.h                |  2 --
 xen/include/asm-x86/regs.h                |  2 --
 xen/include/asm-x86/shadow.h              |  5 ++---
 xen/include/xen/domain.h                  | 17 +++++++++++---
 xen/include/xen/tasklet.h                 | 10 ++++-----
 28 files changed, 158 insertions(+), 109 deletions(-)

-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 0/6] xen: Support continuations from tasklets

Posted by Jan Beulich 4 years, 4 months ago

On 05.12.2019 23:30, Andrew Cooper wrote:
> From testing this series, I have re-confirmed the previous reported
> observation that:
> 
>   # while :; do xen-hptool smt-enable; xen-hptool smt-disable; done
> 
> in dom0 eventually causes the serial console to cease working (wedge midway
> through printing a line).
> 
> There are sporadic "Broke affinity for IRQ26, new: ffff" messages, but the
> serial always seems to break shortly after the first "Broke affinity for
> IRQ30, new: ffff".  Both IRQs are non-descript PCI-MSI/-X interrupts bound to
> dom0.

And neither IRQ30 not IRQ26 are the serial ones? And serial does
use an IRQ (i.e. isn't running in polling mode)? I'll see if I
can repro (and then maybe be able to debug).

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 0/6] xen: Support continuations from tasklets

Posted by Andrew Cooper 4 years, 4 months ago

On 06/12/2019 09:58, Jan Beulich wrote:
> On 05.12.2019 23:30, Andrew Cooper wrote:
>> From testing this series, I have re-confirmed the previous reported
>> observation that:
>>
>>   # while :; do xen-hptool smt-enable; xen-hptool smt-disable; done
>>
>> in dom0 eventually causes the serial console to cease working (wedge midway
>> through printing a line).
>>
>> There are sporadic "Broke affinity for IRQ26, new: ffff" messages, but the
>> serial always seems to break shortly after the first "Broke affinity for
>> IRQ30, new: ffff".  Both IRQs are non-descript PCI-MSI/-X interrupts bound to
>> dom0.
> And neither IRQ30 not IRQ26 are the serial ones? And serial does
> use an IRQ (i.e. isn't running in polling mode)? I'll see if I
> can repro (and then maybe be able to debug).

Serial uses IRQ0 and never has affinity changes as it is always bound to
CPU0.

Given our recent fun with the PEOI stack, that was going to be my next
port of call.  However everything else in the system looks to be working
fine, including the disks and network, suggesting that we aren't losing
other interrupts.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 0/6] xen: Support continuations from tasklets

Posted by Jan Beulich 4 years, 4 months ago

On 06.12.2019 11:14, Andrew Cooper wrote:
> On 06/12/2019 09:58, Jan Beulich wrote:
>> On 05.12.2019 23:30, Andrew Cooper wrote:
>>> From testing this series, I have re-confirmed the previous reported
>>> observation that:
>>>
>>>   # while :; do xen-hptool smt-enable; xen-hptool smt-disable; done
>>>
>>> in dom0 eventually causes the serial console to cease working (wedge midway
>>> through printing a line).
>>>
>>> There are sporadic "Broke affinity for IRQ26, new: ffff" messages, but the
>>> serial always seems to break shortly after the first "Broke affinity for
>>> IRQ30, new: ffff".  Both IRQs are non-descript PCI-MSI/-X interrupts bound to
>>> dom0.
>> And neither IRQ30 not IRQ26 are the serial ones? And serial does
>> use an IRQ (i.e. isn't running in polling mode)? I'll see if I
>> can repro (and then maybe be able to debug).
> 
> Serial uses IRQ0 and never has affinity changes as it is always bound to
> CPU0.

IRQ0? DYM IRQ3 or IRQ4? (In any event the important part is for it
to be in the ISA range, rather than the PCI one.)

> Given our recent fun with the PEOI stack, that was going to be my next
> port of call.  However everything else in the system looks to be working
> fine, including the disks and network, suggesting that we aren't losing
> other interrupts.

The high priority vector we use for serial makes the PEOI stack
unrelated afaict, as mentioned in the discussion there as well.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 0/6] xen: Support continuations from tasklets

Posted by Andrew Cooper 4 years, 4 months ago

On 06/12/2019 10:18, Jan Beulich wrote:
> On 06.12.2019 11:14, Andrew Cooper wrote:
>> On 06/12/2019 09:58, Jan Beulich wrote:
>>> On 05.12.2019 23:30, Andrew Cooper wrote:
>>>> From testing this series, I have re-confirmed the previous reported
>>>> observation that:
>>>>
>>>>   # while :; do xen-hptool smt-enable; xen-hptool smt-disable; done
>>>>
>>>> in dom0 eventually causes the serial console to cease working (wedge midway
>>>> through printing a line).
>>>>
>>>> There are sporadic "Broke affinity for IRQ26, new: ffff" messages, but the
>>>> serial always seems to break shortly after the first "Broke affinity for
>>>> IRQ30, new: ffff".  Both IRQs are non-descript PCI-MSI/-X interrupts bound to
>>>> dom0.
>>> And neither IRQ30 not IRQ26 are the serial ones? And serial does
>>> use an IRQ (i.e. isn't running in polling mode)? I'll see if I
>>> can repro (and then maybe be able to debug).
>> Serial uses IRQ0 and never has affinity changes as it is always bound to
>> CPU0.
> IRQ0? DYM IRQ3 or IRQ4? (In any event the important part is for it
> to be in the ISA range, rather than the PCI one.)

I do mean IRQ4, sorry.  Mixed it up with the timer IRQ.

On SMT disable, it does lose half of its affinity.

(XEN)    IRQ:   4 vec:f1 IO-APIC-edge    status=000 aff:{0-7}/{0,2,4,6}
drivers/char/ns16550.c#ns16550_interrupt()

So this is more likely to be related to a irq-migration than LAPIC acks.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel