[PATCH v2 00/15] xen: drop hypercall function tables

Juergen Gross posted 15 patches 2 years, 5 months ago
Failed in applying to current master (apply log)
There is a newer version of this series
.gitignore                               |   5 +-
tools/misc/xenperf.c                     |   5 +
xen/Makefile                             |  10 +
xen/arch/arm/domain.c                    |  15 +-
xen/arch/arm/hvm.c                       |   3 +-
xen/arch/arm/physdev.c                   |   2 +-
xen/arch/arm/platform_hypercall.c        |   1 +
xen/arch/arm/traps.c                     | 124 ++-------
xen/arch/x86/compat.c                    |  14 +-
xen/arch/x86/cpu/vpmu.c                  |   1 +
xen/arch/x86/domain.c                    |  11 +-
xen/arch/x86/domctl.c                    |   4 +-
xen/arch/x86/hvm/hypercall.c             | 176 ++-----------
xen/arch/x86/hypercall.c                 |  59 -----
xen/arch/x86/mm.c                        |   1 -
xen/arch/x86/mm/paging.c                 |   3 +-
xen/arch/x86/platform_hypercall.c        |   1 +
xen/arch/x86/pv/callback.c               |  20 +-
xen/arch/x86/pv/emul-priv-op.c           |   2 +-
xen/arch/x86/pv/hypercall.c              | 192 ++------------
xen/arch/x86/pv/iret.c                   |   5 +-
xen/arch/x86/pv/misc-hypercalls.c        |  14 +-
xen/arch/x86/pv/shim.c                   |  54 ++--
xen/arch/x86/traps.c                     |   2 +-
xen/arch/x86/x86_64/compat.c             |   1 -
xen/arch/x86/x86_64/compat/mm.c          |   1 +
xen/arch/x86/x86_64/domain.c             |  16 +-
xen/arch/x86/x86_64/mm.c                 |   2 -
xen/arch/x86/x86_64/platform_hypercall.c |   3 +-
xen/common/argo.c                        |  12 +-
xen/common/compat/domain.c               |  14 +-
xen/common/compat/grant_table.c          |   1 +
xen/common/compat/multicall.c            |   2 +-
xen/common/domain.c                      |  11 +-
xen/common/event_channel.c               |  10 +
xen/common/grant_table.c                 |  10 +
xen/common/kexec.c                       |   6 +-
xen/common/multicall.c                   |   2 +-
xen/common/trace.c                       |   2 +-
xen/include/asm-arm/hypercall.h          |   7 +-
xen/include/asm-x86/hypercall.h          | 204 ++++-----------
xen/include/asm-x86/paging.h             |   3 -
xen/include/asm-x86/pv/shim.h            |   3 +
xen/include/hypercall-defs.c             | 285 +++++++++++++++++++++
xen/include/xen/hypercall.h              | 181 +-------------
xen/include/xen/perfc_defn.h             |   1 -
xen/scripts/gen_hypercall.awk            | 306 +++++++++++++++++++++++
47 files changed, 870 insertions(+), 937 deletions(-)
create mode 100644 xen/include/hypercall-defs.c
create mode 100644 xen/scripts/gen_hypercall.awk
[PATCH v2 00/15] xen: drop hypercall function tables
Posted by Juergen Gross 2 years, 5 months ago
In order to avoid indirect function calls on the hypercall path as
much as possible this series is removing the hypercall function tables
and is replacing the hypercall handler calls via the function array
by automatically generated call macros.

Another by-product of generating the call macros is the automatic
generating of the hypercall handler prototypes from the same data base
which is used to generate the macros.

This has the additional advantage of using type safe calls of the
handlers and to ensure related handler (e.g. PV and HVM ones) share
the same prototypes.

A very brief performance test (parallel build of the Xen hypervisor
in a 6 vcpu guest) showed a very slim improvement (less than 1%) of
the performance with the patches applied. The test was performed using
a PV and a PVH guest.

Changes in V2:
- new patches 6, 14, 15
- patch 7: support hypercall priorities for faster code
- comments addressed

Juergen Gross (15):
  xen: limit number of hypercall parameters to 5
  xen: move do_vcpu_op() to arch specific code
  xen: harmonize return types of hypercall handlers
  xen/x86: modify hvm_memory_op() prototype
  xen: don't include asm/hypercall.h from C sources
  add .gitignore entries for *.[is] below xen
  xen: generate hypercall interface related code
  xen: use generated prototypes for hypercall handlers
  x86/pv-shim: don't modify hypercall table
  xen/x86: don't use hypercall table for calling compat hypercalls
  xen/x86: call hypercall handlers via generated macro
  xen/arm: call hypercall handlers via generated macro
  xen/x86: add hypercall performance counters for hvm, correct pv
  xen: drop calls_to_multicall performance counter
  tools/xenperf: update hypercall names

 .gitignore                               |   5 +-
 tools/misc/xenperf.c                     |   5 +
 xen/Makefile                             |  10 +
 xen/arch/arm/domain.c                    |  15 +-
 xen/arch/arm/hvm.c                       |   3 +-
 xen/arch/arm/physdev.c                   |   2 +-
 xen/arch/arm/platform_hypercall.c        |   1 +
 xen/arch/arm/traps.c                     | 124 ++-------
 xen/arch/x86/compat.c                    |  14 +-
 xen/arch/x86/cpu/vpmu.c                  |   1 +
 xen/arch/x86/domain.c                    |  11 +-
 xen/arch/x86/domctl.c                    |   4 +-
 xen/arch/x86/hvm/hypercall.c             | 176 ++-----------
 xen/arch/x86/hypercall.c                 |  59 -----
 xen/arch/x86/mm.c                        |   1 -
 xen/arch/x86/mm/paging.c                 |   3 +-
 xen/arch/x86/platform_hypercall.c        |   1 +
 xen/arch/x86/pv/callback.c               |  20 +-
 xen/arch/x86/pv/emul-priv-op.c           |   2 +-
 xen/arch/x86/pv/hypercall.c              | 192 ++------------
 xen/arch/x86/pv/iret.c                   |   5 +-
 xen/arch/x86/pv/misc-hypercalls.c        |  14 +-
 xen/arch/x86/pv/shim.c                   |  54 ++--
 xen/arch/x86/traps.c                     |   2 +-
 xen/arch/x86/x86_64/compat.c             |   1 -
 xen/arch/x86/x86_64/compat/mm.c          |   1 +
 xen/arch/x86/x86_64/domain.c             |  16 +-
 xen/arch/x86/x86_64/mm.c                 |   2 -
 xen/arch/x86/x86_64/platform_hypercall.c |   3 +-
 xen/common/argo.c                        |  12 +-
 xen/common/compat/domain.c               |  14 +-
 xen/common/compat/grant_table.c          |   1 +
 xen/common/compat/multicall.c            |   2 +-
 xen/common/domain.c                      |  11 +-
 xen/common/event_channel.c               |  10 +
 xen/common/grant_table.c                 |  10 +
 xen/common/kexec.c                       |   6 +-
 xen/common/multicall.c                   |   2 +-
 xen/common/trace.c                       |   2 +-
 xen/include/asm-arm/hypercall.h          |   7 +-
 xen/include/asm-x86/hypercall.h          | 204 ++++-----------
 xen/include/asm-x86/paging.h             |   3 -
 xen/include/asm-x86/pv/shim.h            |   3 +
 xen/include/hypercall-defs.c             | 285 +++++++++++++++++++++
 xen/include/xen/hypercall.h              | 181 +-------------
 xen/include/xen/perfc_defn.h             |   1 -
 xen/scripts/gen_hypercall.awk            | 306 +++++++++++++++++++++++
 47 files changed, 870 insertions(+), 937 deletions(-)
 create mode 100644 xen/include/hypercall-defs.c
 create mode 100644 xen/scripts/gen_hypercall.awk

-- 
2.26.2


Re: [PATCH v2 00/15] xen: drop hypercall function tables
Posted by Michal Orzel 2 years, 5 months ago
Hi Juergen,

On 01.11.2021 16:20, Juergen Gross wrote:
> In order to avoid indirect function calls on the hypercall path as
> much as possible this series is removing the hypercall function tables
> and is replacing the hypercall handler calls via the function array
> by automatically generated call macros.
> 
> Another by-product of generating the call macros is the automatic
> generating of the hypercall handler prototypes from the same data base
> which is used to generate the macros.
> 
> This has the additional advantage of using type safe calls of the
> handlers and to ensure related handler (e.g. PV and HVM ones) share
> the same prototypes.
> 
> A very brief performance test (parallel build of the Xen hypervisor
> in a 6 vcpu guest) showed a very slim improvement (less than 1%) of
> the performance with the patches applied. The test was performed using
> a PV and a PVH guest.
> 
> Changes in V2:
> - new patches 6, 14, 15
> - patch 7: support hypercall priorities for faster code
> - comments addressed

For the ARM part:
Apart from the issue I found and reported in patch 7/15, the build was successful.
I tested the following basics also successfully:
-booting a dom0
-booting domUs
-networking between guest and dom0 using NAT
-stressing hypercall xen_version

so,

Tested-by: Michal Orzel <michal.orzel@arm.com>

Cheers,
Michal

Re: [PATCH v2 00/15] xen: drop hypercall function tables
Posted by Juergen Gross 2 years, 5 months ago
On 03.11.21 12:35, Michal Orzel wrote:
> Hi Juergen,
> 
> On 01.11.2021 16:20, Juergen Gross wrote:
>> In order to avoid indirect function calls on the hypercall path as
>> much as possible this series is removing the hypercall function tables
>> and is replacing the hypercall handler calls via the function array
>> by automatically generated call macros.
>>
>> Another by-product of generating the call macros is the automatic
>> generating of the hypercall handler prototypes from the same data base
>> which is used to generate the macros.
>>
>> This has the additional advantage of using type safe calls of the
>> handlers and to ensure related handler (e.g. PV and HVM ones) share
>> the same prototypes.
>>
>> A very brief performance test (parallel build of the Xen hypervisor
>> in a 6 vcpu guest) showed a very slim improvement (less than 1%) of
>> the performance with the patches applied. The test was performed using
>> a PV and a PVH guest.
>>
>> Changes in V2:
>> - new patches 6, 14, 15
>> - patch 7: support hypercall priorities for faster code
>> - comments addressed
> 
> For the ARM part:
> Apart from the issue I found and reported in patch 7/15, the build was successful.
> I tested the following basics also successfully:
> -booting a dom0
> -booting domUs
> -networking between guest and dom0 using NAT
> -stressing hypercall xen_version
> 
> so,
> 
> Tested-by: Michal Orzel <michal.orzel@arm.com>

Thanks, much appreciated!


Juergen

Re: [PATCH v2 00/15] xen: drop hypercall function tables
Posted by Andrew Cooper 2 years, 5 months ago
On 01/11/2021 15:20, Juergen Gross wrote:
> In order to avoid indirect function calls on the hypercall path as
> much as possible this series is removing the hypercall function tables
> and is replacing the hypercall handler calls via the function array
> by automatically generated call macros.
>
> Another by-product of generating the call macros is the automatic
> generating of the hypercall handler prototypes from the same data base
> which is used to generate the macros.
>
> This has the additional advantage of using type safe calls of the
> handlers and to ensure related handler (e.g. PV and HVM ones) share
> the same prototypes.
>
> A very brief performance test (parallel build of the Xen hypervisor
> in a 6 vcpu guest) showed a very slim improvement (less than 1%) of
> the performance with the patches applied. The test was performed using
> a PV and a PVH guest.
>
> Changes in V2:
> - new patches 6, 14, 15
> - patch 7: support hypercall priorities for faster code
> - comments addressed
>
> Juergen Gross (15):
>   xen: limit number of hypercall parameters to 5
>   xen: move do_vcpu_op() to arch specific code
>   xen: harmonize return types of hypercall handlers
>   xen/x86: modify hvm_memory_op() prototype
>   xen: don't include asm/hypercall.h from C sources
>   add .gitignore entries for *.[is] below xen
>   xen: generate hypercall interface related code
>   xen: use generated prototypes for hypercall handlers
>   x86/pv-shim: don't modify hypercall table
>   xen/x86: don't use hypercall table for calling compat hypercalls
>   xen/x86: call hypercall handlers via generated macro
>   xen/arm: call hypercall handlers via generated macro
>   xen/x86: add hypercall performance counters for hvm, correct pv
>   xen: drop calls_to_multicall performance counter
>   tools/xenperf: update hypercall names

https://gitlab.com/xen-project/people/andyhhp/xen/-/jobs/1752237172

Something here causes

hypercall.c: In function 'hvm_hypercall':
hypercall.c:174:23: error: unused variable 'r8' [-Werror=unused-variable]
  174 |         unsigned long r8 = regs->r8;
      |                       ^~
hypercall.c:190:22: error: unused variable 'edi' [-Werror=unused-variable]
  190 |         unsigned int edi = regs->edi;
      |                      ^~~
cc1: all warnings being treated as errors

I suspect it will be "call hypercall handlers via generated macro", but
I haven't investigated further.

~Andrew

Re: [PATCH v2 00/15] xen: drop hypercall function tables
Posted by Jan Beulich 2 years, 5 months ago
On 05.11.2021 15:18, Andrew Cooper wrote:
> On 01/11/2021 15:20, Juergen Gross wrote:
>> In order to avoid indirect function calls on the hypercall path as
>> much as possible this series is removing the hypercall function tables
>> and is replacing the hypercall handler calls via the function array
>> by automatically generated call macros.
>>
>> Another by-product of generating the call macros is the automatic
>> generating of the hypercall handler prototypes from the same data base
>> which is used to generate the macros.
>>
>> This has the additional advantage of using type safe calls of the
>> handlers and to ensure related handler (e.g. PV and HVM ones) share
>> the same prototypes.
>>
>> A very brief performance test (parallel build of the Xen hypervisor
>> in a 6 vcpu guest) showed a very slim improvement (less than 1%) of
>> the performance with the patches applied. The test was performed using
>> a PV and a PVH guest.
>>
>> Changes in V2:
>> - new patches 6, 14, 15
>> - patch 7: support hypercall priorities for faster code
>> - comments addressed
>>
>> Juergen Gross (15):
>>   xen: limit number of hypercall parameters to 5
>>   xen: move do_vcpu_op() to arch specific code
>>   xen: harmonize return types of hypercall handlers
>>   xen/x86: modify hvm_memory_op() prototype
>>   xen: don't include asm/hypercall.h from C sources
>>   add .gitignore entries for *.[is] below xen
>>   xen: generate hypercall interface related code
>>   xen: use generated prototypes for hypercall handlers
>>   x86/pv-shim: don't modify hypercall table
>>   xen/x86: don't use hypercall table for calling compat hypercalls
>>   xen/x86: call hypercall handlers via generated macro
>>   xen/arm: call hypercall handlers via generated macro
>>   xen/x86: add hypercall performance counters for hvm, correct pv
>>   xen: drop calls_to_multicall performance counter
>>   tools/xenperf: update hypercall names
> 
> https://gitlab.com/xen-project/people/andyhhp/xen/-/jobs/1752237172
> 
> Something here causes
> 
> hypercall.c: In function 'hvm_hypercall':
> hypercall.c:174:23: error: unused variable 'r8' [-Werror=unused-variable]
>   174 |         unsigned long r8 = regs->r8;
>       |                       ^~
> hypercall.c:190:22: error: unused variable 'edi' [-Werror=unused-variable]
>   190 |         unsigned int edi = regs->edi;
>       |                      ^~~
> cc1: all warnings being treated as errors
> 
> I suspect it will be "call hypercall handlers via generated macro", but
> I haven't investigated further.

I suspect that's a randconfig build with !HYPFS + !ARGO. The hypfs and
argo hypercalls are the only ones with 5 parameters that HVM wires up.
A similar issue ought to exist in PV hypercall handling though, as the
compat form of update_va_mapping_otherdomain is the only other one
taking 5 arguments.

As to possible solutions - besides maybe adding (void)r8 to the -ENOSYS
path, I was wondering anyway in how far the local variables are still
warranted to retain.

Jan


Re: [PATCH v2 00/15] xen: drop hypercall function tables
Posted by Juergen Gross 2 years, 5 months ago
On 08.11.21 09:36, Jan Beulich wrote:
> On 05.11.2021 15:18, Andrew Cooper wrote:
>> On 01/11/2021 15:20, Juergen Gross wrote:
>>> In order to avoid indirect function calls on the hypercall path as
>>> much as possible this series is removing the hypercall function tables
>>> and is replacing the hypercall handler calls via the function array
>>> by automatically generated call macros.
>>>
>>> Another by-product of generating the call macros is the automatic
>>> generating of the hypercall handler prototypes from the same data base
>>> which is used to generate the macros.
>>>
>>> This has the additional advantage of using type safe calls of the
>>> handlers and to ensure related handler (e.g. PV and HVM ones) share
>>> the same prototypes.
>>>
>>> A very brief performance test (parallel build of the Xen hypervisor
>>> in a 6 vcpu guest) showed a very slim improvement (less than 1%) of
>>> the performance with the patches applied. The test was performed using
>>> a PV and a PVH guest.
>>>
>>> Changes in V2:
>>> - new patches 6, 14, 15
>>> - patch 7: support hypercall priorities for faster code
>>> - comments addressed
>>>
>>> Juergen Gross (15):
>>>    xen: limit number of hypercall parameters to 5
>>>    xen: move do_vcpu_op() to arch specific code
>>>    xen: harmonize return types of hypercall handlers
>>>    xen/x86: modify hvm_memory_op() prototype
>>>    xen: don't include asm/hypercall.h from C sources
>>>    add .gitignore entries for *.[is] below xen
>>>    xen: generate hypercall interface related code
>>>    xen: use generated prototypes for hypercall handlers
>>>    x86/pv-shim: don't modify hypercall table
>>>    xen/x86: don't use hypercall table for calling compat hypercalls
>>>    xen/x86: call hypercall handlers via generated macro
>>>    xen/arm: call hypercall handlers via generated macro
>>>    xen/x86: add hypercall performance counters for hvm, correct pv
>>>    xen: drop calls_to_multicall performance counter
>>>    tools/xenperf: update hypercall names
>>
>> https://gitlab.com/xen-project/people/andyhhp/xen/-/jobs/1752237172
>>
>> Something here causes
>>
>> hypercall.c: In function 'hvm_hypercall':
>> hypercall.c:174:23: error: unused variable 'r8' [-Werror=unused-variable]
>>    174 |         unsigned long r8 = regs->r8;
>>        |                       ^~
>> hypercall.c:190:22: error: unused variable 'edi' [-Werror=unused-variable]
>>    190 |         unsigned int edi = regs->edi;
>>        |                      ^~~
>> cc1: all warnings being treated as errors
>>
>> I suspect it will be "call hypercall handlers via generated macro", but
>> I haven't investigated further.
> 
> I suspect that's a randconfig build with !HYPFS + !ARGO. The hypfs and
> argo hypercalls are the only ones with 5 parameters that HVM wires up.
> A similar issue ought to exist in PV hypercall handling though, as the
> compat form of update_va_mapping_otherdomain is the only other one
> taking 5 arguments.
> 
> As to possible solutions - besides maybe adding (void)r8 to the -ENOSYS
> path, I was wondering anyway in how far the local variables are still
> warranted to retain.

I think dropping those local variables is the easiest solution. And I
think this applies to the PV variant, too.


Juergen