Patch series available in this git branch:
https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git br.build-system-xen-include-rework-v1

Hi,

This patch series is about 2 improvement. First one is to use $(if_changed, )
in "include/Makefile" to make the generation of the compat headers less verbose
and to have the command line part of the decision to rebuild the headers.
Second one is to replace one slow script by a much faster one, and save time
when generating the headers.

Thanks.

Anthony PERARD (4):
  build: xen/include: use if_changed
  build: set PERL
  build: replace get-fields.sh by a perl script
  build: remove auto.conf prerequisite from compat/xlat.h target

 xen/Makefile                 |   1 +
 xen/include/Makefile         | 106 ++++---
 xen/tools/compat-xlat-header | 539 +++++++++++++++++++++++++++++++++++
 xen/tools/get-fields.sh      | 528 ----------------------------------
 4 files changed, 614 insertions(+), 560 deletions(-)
 create mode 100755 xen/tools/compat-xlat-header
 delete mode 100644 xen/tools/get-fields.sh

-- 
Anthony PERARD

On 01/06/2022 17:59, Anthony PERARD wrote:
> Patch series available in this git branch:
> https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git br.build-system-xen-include-rework-v1
>
> Hi,
>
> This patch series is about 2 improvement. First one is to use $(if_changed, )
> in "include/Makefile" to make the generation of the compat headers less verbose
> and to have the command line part of the decision to rebuild the headers.
> Second one is to replace one slow script by a much faster one, and save time
> when generating the headers.
>
> Thanks.
>
> Anthony PERARD (4):
>   build: xen/include: use if_changed
>   build: set PERL
>   build: replace get-fields.sh by a perl script
>   build: remove auto.conf prerequisite from compat/xlat.h target
>
>  xen/Makefile                 |   1 +
>  xen/include/Makefile         | 106 ++++---
>  xen/tools/compat-xlat-header | 539 +++++++++++++++++++++++++++++++++++
>  xen/tools/get-fields.sh      | 528 ----------------------------------

Excellent.  I was planning to ask you about this.  (I also need to
refreshing my half series cleaning up other bits of the build.)

One trivial observation is that it would probably be nicer to name the
script with a .pl extension.

Any numbers on what the speedup in patch 3 is?

Are the generated compat headers identical before and after this
series?  If yes, I'm very tempted to ack and commit it straight away.

~Andrew

On Wed, Jun 01, 2022 at 05:17:36PM +0000, Andrew Cooper wrote:
> On 01/06/2022 17:59, Anthony PERARD wrote:
> > Patch series available in this git branch:
> > https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git br.build-system-xen-include-rework-v1
> >
> > Hi,
> >
> > This patch series is about 2 improvement. First one is to use $(if_changed, )
> > in "include/Makefile" to make the generation of the compat headers less verbose
> > and to have the command line part of the decision to rebuild the headers.
> > Second one is to replace one slow script by a much faster one, and save time
> > when generating the headers.
> >
> > Thanks.
> >
> > Anthony PERARD (4):
> >   build: xen/include: use if_changed
> >   build: set PERL
> >   build: replace get-fields.sh by a perl script
> >   build: remove auto.conf prerequisite from compat/xlat.h target
> >
> >  xen/Makefile                 |   1 +
> >  xen/include/Makefile         | 106 ++++---
> >  xen/tools/compat-xlat-header | 539 +++++++++++++++++++++++++++++++++++
> >  xen/tools/get-fields.sh      | 528 ----------------------------------
> 
> Excellent.  I was planning to ask you about this.  (I also need to
> refreshing my half series cleaning up other bits of the build.)
> 
> One trivial observation is that it would probably be nicer to name the
> script with a .pl extension.

Sound fine, I don't think it matter much here.

> Any numbers on what the speedup in patch 3 is?

Yes, on my machine when doing a full build, with `ccache` enabled, it
saves about 1.17 seconds (out of ~17s), and without ccache, it saves
about 2.0 seconds (out of ~37s).

Without ccache:

before:
    $ for i in `seq 4`; do time make -j$(nproc) -s O=build 2>/dev/null >/dev/null; rm -r build; done
    make --no-print-directory -j$(nproc) -s O=build > /dev/null  244.98s user 29.24s system 683% cpu 40.146 total
    make --no-print-directory -j$(nproc) -s O=build > /dev/null  47.05s user 11.50s system 332% cpu 17.610 total
    make --no-print-directory -j$(nproc) -s O=build > /dev/null  47.35s user 11.22s system 330% cpu 17.733 total
    make --no-print-directory -j$(nproc) -s O=build > /dev/null  47.31s user 11.23s system 333% cpu 17.577 total

after:
    $ for i in `seq 4`; do time make -j$(nproc) -s O=build 2>/dev/null>/dev/null; rm -r build; done
    make --no-print-directory -j$(nproc) -s O=build 2> /dev/null > /dev/null  237.28s user 25.97s system 667% cpu 39.456 total
    make --no-print-directory -j$(nproc) -s O=build 2> /dev/null > /dev/null  37.60s user 8.20s system 282% cpu 16.214 total
    make --no-print-directory -j$(nproc) -s O=build 2> /dev/null > /dev/null  37.95s user 8.67s system 279% cpu 16.651 total
    make --no-print-directory -j$(nproc) -s O=build 2> /dev/null > /dev/null  38.02s user 8.40s system 280% cpu 16.545 total

And without ccache:

before:
    $ for i in `seq 4`; do time make -j$(nproc) -s O=build 2>/dev/null>/dev/null; rm -r build; done
    make --no-print-directory -j$(nproc) -s O=build 2> /dev/null > /dev/null  206.37s user 22.19s system 640% cpu 35.695 total
    make --no-print-directory -j$(nproc) -s O=build 2> /dev/null > /dev/null  221.45s user 22.26s system 667% cpu 36.537 total
    make --no-print-directory -j$(nproc) -s O=build 2> /dev/null > /dev/null  233.95s user 23.80s system 686% cpu 37.518 total
    make --no-print-directory -j$(nproc) -s O=build 2> /dev/null > /dev/null  234.27s user 23.83s system 680% cpu 37.923 total

after:
    $ for i in `seq 4`; do time make -j$(nproc) -s O=build 2>/dev/null>/dev/null; rm -r build; done
    make --no-print-directory -j$(nproc) -s O=build 2> /dev/null > /dev/null  198.62s user 18.64s system 642% cpu 33.841 total
    make --no-print-directory -j$(nproc) -s O=build 2> /dev/null > /dev/null  202.91s user 19.46s system 655% cpu 33.912 total
    make --no-print-directory -j$(nproc) -s O=build 2> /dev/null > /dev/null  224.42s user 20.89s system 680% cpu 36.025 total
    make --no-print-directory -j$(nproc) -s O=build 2> /dev/null > /dev/null  222.93s user 21.29s system 683% cpu 35.708 total


> Are the generated compat headers identical before and after this
> series?  If yes, I'm very tempted to ack and commit it straight away.

Yes, the headers are identical. Hopefully, I've managed to check with
all compat headers enabled, since that depends on .config.

Cheers,

-- 
Anthony PERARD