[PATCH v11 00/30] x86/resctrl: Move the resctrl filesystem code to /fs/resctrl

James Morse posted 30 patches 9 months ago
There is a newer version of this series
Documentation/arch/x86/index.rst              |    1 -
Documentation/filesystems/index.rst           |    1 +
.../{arch/x86 => filesystems}/resctrl.rst     |    6 +-
MAINTAINERS                                   |    5 +-
arch/Kconfig                                  |    8 +
arch/x86/Kconfig                              |   11 +-
arch/x86/include/asm/resctrl.h                |   19 +-
arch/x86/kernel/cpu/resctrl/Makefile          |    2 +
arch/x86/kernel/cpu/resctrl/core.c            |   31 +-
arch/x86/kernel/cpu/resctrl/ctrlmondata.c     |  635 ---
arch/x86/kernel/cpu/resctrl/internal.h        |  399 +-
arch/x86/kernel/cpu/resctrl/monitor.c         |  918 +---
arch/x86/kernel/cpu/resctrl/pseudo_lock.c     | 1092 +----
.../resctrl/{trace.h => pseudo_lock_trace.h}  |   26 +-
arch/x86/kernel/cpu/resctrl/rdtgroup.c        | 4164 +---------------
arch/x86/kernel/process_32.c                  |    2 +-
arch/x86/kernel/process_64.c                  |    2 +-
fs/Kconfig                                    |    1 +
fs/Makefile                                   |    1 +
fs/resctrl/Kconfig                            |   39 +
fs/resctrl/Makefile                           |    6 +
fs/resctrl/ctrlmondata.c                      |  661 +++
fs/resctrl/internal.h                         |  426 ++
fs/resctrl/monitor.c                          |  929 ++++
fs/resctrl/monitor_trace.h                    |   33 +
fs/resctrl/pseudo_lock.c                      | 1105 +++++
fs/resctrl/rdtgroup.c                         | 4353 +++++++++++++++++
include/linux/cpumask.h                       |   75 +-
include/linux/find.h                          |   25 +
include/linux/resctrl.h                       |   36 +-
include/linux/resctrl_types.h                 |   16 +-
lib/find_bit.c                                |   11 +
32 files changed, 7772 insertions(+), 7267 deletions(-)
rename Documentation/{arch/x86 => filesystems}/resctrl.rst (99%)
rename arch/x86/kernel/cpu/resctrl/{trace.h => pseudo_lock_trace.h} (56%)
create mode 100644 fs/resctrl/Kconfig
create mode 100644 fs/resctrl/Makefile
create mode 100644 fs/resctrl/ctrlmondata.c
create mode 100644 fs/resctrl/internal.h
create mode 100644 fs/resctrl/monitor.c
create mode 100644 fs/resctrl/monitor_trace.h
create mode 100644 fs/resctrl/pseudo_lock.c
create mode 100644 fs/resctrl/rdtgroup.c
[PATCH v11 00/30] x86/resctrl: Move the resctrl filesystem code to /fs/resctrl
Posted by James Morse 9 months ago
Changes since v10:
 * Changes the online domains check in patch 7 to walk control and monitor domains
   separately.

---

Patches 24-29 should be squashed together when merged, taking the commit message
of patch 25. It probably makes sense to drop the tags at that point as patch 25 is
generated by a script, and impossible to review. They are posted like this to allow
folk to re-generate patch 25, then review the differences on top. Not squashing them
together would expose a ftrace build warning during bisect. (but who does that!)

The result  should look like this:
git://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git mpam/move_to_fs/v11_final

I can also post the 'final' version to be picked up if that is less work.


This series is based on rc5, and can be retrieved from:
git://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git mpam/move_to_fs/v11

With the exception of invalid configurations for the configurable-events, there
should be no changes in behaviour caused by this series. It is now possible for
throttle_mode to report 'undefined', but no known platform will do this.
resctrl_exit() is now something that can be called, but x86 doesn't do this.

The driving pattern is to make things like struct rdtgroup private to resctrl.
Features like pseudo-lock aren't going to work on arm64, the ability to disable
it at compile time is added.

After this, I can start posting the MPAM driver to make use of resctrl on arm64.
(What's MPAM? See the cover letter of the first series. [1])

As ever - bugs welcome,
Thanks,

James

[v10] https://lore.kernel.org/all/20250508171858.9197-1-james.morse@arm.com
[v9] https://lore.kernel.org/all/20250425173809.5529-1-james.morse@arm.com
[v8] https://lore.kernel.org/all/20250411164229.23413-1-james.morse@arm.com
[v7] https://lore.kernel.org/all/20250228195913.24895-1-james.morse@arm.com/
[v6] https://lore.kernel.org/lkml/20250207181823.6378-1-james.morse@arm.com/
[v5] https://lore.kernel.org/r/20241004180347.19985-1-james.morse@arm.com
[v4] https://lore.kernel.org/all/20240802172853.22529-1-james.morse@arm.com/
[v3] https://lore.kernel.org/r/20240614150033.10454-1-james.morse@arm.com
[v2] https://lore.kernel.org/r/20240426150537.8094-1-Dave.Martin@arm.com
[v1] https://lore.kernel.org/r/20240321165106.31602-1-james.morse@arm.com
[1] https://lore.kernel.org/lkml/20201030161120.227225-1-james.morse@arm.com/



Amit Singh Tomar (1):
  x86/resctrl: Remove the limit on the number of CLOSID

Dave Martin (3):
  x86/resctrl: Squelch whitespace anomalies in resctrl core code
  x86/resctrl: Prefer alloc(sizeof(*foo)) idiom in rdt_init_fs_context()
  x86/resctrl: Relax some asm #includes

James Morse (22):
  x86/resctrl: Rename resctrl_sched_in() to begin with "resctrl_arch_"
  x86/resctrl: Check all domains are offline in resctrl_exit()
  x86/resctrl: resctrl_exit() teardown resctrl but leave the mount point
  x86/resctrl: Drop __init/__exit on assorted symbols
  x86/resctrl: Move is_mba_sc() out of core.c
  x86/resctrl: Add end-marker to the resctrl_event_id enum
  x86/resctrl: Expand the width of domid by replacing mon_data_bits
  x86/resctrl: Split trace.h
  x86/resctrl: Add 'resctrl' to the title of the resctrl documentation
  fs/resctrl: Add boiler plate for external resctrl code
  x86/resctrl: Move the filesystem bits to headers visible to fs/resctrl
  x86/resctrl: Move enum resctrl_event_id to resctrl.h
  x86/resctrl: Fix types in resctrl_arch_mon_ctx_{alloc,free}() stubs
  x86/resctrl: Move pseudo lock prototypes to include/linux/resctrl.h
  x86/resctrl: Always initialise rid field in rdt_resources_all[]
  x86/resctrl: Remove a newline to avoid confusing the code move script
  x86,fs/resctrl: Move the resctrl filesystem code to live in
    /fs/resctrl
  x86,fs/resctrl: Remove duplicated trace header files
  fs/resctrl: Remove unnecessary includes
  fs/resctrl: Change internal.h's header guard macros
  x86,fs/resctrl: Move resctrl.rst to live under
    Documentation/filesystems
  MAINTAINERS: Add reviewers for fs/resctrl

Yury Norov [NVIDIA] (4):
  cpumask: relax cpumask_any_but()
  find: add find_first_andnot_bit()
  cpumask: add cpumask_{first,next}_andnot() API
  x86/resctrl: Optimize cpumask_any_housekeeping()

 Documentation/arch/x86/index.rst              |    1 -
 Documentation/filesystems/index.rst           |    1 +
 .../{arch/x86 => filesystems}/resctrl.rst     |    6 +-
 MAINTAINERS                                   |    5 +-
 arch/Kconfig                                  |    8 +
 arch/x86/Kconfig                              |   11 +-
 arch/x86/include/asm/resctrl.h                |   19 +-
 arch/x86/kernel/cpu/resctrl/Makefile          |    2 +
 arch/x86/kernel/cpu/resctrl/core.c            |   31 +-
 arch/x86/kernel/cpu/resctrl/ctrlmondata.c     |  635 ---
 arch/x86/kernel/cpu/resctrl/internal.h        |  399 +-
 arch/x86/kernel/cpu/resctrl/monitor.c         |  918 +---
 arch/x86/kernel/cpu/resctrl/pseudo_lock.c     | 1092 +----
 .../resctrl/{trace.h => pseudo_lock_trace.h}  |   26 +-
 arch/x86/kernel/cpu/resctrl/rdtgroup.c        | 4164 +---------------
 arch/x86/kernel/process_32.c                  |    2 +-
 arch/x86/kernel/process_64.c                  |    2 +-
 fs/Kconfig                                    |    1 +
 fs/Makefile                                   |    1 +
 fs/resctrl/Kconfig                            |   39 +
 fs/resctrl/Makefile                           |    6 +
 fs/resctrl/ctrlmondata.c                      |  661 +++
 fs/resctrl/internal.h                         |  426 ++
 fs/resctrl/monitor.c                          |  929 ++++
 fs/resctrl/monitor_trace.h                    |   33 +
 fs/resctrl/pseudo_lock.c                      | 1105 +++++
 fs/resctrl/rdtgroup.c                         | 4353 +++++++++++++++++
 include/linux/cpumask.h                       |   75 +-
 include/linux/find.h                          |   25 +
 include/linux/resctrl.h                       |   36 +-
 include/linux/resctrl_types.h                 |   16 +-
 lib/find_bit.c                                |   11 +
 32 files changed, 7772 insertions(+), 7267 deletions(-)
 rename Documentation/{arch/x86 => filesystems}/resctrl.rst (99%)
 rename arch/x86/kernel/cpu/resctrl/{trace.h => pseudo_lock_trace.h} (56%)
 create mode 100644 fs/resctrl/Kconfig
 create mode 100644 fs/resctrl/Makefile
 create mode 100644 fs/resctrl/ctrlmondata.c
 create mode 100644 fs/resctrl/internal.h
 create mode 100644 fs/resctrl/monitor.c
 create mode 100644 fs/resctrl/monitor_trace.h
 create mode 100644 fs/resctrl/pseudo_lock.c
 create mode 100644 fs/resctrl/rdtgroup.c

-- 
2.39.5
Re: [PATCH v11 00/30] x86/resctrl: Move the resctrl filesystem code to /fs/resctrl
Posted by Reinette Chatre 8 months, 4 weeks ago
Hi James,

On 5/13/25 10:15 AM, James Morse wrote:
> Changes since v10:
>  * Changes the online domains check in patch 7 to walk control and monitor domains
>    separately.
> 
> ---
> 
> Patches 24-29 should be squashed together when merged, taking the commit message
> of patch 25. It probably makes sense to drop the tags at that point as patch 25 is
> generated by a script, and impossible to review. They are posted like this to allow
> folk to re-generate patch 25, then review the differences on top. Not squashing them
> together would expose a ftrace build warning during bisect. (but who does that!)
> 
> The result  should look like this:
> git://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git mpam/move_to_fs/v11_final

This branch does not exist.

Reinette
Re: [PATCH v11 00/30] x86/resctrl: Move the resctrl filesystem code to /fs/resctrl
Posted by Fenghua Yu 8 months, 4 weeks ago
Hi, Reinette,

On 5/13/25 14:19, Reinette Chatre wrote:
> Hi James,
>
> On 5/13/25 10:15 AM, James Morse wrote:
>> Changes since v10:
>>   * Changes the online domains check in patch 7 to walk control and monitor domains
>>     separately.
>>
>> ---
>>
>> Patches 24-29 should be squashed together when merged, taking the commit message
>> of patch 25. It probably makes sense to drop the tags at that point as patch 25 is
>> generated by a script, and impossible to review. They are posted like this to allow
>> folk to re-generate patch 25, then review the differences on top. Not squashing them
>> together would expose a ftrace build warning during bisect. (but who does that!)
>>
>> The result  should look like this:
>> git://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git mpam/move_to_fs/v11_final
> This branch does not exist.

This branch is existing now. I can clone, build, boot, and run some 
light tests on this branch. All work fine for me.

Thanks.

-Fenghua
Re: [PATCH v11 00/30] x86/resctrl: Move the resctrl filesystem code to /fs/resctrl
Posted by Ning, Hongyu 8 months, 3 weeks ago

On 5/14/2025 1:15 AM, James Morse wrote:
> Changes since v10:
>   * Changes the online domains check in patch 7 to walk control and monitor domains
>     separately.
> 
> ---
> 
> Patches 24-29 should be squashed together when merged, taking the commit message
> of patch 25. It probably makes sense to drop the tags at that point as patch 25 is
> generated by a script, and impossible to review. They are posted like this to allow
> folk to re-generate patch 25, then review the differences on top. Not squashing them
> together would expose a ftrace build warning during bisect. (but who does that!)
> 
> The result  should look like this:
> git://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git mpam/move_to_fs/v11_final
> 
> I can also post the 'final' version to be picked up if that is less work.
> 
> 
> This series is based on rc5, and can be retrieved from:
> git://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git mpam/move_to_fs/v11
> 

Hi James,

sorry for late jumping in.

I've tested this v11 patch series (also v10 from last week) based on 
Intel 5th Gen Xeon server platform, all basic resctrl functions work fine.

Tested-by: Hongyu Ning <hongyu.ning@linux.intel.com>

> With the exception of invalid configurations for the configurable-events, there
> should be no changes in behaviour caused by this series. It is now possible for
> throttle_mode to report 'undefined', but no known platform will do this.
> resctrl_exit() is now something that can be called, but x86 doesn't do this.
> 
> The driving pattern is to make things like struct rdtgroup private to resctrl.
> Features like pseudo-lock aren't going to work on arm64, the ability to disable
> it at compile time is added.
> 
> After this, I can start posting the MPAM driver to make use of resctrl on arm64.
> (What's MPAM? See the cover letter of the first series. [1])
> 
> As ever - bugs welcome,
> Thanks,
> 
> James
> 
> [v10] https://lore.kernel.org/all/20250508171858.9197-1-james.morse@arm.com
> [v9] https://lore.kernel.org/all/20250425173809.5529-1-james.morse@arm.com
> [v8] https://lore.kernel.org/all/20250411164229.23413-1-james.morse@arm.com
> [v7] https://lore.kernel.org/all/20250228195913.24895-1-james.morse@arm.com/
> [v6] https://lore.kernel.org/lkml/20250207181823.6378-1-james.morse@arm.com/
> [v5] https://lore.kernel.org/r/20241004180347.19985-1-james.morse@arm.com
> [v4] https://lore.kernel.org/all/20240802172853.22529-1-james.morse@arm.com/
> [v3] https://lore.kernel.org/r/20240614150033.10454-1-james.morse@arm.com
> [v2] https://lore.kernel.org/r/20240426150537.8094-1-Dave.Martin@arm.com
> [v1] https://lore.kernel.org/r/20240321165106.31602-1-james.morse@arm.com
> [1] https://lore.kernel.org/lkml/20201030161120.227225-1-james.morse@arm.com/
> 
> 
> 
> Amit Singh Tomar (1):
>    x86/resctrl: Remove the limit on the number of CLOSID
> 
> Dave Martin (3):
>    x86/resctrl: Squelch whitespace anomalies in resctrl core code
>    x86/resctrl: Prefer alloc(sizeof(*foo)) idiom in rdt_init_fs_context()
>    x86/resctrl: Relax some asm #includes
> 
> James Morse (22):
>    x86/resctrl: Rename resctrl_sched_in() to begin with "resctrl_arch_"
>    x86/resctrl: Check all domains are offline in resctrl_exit()
>    x86/resctrl: resctrl_exit() teardown resctrl but leave the mount point
>    x86/resctrl: Drop __init/__exit on assorted symbols
>    x86/resctrl: Move is_mba_sc() out of core.c
>    x86/resctrl: Add end-marker to the resctrl_event_id enum
>    x86/resctrl: Expand the width of domid by replacing mon_data_bits
>    x86/resctrl: Split trace.h
>    x86/resctrl: Add 'resctrl' to the title of the resctrl documentation
>    fs/resctrl: Add boiler plate for external resctrl code
>    x86/resctrl: Move the filesystem bits to headers visible to fs/resctrl
>    x86/resctrl: Move enum resctrl_event_id to resctrl.h
>    x86/resctrl: Fix types in resctrl_arch_mon_ctx_{alloc,free}() stubs
>    x86/resctrl: Move pseudo lock prototypes to include/linux/resctrl.h
>    x86/resctrl: Always initialise rid field in rdt_resources_all[]
>    x86/resctrl: Remove a newline to avoid confusing the code move script
>    x86,fs/resctrl: Move the resctrl filesystem code to live in
>      /fs/resctrl
>    x86,fs/resctrl: Remove duplicated trace header files
>    fs/resctrl: Remove unnecessary includes
>    fs/resctrl: Change internal.h's header guard macros
>    x86,fs/resctrl: Move resctrl.rst to live under
>      Documentation/filesystems
>    MAINTAINERS: Add reviewers for fs/resctrl
> 
> Yury Norov [NVIDIA] (4):
>    cpumask: relax cpumask_any_but()
>    find: add find_first_andnot_bit()
>    cpumask: add cpumask_{first,next}_andnot() API
>    x86/resctrl: Optimize cpumask_any_housekeeping()
> 
>   Documentation/arch/x86/index.rst              |    1 -
>   Documentation/filesystems/index.rst           |    1 +
>   .../{arch/x86 => filesystems}/resctrl.rst     |    6 +-
>   MAINTAINERS                                   |    5 +-
>   arch/Kconfig                                  |    8 +
>   arch/x86/Kconfig                              |   11 +-
>   arch/x86/include/asm/resctrl.h                |   19 +-
>   arch/x86/kernel/cpu/resctrl/Makefile          |    2 +
>   arch/x86/kernel/cpu/resctrl/core.c            |   31 +-
>   arch/x86/kernel/cpu/resctrl/ctrlmondata.c     |  635 ---
>   arch/x86/kernel/cpu/resctrl/internal.h        |  399 +-
>   arch/x86/kernel/cpu/resctrl/monitor.c         |  918 +---
>   arch/x86/kernel/cpu/resctrl/pseudo_lock.c     | 1092 +----
>   .../resctrl/{trace.h => pseudo_lock_trace.h}  |   26 +-
>   arch/x86/kernel/cpu/resctrl/rdtgroup.c        | 4164 +---------------
>   arch/x86/kernel/process_32.c                  |    2 +-
>   arch/x86/kernel/process_64.c                  |    2 +-
>   fs/Kconfig                                    |    1 +
>   fs/Makefile                                   |    1 +
>   fs/resctrl/Kconfig                            |   39 +
>   fs/resctrl/Makefile                           |    6 +
>   fs/resctrl/ctrlmondata.c                      |  661 +++
>   fs/resctrl/internal.h                         |  426 ++
>   fs/resctrl/monitor.c                          |  929 ++++
>   fs/resctrl/monitor_trace.h                    |   33 +
>   fs/resctrl/pseudo_lock.c                      | 1105 +++++
>   fs/resctrl/rdtgroup.c                         | 4353 +++++++++++++++++
>   include/linux/cpumask.h                       |   75 +-
>   include/linux/find.h                          |   25 +
>   include/linux/resctrl.h                       |   36 +-
>   include/linux/resctrl_types.h                 |   16 +-
>   lib/find_bit.c                                |   11 +
>   32 files changed, 7772 insertions(+), 7267 deletions(-)
>   rename Documentation/{arch/x86 => filesystems}/resctrl.rst (99%)
>   rename arch/x86/kernel/cpu/resctrl/{trace.h => pseudo_lock_trace.h} (56%)
>   create mode 100644 fs/resctrl/Kconfig
>   create mode 100644 fs/resctrl/Makefile
>   create mode 100644 fs/resctrl/ctrlmondata.c
>   create mode 100644 fs/resctrl/internal.h
>   create mode 100644 fs/resctrl/monitor.c
>   create mode 100644 fs/resctrl/monitor_trace.h
>   create mode 100644 fs/resctrl/pseudo_lock.c
>   create mode 100644 fs/resctrl/rdtgroup.c
>
Re: [PATCH v11 00/30] x86/resctrl: Move the resctrl filesystem code to /fs/resctrl
Posted by James Morse 8 months, 3 weeks ago
Hello!

On 16/05/2025 06:53, Ning, Hongyu wrote:
> On 5/14/2025 1:15 AM, James Morse wrote:
>> Changes since v10:
>>   * Changes the online domains check in patch 7 to walk control and monitor domains
>>     separately.
>>
>> ---
>>
>> Patches 24-29 should be squashed together when merged, taking the commit message
>> of patch 25. It probably makes sense to drop the tags at that point as patch 25 is
>> generated by a script, and impossible to review. They are posted like this to allow
>> folk to re-generate patch 25, then review the differences on top. Not squashing them
>> together would expose a ftrace build warning during bisect. (but who does that!)
>>
>> The result  should look like this:
>> git://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git mpam/move_to_fs/v11_final
>>
>> I can also post the 'final' version to be picked up if that is less work.
>>
>>
>> This series is based on rc5, and can be retrieved from:
>> git://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git mpam/move_to_fs/v11
>>
> 
> Hi James,
> 
> sorry for late jumping in.
> 
> I've tested this v11 patch series (also v10 from last week) based on Intel 5th Gen Xeon
> server platform, all basic resctrl functions work fine.
> 
> Tested-by: Hongyu Ning <hongyu.ning@linux.intel.com>

Ah - just missed the v12. Thanks for taking it for a spin though!

James
Re: [PATCH v11 00/30] x86/resctrl: Move the resctrl filesystem code to /fs/resctrl
Posted by Luck, Tony 8 months, 4 weeks ago
On Tue, May 13, 2025 at 05:15:17PM +0000, James Morse wrote:
> Changes since v10:
>  * Changes the online domains check in patch 7 to walk control and monitor domains
>    separately.

Built, booted, and lightly tested on regular and SNC systems. All looks
fine to me.

Tested-by: Tony Luck <tony.luck@intel.com>

-Tony
Re: [PATCH v11 00/30] x86/resctrl: Move the resctrl filesystem code to /fs/resctrl
Posted by James Morse 8 months, 3 weeks ago
Hi Tony,

On 13/05/2025 19:32, Luck, Tony wrote:
> On Tue, May 13, 2025 at 05:15:17PM +0000, James Morse wrote:
>> Changes since v10:
>>  * Changes the online domains check in patch 7 to walk control and monitor domains
>>    separately.
> 
> Built, booted, and lightly tested on regular and SNC systems. All looks
> fine to me.
> 
> Tested-by: Tony Luck <tony.luck@intel.com>


Thanks for testing that!

James
Re: [PATCH v11 00/30] x86/resctrl: Move the resctrl filesystem code to /fs/resctrl
Posted by Reinette Chatre 8 months, 4 weeks ago
Dear Boris and x86 maintainers,

On 5/13/25 10:15 AM, James Morse wrote:
> Patches 24-29 should be squashed together when merged, taking the commit message
> of patch 25. It probably makes sense to drop the tags at that point as patch 25 is
> generated by a script, and impossible to review. They are posted like this to allow
> folk to re-generate patch 25, then review the differences on top. Not squashing them
> together would expose a ftrace build warning during bisect. (but who does that!)
> 
> The result  should look like this:
> git://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git mpam/move_to_fs/v11_final

(awaiting this branch's availability)

> 
> I can also post the 'final' version to be picked up if that is less work.
> 
> 
> This series is based on rc5, and can be retrieved from:
> git://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git mpam/move_to_fs/v11
> 
> With the exception of invalid configurations for the configurable-events, there
> should be no changes in behaviour caused by this series. It is now possible for
> throttle_mode to report 'undefined', but no known platform will do this.
> resctrl_exit() is now something that can be called, but x86 doesn't do this.
> 
> The driving pattern is to make things like struct rdtgroup private to resctrl.
> Features like pseudo-lock aren't going to work on arm64, the ability to disable
> it at compile time is added.
> 

Could you please consider this work for inclusion? Please provide guidance on
how to support inclusion of this work ... James mentions a couple of
options above.

Thank you very much.

Reinette
Re: [PATCH v11 00/30] x86/resctrl: Move the resctrl filesystem code to /fs/resctrl
Posted by Reinette Chatre 8 months, 4 weeks ago
Hi James,

On 5/13/25 10:15 AM, James Morse wrote:
> 
> I can also post the 'final' version to be picked up if that is less work.
> 

Could you please go ahead and post the "final" version? Boris confirmed
that this would work best with the tip processes and the series will be
considered if it can be ready soon.

Thank you very much.

Reinette