The current policy for minimum supported versions of tools, compilers,
etc. is unsatisfactory: For many dependencies no minimum version is
specified.  For those where a version is stated, updating it is a
decision that has to be explicitly taken for that tool.

The result is persistent debates over what is good to support,
conducted in detail in the context of individual patches.

Decisions about support involve tradeoffs, often tradeoffs between the
interests of different people.  Currently we don't have anything
resembling a guideline.  The result is that the individual debates are
inconclusive; and also, this framework does not lead to good feelings
amongst participants.

I suggest instead that we adopt a date-based policy: we define a
maximum *age* of dependencies that we will support.

The existing documentation about actually known working versions
then becomes a practical consequence of that policy.

In this patch I propose a cutoff of 6 years.
Obviously there will be debate about the precise value.

It will also be necessary to make exceptions, and/or to make different
rules for different architectures.  In particular, new architectures,
new configurations, or new features, may need an absolute earliest
tooling date which is considerably less than the usual limit.

I have tried to transcribe the current compiler version info into this
format.  The dates in the exceptions are all more recent than my
suggested 6 year cutoff, so if this patch is applied to staging and
not applied retrospectively, they could be removed.

I'm not sure if this policy should be here in README (where the
version support was until now) or in SUPPORT.md.

Signed-off-by: Ian Jackson <iwj@xenproject.org>
---
 README | 38 +++++++++++++++++++++++++-------------
 1 file changed, 25 insertions(+), 13 deletions(-)

diff --git a/README b/README
index 562b808080..5859f8bbf4 100644
--- a/README
+++ b/README
@@ -35,19 +35,8 @@ Second, there are a number of prerequisites for building a Xen source
 release. Make sure you have all the following installed, either by
 visiting the project webpage or installing a pre-built package
 provided by your OS distributor:
-    * GNU Make v3.80 or later
-    * C compiler and linker:
-      - For x86:
-        - GCC 4.1.2_20070115 or later
-        - GNU Binutils 2.16.91.0.5 or later
-        or
-        - Clang/LLVM 3.5 or later
-      - For ARM 32-bit:
-        - GCC 4.9 or later
-        - GNU Binutils 2.24 or later
-      - For ARM 64-bit:
-        - GCC 5.1 or later
-        - GNU Binutils 2.24 or later
+    * GNU Make
+    * C compiler and linker (x86: GCC or CLang; ARM: GCC)
     * Development install of zlib (e.g., zlib-dev)
     * Development install of Python 2.6 or later (e.g., python-dev)
     * Development install of curses (e.g., libncurses-dev)
@@ -65,6 +54,29 @@ provided by your OS distributor:
     * GNU bison and GNU flex
     * ACPI ASL compiler (iasl)
 
+In general, tools and compilers no more than 6 years old are
+supported (measured from the release date of the Xen version).
+However:
+    * x86: CLang/LLVM earlier than 3.5 is unsppported
+    * ARM 64-bit, dependencies older than 2015-04-22 are unsupported
+    * ARM 32-bit, dependencies older than 2014-04-22 are unsupported
+
+FYI, we believe the following versions work with this version of Xen:
+    * GNU Make
+       - v3.80 or later
+    * C compiler and linker:
+      - For x86:
+        - GCC 4.1.2_20070115 or later
+        - GNU Binutils 2.16.91.0.5 or later
+        or
+        - Clang/LLVM 3.5 or later
+      - For ARM 32-bit:
+        - GCC 4.9 or later
+        - GNU Binutils 2.24 or later
+      - For ARM 64-bit:
+        - GCC 5.1 or later
+        - GNU Binutils 2.24 or later
+
 In addition to the above there are a number of optional build
 prerequisites. Omitting these will cause the related features to be
 disabled at compile time:
-- 
2.20.1

On Fri, Aug 13, 2021 at 12:37:27PM +0100, Ian Jackson wrote:
> The current policy for minimum supported versions of tools, compilers,
> etc. is unsatisfactory: For many dependencies no minimum version is
> specified.  For those where a version is stated, updating it is a
> decision that has to be explicitly taken for that tool.
> 
> The result is persistent debates over what is good to support,
> conducted in detail in the context of individual patches.
> 
> Decisions about support involve tradeoffs, often tradeoffs between the
> interests of different people.  Currently we don't have anything
> resembling a guideline.  The result is that the individual debates are
> inconclusive; and also, this framework does not lead to good feelings
> amongst participants.
> 
> I suggest instead that we adopt a date-based policy: we define a
> maximum *age* of dependencies that we will support.

I wonder about another approach: specify supported toolchain version(s)
based on environments we choose to care about. That would be things like
"Debian, including LTS (or even ELTS) one", "RHEL/CentOS until X...",
etc. Based on this, it's easy to derive what's the oldest version that
needs to be supported.
This would be also much friendlier for testing - a clear definition
what environments should be used (in gitlab-ci, I guess).

Thoughts?

> The existing documentation about actually known working versions
> then becomes a practical consequence of that policy.
> 
> In this patch I propose a cutoff of 6 years.
> Obviously there will be debate about the precise value.
> 
> It will also be necessary to make exceptions, and/or to make different
> rules for different architectures.  In particular, new architectures,
> new configurations, or new features, may need an absolute earliest
> tooling date which is considerably less than the usual limit.
> 
> I have tried to transcribe the current compiler version info into this
> format.  The dates in the exceptions are all more recent than my
> suggested 6 year cutoff, so if this patch is applied to staging and
> not applied retrospectively, they could be removed.
> 
> I'm not sure if this policy should be here in README (where the
> version support was until now) or in SUPPORT.md.

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab

> On Aug 18, 2021, at 12:16 PM, Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com> wrote:
> 
> On Fri, Aug 13, 2021 at 12:37:27PM +0100, Ian Jackson wrote:
>> The current policy for minimum supported versions of tools, compilers,
>> etc. is unsatisfactory: For many dependencies no minimum version is
>> specified.  For those where a version is stated, updating it is a
>> decision that has to be explicitly taken for that tool.
>> 
>> The result is persistent debates over what is good to support,
>> conducted in detail in the context of individual patches.
>> 
>> Decisions about support involve tradeoffs, often tradeoffs between the
>> interests of different people.  Currently we don't have anything
>> resembling a guideline.  The result is that the individual debates are
>> inconclusive; and also, this framework does not lead to good feelings
>> amongst participants.
>> 
>> I suggest instead that we adopt a date-based policy: we define a
>> maximum *age* of dependencies that we will support.
> 
> I wonder about another approach: specify supported toolchain version(s)
> based on environments we choose to care about. That would be things like
> "Debian, including LTS (or even ELTS) one", "RHEL/CentOS until X...",
> etc. Based on this, it's easy to derive what's the oldest version that
> needs to be supported.
> This would be also much friendlier for testing - a clear definition
> what environments should be used (in gitlab-ci, I guess).

This is in fact what I’ve been thinking and talking about proposing for a very long time.  As far as an open-source offering, what we really want is for the newest version of Xen to build on all currently-supported distros.  If the distro maintainers themselves no longer want to support a distro, I don’t see why we should make the effort to do so.

As you say, this should make testing super easy as well: All we have to do is have docker images on gitlab for all the supported distros.

 -George

On 13.08.2021 13:37, Ian Jackson wrote:
> The current policy for minimum supported versions of tools, compilers,
> etc. is unsatisfactory: For many dependencies no minimum version is
> specified.  For those where a version is stated, updating it is a
> decision that has to be explicitly taken for that tool.

Considering your submission of this having been close to a glibc
version issue you and I have been discussing, I wonder whether
"etc" above includes library dependencies as well.

In any event the precise scope of what is meant to be covered is
quite important to me: There are affected entities that I'm happy
to replace on older distros (binutils, gcc). There are potentially
affected entities that I'm less happy to replace, but at the time
I did work my way through it for example for Python (to still be
able to build qemu, the community of which doesn't appear to care
at all to have their stuff buildable in older environments). The
point where I'd be really in trouble would be when base platform
libraries like glibc are required to be a certain minimum version:
I'd then be (potentially severely) restricted in what systems I
can actually test stuff on.

In addition I see a difference between actively breaking e.g.
building with older tool chains vs (like you have it in your
README adjustment) merely a statement about what we believe
things may work with, leaving room for people to fix issues with
their (older) environments, and such changes then not getting
rejected simply because of policy.

> The result is persistent debates over what is good to support,
> conducted in detail in the context of individual patches.
> 
> Decisions about support involve tradeoffs, often tradeoffs between the
> interests of different people.  Currently we don't have anything
> resembling a guideline.  The result is that the individual debates are
> inconclusive; and also, this framework does not lead to good feelings
> amongst participants.
> 
> I suggest instead that we adopt a date-based policy: we define a
> maximum *age* of dependencies that we will support.
> 
> The existing documentation about actually known working versions
> then becomes a practical consequence of that policy.
> 
> In this patch I propose a cutoff of 6 years.
> Obviously there will be debate about the precise value.

Indeed I consider this way too short. Purely as a personal (and
abstract) view (realizing this isn't really practical, and knowing
there are reasons why I'd actually like to see a bump of the
baseline) I'd prefer if there weren't minimum version requirements
at all (apart from maybe - along the lines of ...

> It will also be necessary to make exceptions, and/or to make different
> rules for different architectures.  In particular, new architectures,
> new configurations, or new features, may need an absolute earliest
> tooling date which is considerably less than the usual limit.

... this - a baseline determined when Xen became an open source
project). Advanced features may of course be dependent on better
capabilities, as long as there's a way to disable building or
use of these features.

While generally I find Marek's proposal better to tie the baseline
to distros of interest, in a way it only shifts the issue, I'm
afraid.

Jan

Jan Beulich writes ("Re: [PATCH] RFC: Version support policy"):
> On 13.08.2021 13:37, Ian Jackson wrote:
> > The current policy for minimum supported versions of tools, compilers,
> > etc. is unsatisfactory: For many dependencies no minimum version is
> > specified.  For those where a version is stated, updating it is a
> > decision that has to be explicitly taken for that tool.
> 
> Considering your submission of this having been close to a glibc
> version issue you and I have been discussing, I wonder whether
> "etc" above includes library dependencies as well.

Yes.  This is intending to cover all dependencies of whatever nature.

> In addition I see a difference between actively breaking e.g.
> building with older tool chains vs (like you have it in your
> README adjustment) merely a statement about what we believe
> things may work with, leaving room for people to fix issues with
> their (older) environments, and such changes then not getting
> rejected simply because of policy.

This is a valid concern.  I was thinking about this and I think
something needs to be written about this somewhere but the REAME isn't
the right place.  CODING_STYLE maybe.

> > In this patch I propose a cutoff of 6 years.
> > Obviously there will be debate about the precise value.
> 
> Indeed I consider this way too short. Purely as a personal (and
> abstract) view (realizing this isn't really practical, and knowing
> there are reasons why I'd actually like to see a bump of the
> baseline) I'd prefer if there weren't minimum version requirements
> at all (apart from maybe - along the lines of ...
> 
> > It will also be necessary to make exceptions, and/or to make different
> > rules for different architectures.  In particular, new architectures,
> > new configurations, or new features, may need an absolute earliest
> > tooling date which is considerably less than the usual limit.
> 
> ... this - a baseline determined when Xen became an open source
> project).

I don't think that is workable.  Effectively, it means we are
targeting a constantly-obsolescing dependency environment.  It
would prevent us from adopting even very-well-established facilities
and improvements in our dependencies.

Effectively, it would force us to continue to write using 10- or
20-year-old idioms.  Idioms many of which have been found to be
suboptimal, and which in some cases are becoming unsupported.

Ian.

On 19.08.2021 13:55, Ian Jackson wrote:
> Jan Beulich writes ("Re: [PATCH] RFC: Version support policy"):
>> On 13.08.2021 13:37, Ian Jackson wrote:
>>> In this patch I propose a cutoff of 6 years.
>>> Obviously there will be debate about the precise value.
>>
>> Indeed I consider this way too short. Purely as a personal (and
>> abstract) view (realizing this isn't really practical, and knowing
>> there are reasons why I'd actually like to see a bump of the
>> baseline) I'd prefer if there weren't minimum version requirements
>> at all (apart from maybe - along the lines of ...
>>
>>> It will also be necessary to make exceptions, and/or to make different
>>> rules for different architectures.  In particular, new architectures,
>>> new configurations, or new features, may need an absolute earliest
>>> tooling date which is considerably less than the usual limit.
>>
>> ... this - a baseline determined when Xen became an open source
>> project).
> 
> I don't think that is workable.  Effectively, it means we are
> targeting a constantly-obsolescing dependency environment.  It
> would prevent us from adopting even very-well-established facilities
> and improvements in our dependencies.
> 
> Effectively, it would force us to continue to write using 10- or
> 20-year-old idioms.  Idioms many of which have been found to be
> suboptimal, and which in some cases are becoming unsupported.

Right - that's why I did write "knowing there are reasons why I'd
actually like to see a bump of the baseline". I'm really of two
minds here, and either route has perhaps severe drawbacks.

Jan

> On Aug 19, 2021, at 10:18 AM, Jan Beulich <JBeulich@suse.com> wrote:
> 
> On 13.08.2021 13:37, Ian Jackson wrote:
>> The current policy for minimum supported versions of tools, compilers,
>> etc. is unsatisfactory: For many dependencies no minimum version is
>> specified.  For those where a version is stated, updating it is a
>> decision that has to be explicitly taken for that tool.
> 
> Considering your submission of this having been close to a glibc
> version issue you and I have been discussing, I wonder whether
> "etc" above includes library dependencies as well.
> 
> In any event the precise scope of what is meant to be covered is
> quite important to me: There are affected entities that I'm happy
> to replace on older distros (binutils, gcc). There are potentially
> affected entities that I'm less happy to replace, but at the time
> I did work my way through it for example for Python (to still be
> able to build qemu, the community of which doesn't appear to care
> at all to have their stuff buildable in older environments). The
> point where I'd be really in trouble would be when base platform
> libraries like glibc are required to be a certain minimum version:
> I'd then be (potentially severely) restricted in what systems I
> can actually test stuff on.

The question here is, why would someone running a 10-year-old distro that’s been out of support for 6 years want to run a bleeding edge version of Xen?  I understand wanting to run Xen 4.16 on (say) Ubuntu 18.04, but who on earth would want to run Xen 4.16 on Ubuntu 14.04, and why?  If such people exist, is it really worth the effort to try to support them?

> In addition I see a difference between actively breaking e.g.
> building with older tool chains vs (like you have it in your
> README adjustment) merely a statement about what we believe
> things may work with, leaving room for people to fix issues with
> their (older) environments, and such changes then not getting
> rejected simply because of policy.

Yes; I think the principle should be that we *promise* to keep it working on the currently-supported releases of a specific set of distros (e.g., Debian, Ubuntu, Fedora, SUSE, RHEL).  Working on older versions can be best-effort; if simple changes make it compatible with older versions, and aren’t too burdensome from a code complexity point of view, they can be accepted.

One of the issues however is build-time checks.  If we have a build-time check for version X, but only test it on X+10 or later, then the build may break in strange ways when someone tries it on something in between.

I think it’s too much effort to ask developers to try to find the actual minimum version of each individual dependency as things evolve.

> While generally I find Marek's proposal better to tie the baseline
> to distros of interest, in a way it only shifts the issue, I'm
> afraid.

What do you mean “shifts the issue”?  You mean shifts it from versions of individual components to versions of distros?

That’s why I think we should support only currently-supported distros.  If the distro’s maintainers don’t consider the distro worth supporting any more, I don’t see why we should make the effort to do so.

 -George

On 14.02.2022 22:50, George Dunlap wrote:
>> On Aug 19, 2021, at 10:18 AM, Jan Beulich <JBeulich@suse.com> wrote:
>> On 13.08.2021 13:37, Ian Jackson wrote:
>>> The current policy for minimum supported versions of tools, compilers,
>>> etc. is unsatisfactory: For many dependencies no minimum version is
>>> specified.  For those where a version is stated, updating it is a
>>> decision that has to be explicitly taken for that tool.
>>
>> Considering your submission of this having been close to a glibc
>> version issue you and I have been discussing, I wonder whether
>> "etc" above includes library dependencies as well.
>>
>> In any event the precise scope of what is meant to be covered is
>> quite important to me: There are affected entities that I'm happy
>> to replace on older distros (binutils, gcc). There are potentially
>> affected entities that I'm less happy to replace, but at the time
>> I did work my way through it for example for Python (to still be
>> able to build qemu, the community of which doesn't appear to care
>> at all to have their stuff buildable in older environments). The
>> point where I'd be really in trouble would be when base platform
>> libraries like glibc are required to be a certain minimum version:
>> I'd then be (potentially severely) restricted in what systems I
>> can actually test stuff on.
> 
> The question here is, why would someone running a 10-year-old distro that’s been out of support for 6 years want to run a bleeding edge version of Xen?  I understand wanting to run Xen 4.16 on (say) Ubuntu 18.04, but who on earth would want to run Xen 4.16 on Ubuntu 14.04, and why?  If such people exist, is it really worth the effort to try to support them?

I do this, for the very simple reason of wanting (needing) to be able
to test a large range of Xen versions all on the same small set of
hardware. Internally we're still maintaining versions back to at least
4.4; upon customer request we (I) may end up needing to even play with
4.0.

>> In addition I see a difference between actively breaking e.g.
>> building with older tool chains vs (like you have it in your
>> README adjustment) merely a statement about what we believe
>> things may work with, leaving room for people to fix issues with
>> their (older) environments, and such changes then not getting
>> rejected simply because of policy.
> 
> Yes; I think the principle should be that we *promise* to keep it working on the currently-supported releases of a specific set of distros (e.g., Debian, Ubuntu, Fedora, SUSE, RHEL).  Working on older versions can be best-effort; if simple changes make it compatible with older versions, and aren’t too burdensome from a code complexity point of view, they can be accepted.
> 
> One of the issues however is build-time checks.  If we have a build-time check for version X, but only test it on X+10 or later, then the build may break in strange ways when someone tries it on something in between.

Well, because most people only test on "X+10 or later", is has been
frequently me to run into issues with, in particular, old gcc versions.
And I've been making fixes / workarounds for those. Hence I wouldn't
consider the full range entirely untested. Obviously not every version
in the range would see testing, unless we specifically arranged for
doing so in, say, CI.

> I think it’s too much effort to ask developers to try to find the actual minimum version of each individual dependency as things evolve.

Hmm. On one hand I agree that it may be a lot to ask for. Otoh I
generally take the position that it is okay for advanced functionality
to be unavailable unless certain dependencies are met, but that base
functionality should be provided (almost) indefinitely far backwards.
I did add "(almost)" because I think it is fair for a project to draw
a baseline at the time it is founded. No-one would expect Xen to be
possible to be built with K&R C compilers.

Beyond that raising the baseline for any component needed for building
needs to consider how difficult it is for people to meet that new
requirement. Speaking for myself, I find it acceptable to build certain
leaf components (binutils, gcc, make, etc, and I've even worked my way
through building newer Python), but things get more hairy when e.g. a
shared library needs replacement. (Prime example of the latter would be
libelf, which Linux'es objtool depends upon being a half way recent
version.)

>> While generally I find Marek's proposal better to tie the baseline
>> to distros of interest, in a way it only shifts the issue, I'm
>> afraid.
> 
> What do you mean “shifts the issue”?  You mean shifts it from versions of individual components to versions of distros?

(Half a year later I first had to go back and check Marek's reply.)
Yes. Individual component versions would then be inferred from distro
versions. While I don't know how other distros handle this, in ours
parts of the tool chain also used to get updated during the lifetime
of a distro version (where I already mean to limit "version" to e.g.
service packs). For binutils typically by simple replacing the older
version, while for gcc typically by making a newer major version
available as an option. In such cases it then of course becomes fuzzy
what the "distro => component" mapping would be.

> That’s why I think we should support only currently-supported distros.  If the distro’s maintainers don’t consider the distro worth supporting any more, I don’t see why we should make the effort to do so.

And "currently supported" ends when? After "normal" EOL, or at the
end of what's often called LTS / LTSS? For the latter case: I've
recently learned that we've gained further sub-classes of LTSS, with
different life times.

Jan

> On Feb 15, 2022, at 8:20 AM, Jan Beulich <jbeulich@suse.com> wrote:
> 
> On 14.02.2022 22:50, George Dunlap wrote:
>>> On Aug 19, 2021, at 10:18 AM, Jan Beulich <JBeulich@suse.com> wrote:
>>> On 13.08.2021 13:37, Ian Jackson wrote:
>>>> The current policy for minimum supported versions of tools, compilers,
>>>> etc. is unsatisfactory: For many dependencies no minimum version is
>>>> specified.  For those where a version is stated, updating it is a
>>>> decision that has to be explicitly taken for that tool.
>>> 
>>> Considering your submission of this having been close to a glibc
>>> version issue you and I have been discussing, I wonder whether
>>> "etc" above includes library dependencies as well.
>>> 
>>> In any event the precise scope of what is meant to be covered is
>>> quite important to me: There are affected entities that I'm happy
>>> to replace on older distros (binutils, gcc). There are potentially
>>> affected entities that I'm less happy to replace, but at the time
>>> I did work my way through it for example for Python (to still be
>>> able to build qemu, the community of which doesn't appear to care
>>> at all to have their stuff buildable in older environments). The
>>> point where I'd be really in trouble would be when base platform
>>> libraries like glibc are required to be a certain minimum version:
>>> I'd then be (potentially severely) restricted in what systems I
>>> can actually test stuff on.
>> 
>> The question here is, why would someone running a 10-year-old distro that’s been out of support for 6 years want to run a bleeding edge version of Xen?  I understand wanting to run Xen 4.16 on (say) Ubuntu 18.04, but who on earth would want to run Xen 4.16 on Ubuntu 14.04, and why?  If such people exist, is it really worth the effort to try to support them?
> 
> I do this, for the very simple reason of wanting (needing) to be able
> to test a large range of Xen versions all on the same small set of
> hardware. Internally we're still maintaining versions back to at least
> 4.4; upon customer request we (I) may end up needing to even play with
> 4.0.

You don’t mention what software you’re talking about for versions 4.4 and 4.0, so I assume you mean Xen.

What I’m hearing you say is:

1. You have a handful of test hardware upon which you do your own manual testing.

2. You need to test at least as far back as Xen 4.4, possibly as far back as Xen 4.0, since you have customers that are using those versions.

3. It’s not feasible to test Xen 4.4 on a modern version of SUSE.  Presumably this is some combination of 3a. The customers using those versions are in fact using versions of SUSE from that timeframe as well, so thats what needs testing and 3b. It’s impractical to get Xen 4.4 to build on a modern version of SUSE.

4. It’s not feasible to use different SUSE versions on this hardware, such that each version of Xen is being tested with a version of SUSE from the appropriate time frame. Presumably this is some combination of 4a. You don’t want the hassle of re-installing the machine every time you want to test it (and it’s not feasible / too much of a hassle to maintain multiple parallel installations on the machine) 4b. Newer versions of SUSE wouldn’t run on this machine, since it’s so old.

Is that what I’m hearing?

So first of all, you are not an end-user, and running this sort of test is not “running Xen”.  I’m talking about end-users actually using Xen 4.16 “in anger” as they say in the UK, on Ubuntu 14.04; not for testing, but because they actually needed to use virtualization to solve a problem that they had.  Are there any people out there who need a hypervisor to solve a problem they have, and want to use Xen 4.16 with Ubuntu 14.04?

 -George