With in-tree builds Config.mk includes a .config file (if present) from
the top-level directory. Similar functionality is wanted with out-of-
tree builds. Yet the concept of "top-level directory" becomes fuzzy in
that case, because there is not really a requirement to have identical
top-level directory structure in the output tree; in fact there's no
need for anything top-level-ish there. Look for such a .config, but only
if the tree layout matches (read: if the directory we're building in is
named "xen").

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
RFC: The directory name based heuristic of course isn't nice. But I
     couldn't think of anything better. Suggestions?

RFC: There also being a .config in the top-level source dir would be a
     little problematic: It would be included _after_ the one in the
     object tree. Yet if such a scenario is to be expected/supported at
     all, it makes more sense the other way around.

--- a/xen/Makefile
+++ b/xen/Makefile
@@ -236,8 +236,17 @@ endif
 
 include scripts/Kbuild.include
 
-# Don't break if the build process wasn't called from the top level
-# we need XEN_TARGET_ARCH to generate the proper config
+# Don't break if the build process wasn't called from the top level.  We need
+# XEN_TARGET_ARCH to generate the proper config.  If building outside of the
+# source tree also check whether we need to include a "top-level" .config:
+# Config.mk, using $(XEN_ROOT)/.config, would look only in the source tree.
+ifeq ($(building_out_of_srctree),1)
+# Try to avoid including a random unrelated .config: Assume our parent dir
+# is a "top-level" one only when the objtree is .../xen.
+ifeq ($(patsubst %/xen,,$(abs_objtree)),)
+-include ../.config
+endif
+endif
 include $(XEN_ROOT)/Config.mk
 
 # Set ARCH/SUBARCH appropriately.
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -17,6 +17,13 @@ __build:
 
 -include $(objtree)/include/config/auto.conf
 
+# See commentary around the similar contruct in Makefile.
+ifneq ($(abs_objtree),$(abs_srctree))
+ifeq ($(patsubst %/xen,,$(abs_objtree)),)
+../.config: ;
+-include ../.config
+endif
+endif
 include $(XEN_ROOT)/Config.mk
 include $(srctree)/scripts/Kbuild.include

On Wed, Mar 15, 2023 at 03:58:59PM +0100, Jan Beulich wrote:
> With in-tree builds Config.mk includes a .config file (if present) from
> the top-level directory. Similar functionality is wanted with out-of-
> tree builds. Yet the concept of "top-level directory" becomes fuzzy in
> that case, because there is not really a requirement to have identical
> top-level directory structure in the output tree; in fact there's no
> need for anything top-level-ish there. Look for such a .config, but only
> if the tree layout matches (read: if the directory we're building in is
> named "xen").

Well, as long as the "xen/" part of the repository is the only build
system to be able to build out-of-srctree, there isn't going to be a
top-level .config possible in the build tree, as such .config will be
outside of the build tree. Reading outside of the build tree or source
tree might be problematic.

It's a possibility that some project might want to just build the
hypervisor, and they happened to name the build tree "xen", but they
also have a ".config" use for something else. Kconfig is using ".config"
for other reason for example, like we do to build Xen.

It might be better to have a different name instead of ".config", and
putting it in the build tree rather than the parent directory. Maybe
".xenbuild-config" ?


I've been using the optional makefile named "xen-version" to adjust
variable of the build system, with content like:

    export XEN_TARGET_ARCH=arm64
    export CROSS_COMPILE=aarch64-linux-gnu-

so that I can have multiple build tree for different arch, and not have
to do anything other than running make and do the expected build. Maybe
that's not possible because for some reason, the build system still
reconfigure some variable when entering a sub-directory, but that's a
start.


> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> RFC: The directory name based heuristic of course isn't nice. But I
>      couldn't think of anything better. Suggestions?

I can only thing of looking at a file that is in the build-tree rather
than outside of it. A file named ".xenbuild-config" proposed early for
example.

> RFC: There also being a .config in the top-level source dir would be a
>      little problematic: It would be included _after_ the one in the
>      object tree. Yet if such a scenario is to be expected/supported at
>      all, it makes more sense the other way around.

Well, that would mean teaching Config.mk about out-of-tree build that
part of the repository is capable of, or better would be to stop trying
to read ".config" from Config.mk, and handle the file in the different
sub-build systems. Or just let people writing ".config" files to handle
the cases in those .config files.

> --- a/xen/Makefile
> +++ b/xen/Makefile
> @@ -236,8 +236,17 @@ endif
>  
>  include scripts/Kbuild.include
>  
> -# Don't break if the build process wasn't called from the top level
> -# we need XEN_TARGET_ARCH to generate the proper config
> +# Don't break if the build process wasn't called from the top level.  We need
> +# XEN_TARGET_ARCH to generate the proper config.  If building outside of the
> +# source tree also check whether we need to include a "top-level" .config:
> +# Config.mk, using $(XEN_ROOT)/.config, would look only in the source tree.
> +ifeq ($(building_out_of_srctree),1)
> +# Try to avoid including a random unrelated .config: Assume our parent dir
> +# is a "top-level" one only when the objtree is .../xen.
> +ifeq ($(patsubst %/xen,,$(abs_objtree)),)
> +-include ../.config
> +endif
> +endif
>  include $(XEN_ROOT)/Config.mk
>  
>  # Set ARCH/SUBARCH appropriately.
> --- a/xen/Rules.mk
> +++ b/xen/Rules.mk
> @@ -17,6 +17,13 @@ __build:
>  
>  -include $(objtree)/include/config/auto.conf
>  
> +# See commentary around the similar contruct in Makefile.
> +ifneq ($(abs_objtree),$(abs_srctree))
> +ifeq ($(patsubst %/xen,,$(abs_objtree)),)
> +../.config: ;
> +-include ../.config
> +endif
> +endif
>  include $(XEN_ROOT)/Config.mk
>  include $(srctree)/scripts/Kbuild.include

There's another makefile, "scripts/Makefile.clean", doesn't this would
need to be change as well?

Cheers,

-- 
Anthony PERARD

On 05.05.2023 18:08, Anthony PERARD wrote:
> On Wed, Mar 15, 2023 at 03:58:59PM +0100, Jan Beulich wrote:
>> With in-tree builds Config.mk includes a .config file (if present) from
>> the top-level directory. Similar functionality is wanted with out-of-
>> tree builds. Yet the concept of "top-level directory" becomes fuzzy in
>> that case, because there is not really a requirement to have identical
>> top-level directory structure in the output tree; in fact there's no
>> need for anything top-level-ish there. Look for such a .config, but only
>> if the tree layout matches (read: if the directory we're building in is
>> named "xen").
> 
> Well, as long as the "xen/" part of the repository is the only build
> system to be able to build out-of-srctree, there isn't going to be a
> top-level .config possible in the build tree, as such .config will be
> outside of the build tree. Reading outside of the build tree or source
> tree might be problematic.
> 
> It's a possibility that some project might want to just build the
> hypervisor, and they happened to name the build tree "xen", but they
> also have a ".config" use for something else. Kconfig is using ".config"
> for other reason for example, like we do to build Xen.

Right, that's what my first RFC remark is about.

> It might be better to have a different name instead of ".config", and
> putting it in the build tree rather than the parent directory. Maybe
> ".xenbuild-config" ?

Using a less ambiguous name is clearly an option. Putting the file in
the (Xen) build directory, otoh, imo isn't: In the hope that further
sub-trees would be enabled to build out-of-tree (qemu for instance in
principle can already, and I guess at least some of stubdom's sub-
packages also can), the present functionality of the top-level
.config (or whatever its new name) also controlling those builds would
better be retained.

> I've been using the optional makefile named "xen-version" to adjust
> variable of the build system, with content like:
> 
>     export XEN_TARGET_ARCH=arm64
>     export CROSS_COMPILE=aarch64-linux-gnu-
> 
> so that I can have multiple build tree for different arch, and not have
> to do anything other than running make and do the expected build. Maybe
> that's not possible because for some reason, the build system still
> reconfigure some variable when entering a sub-directory, but that's a
> start.

Hmm, interesting idea. I could (ab)use this at least in the short
term. Yet even then the file would further need including from
Rules.mk. Requiring all variables defined there to be exported isn't
a good idea, imo. Iirc not all make variables can usefully be
exported. For example, I have a local extension to CROSS_COMPILE in
place, which uses a variable with a dash in its name.

>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>> ---
>> RFC: The directory name based heuristic of course isn't nice. But I
>>      couldn't think of anything better. Suggestions?
> 
> I can only thing of looking at a file that is in the build-tree rather
> than outside of it. A file named ".xenbuild-config" proposed early for
> example.
> 
>> RFC: There also being a .config in the top-level source dir would be a
>>      little problematic: It would be included _after_ the one in the
>>      object tree. Yet if such a scenario is to be expected/supported at
>>      all, it makes more sense the other way around.
> 
> Well, that would mean teaching Config.mk about out-of-tree build that
> part of the repository is capable of, or better would be to stop trying
> to read ".config" from Config.mk, and handle the file in the different
> sub-build systems.

Hmm, is that a viable option? Or put differently: Wouldn't this mean doing
away with ./Config.mk altogether? Which in turn would mean no central
place anymore where XEN_TARGET_ARCH and friends could be overridden. (I
view this as a capability that we want to retain, if nothing else then for
the - see above - future option of allowing more than just xen/ to be
built out-of-tree.)

> Or just let people writing ".config" files to handle
> the cases in those .config files.

I'm afraid I don't see what you mean here.

>> --- a/xen/Makefile
>> +++ b/xen/Makefile
>> @@ -236,8 +236,17 @@ endif
>>  
>>  include scripts/Kbuild.include
>>  
>> -# Don't break if the build process wasn't called from the top level
>> -# we need XEN_TARGET_ARCH to generate the proper config
>> +# Don't break if the build process wasn't called from the top level.  We need
>> +# XEN_TARGET_ARCH to generate the proper config.  If building outside of the
>> +# source tree also check whether we need to include a "top-level" .config:
>> +# Config.mk, using $(XEN_ROOT)/.config, would look only in the source tree.
>> +ifeq ($(building_out_of_srctree),1)
>> +# Try to avoid including a random unrelated .config: Assume our parent dir
>> +# is a "top-level" one only when the objtree is .../xen.
>> +ifeq ($(patsubst %/xen,,$(abs_objtree)),)
>> +-include ../.config
>> +endif
>> +endif
>>  include $(XEN_ROOT)/Config.mk
>>  
>>  # Set ARCH/SUBARCH appropriately.
>> --- a/xen/Rules.mk
>> +++ b/xen/Rules.mk
>> @@ -17,6 +17,13 @@ __build:
>>  
>>  -include $(objtree)/include/config/auto.conf
>>  
>> +# See commentary around the similar contruct in Makefile.
>> +ifneq ($(abs_objtree),$(abs_srctree))
>> +ifeq ($(patsubst %/xen,,$(abs_objtree)),)
>> +../.config: ;
>> +-include ../.config
>> +endif
>> +endif
>>  include $(XEN_ROOT)/Config.mk
>>  include $(srctree)/scripts/Kbuild.include
> 
> There's another makefile, "scripts/Makefile.clean", doesn't this would
> need to be change as well?

In theory - maybe. But in practice: What override might there be that one
needs to put in place to run "clean". XEN_TARGET_ARCH, for example, better
wouldn't be needed for cleaning. Furthermore the top-level .config hasn't
been included there either, afaict.

Jan

On Mon, May 08, 2023 at 08:23:43AM +0200, Jan Beulich wrote:
> On 05.05.2023 18:08, Anthony PERARD wrote:
> > On Wed, Mar 15, 2023 at 03:58:59PM +0100, Jan Beulich wrote:
> >> With in-tree builds Config.mk includes a .config file (if present) from
> >> the top-level directory. Similar functionality is wanted with out-of-
> >> tree builds. Yet the concept of "top-level directory" becomes fuzzy in
> >> that case, because there is not really a requirement to have identical
> >> top-level directory structure in the output tree; in fact there's no
> >> need for anything top-level-ish there. Look for such a .config, but only
> >> if the tree layout matches (read: if the directory we're building in is
> >> named "xen").
> > 
> > Well, as long as the "xen/" part of the repository is the only build
> > system to be able to build out-of-srctree, there isn't going to be a
> > top-level .config possible in the build tree, as such .config will be
> > outside of the build tree. Reading outside of the build tree or source
> > tree might be problematic.
> > 
> > It's a possibility that some project might want to just build the
> > hypervisor, and they happened to name the build tree "xen", but they
> > also have a ".config" use for something else. Kconfig is using ".config"
> > for other reason for example, like we do to build Xen.
> 
> Right, that's what my first RFC remark is about.
> 
> > It might be better to have a different name instead of ".config", and
> > putting it in the build tree rather than the parent directory. Maybe
> > ".xenbuild-config" ?
> 
> Using a less ambiguous name is clearly an option. Putting the file in
> the (Xen) build directory, otoh, imo isn't: In the hope that further
> sub-trees would be enabled to build out-of-tree (qemu for instance in
> principle can already, and I guess at least some of stubdom's sub-
> packages also can), the present functionality of the top-level
> .config (or whatever its new name) also controlling those builds would
> better be retained.

I'm not sure what out-of-tree build for the whole tree will look like,
but it probably going to be `/path/to/configure && make`. After that,
Config.mk should know what kind of build it's doing, and probably only
load ".config" in the build tree. In the meantime, it feels out of place
for xen/Makefile or xen/Rules.mk to load a ".config" that would be
present in the parent directory of the build dir.

> > I've been using the optional makefile named "xen-version" to adjust
> > variable of the build system, with content like:
> > 
> >     export XEN_TARGET_ARCH=arm64
> >     export CROSS_COMPILE=aarch64-linux-gnu-
> > 
> > so that I can have multiple build tree for different arch, and not have
> > to do anything other than running make and do the expected build. Maybe
> > that's not possible because for some reason, the build system still
> > reconfigure some variable when entering a sub-directory, but that's a
> > start.
> 
> Hmm, interesting idea. I could (ab)use this at least in the short
> term. Yet even then the file would further need including from
> Rules.mk. Requiring all variables defined there to be exported isn't
> a good idea, imo. Iirc not all make variables can usefully be
> exported. For example, I have a local extension to CROSS_COMPILE in
> place, which uses a variable with a dash in its name.
> 
> >> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> >> ---
> >> RFC: The directory name based heuristic of course isn't nice. But I
> >>      couldn't think of anything better. Suggestions?
> > 
> > I can only thing of looking at a file that is in the build-tree rather
> > than outside of it. A file named ".xenbuild-config" proposed early for
> > example.
> > 
> >> RFC: There also being a .config in the top-level source dir would be a
> >>      little problematic: It would be included _after_ the one in the
> >>      object tree. Yet if such a scenario is to be expected/supported at
> >>      all, it makes more sense the other way around.
> > 
> > Well, that would mean teaching Config.mk about out-of-tree build that
> > part of the repository is capable of, or better would be to stop trying
> > to read ".config" from Config.mk, and handle the file in the different
> > sub-build systems.
> 
> Hmm, is that a viable option? Or put differently: Wouldn't this mean doing
> away with ./Config.mk altogether? Which in turn would mean no central
> place anymore where XEN_TARGET_ARCH and friends could be overridden. (I
> view this as a capability that we want to retain, if nothing else then for
> the - see above - future option of allowing more than just xen/ to be
> built out-of-tree.)

No, I'm not trying to get rid of Config.mk. There's a few thing in it
that I'd like to remove from it, but not everything. I don't know how to
deal with ".config" at the moment, but I guess that if Config.mk knew
about out-of-tree build, it probably should only read one ".config", the
one in the build tree.

-- 
Anthony PERARD

On 22.05.2023 17:49, Anthony PERARD wrote:
> On Mon, May 08, 2023 at 08:23:43AM +0200, Jan Beulich wrote:
>> On 05.05.2023 18:08, Anthony PERARD wrote:
>>> On Wed, Mar 15, 2023 at 03:58:59PM +0100, Jan Beulich wrote:
>>>> With in-tree builds Config.mk includes a .config file (if present) from
>>>> the top-level directory. Similar functionality is wanted with out-of-
>>>> tree builds. Yet the concept of "top-level directory" becomes fuzzy in
>>>> that case, because there is not really a requirement to have identical
>>>> top-level directory structure in the output tree; in fact there's no
>>>> need for anything top-level-ish there. Look for such a .config, but only
>>>> if the tree layout matches (read: if the directory we're building in is
>>>> named "xen").
>>>
>>> Well, as long as the "xen/" part of the repository is the only build
>>> system to be able to build out-of-srctree, there isn't going to be a
>>> top-level .config possible in the build tree, as such .config will be
>>> outside of the build tree. Reading outside of the build tree or source
>>> tree might be problematic.
>>>
>>> It's a possibility that some project might want to just build the
>>> hypervisor, and they happened to name the build tree "xen", but they
>>> also have a ".config" use for something else. Kconfig is using ".config"
>>> for other reason for example, like we do to build Xen.
>>
>> Right, that's what my first RFC remark is about.
>>
>>> It might be better to have a different name instead of ".config", and
>>> putting it in the build tree rather than the parent directory. Maybe
>>> ".xenbuild-config" ?
>>
>> Using a less ambiguous name is clearly an option. Putting the file in
>> the (Xen) build directory, otoh, imo isn't: In the hope that further
>> sub-trees would be enabled to build out-of-tree (qemu for instance in
>> principle can already, and I guess at least some of stubdom's sub-
>> packages also can), the present functionality of the top-level
>> .config (or whatever its new name) also controlling those builds would
>> better be retained.
> 
> I'm not sure what out-of-tree build for the whole tree will look like,
> but it probably going to be `/path/to/configure && make`. After that,
> Config.mk should know what kind of build it's doing, and probably only
> load ".config" in the build tree.

Except that the hypervisor build still isn't really connected to
./configure's results.

> In the meantime, it feels out of place
> for xen/Makefile or xen/Rules.mk to load a ".config" that would be
> present in the parent directory of the build dir.

Right, hence me looking for possible alternatives (and using this
approach only for the apparent lack thereof).

>>> I've been using the optional makefile named "xen-version" to adjust
>>> variable of the build system, with content like:
>>>
>>>     export XEN_TARGET_ARCH=arm64
>>>     export CROSS_COMPILE=aarch64-linux-gnu-
>>>
>>> so that I can have multiple build tree for different arch, and not have
>>> to do anything other than running make and do the expected build. Maybe
>>> that's not possible because for some reason, the build system still
>>> reconfigure some variable when entering a sub-directory, but that's a
>>> start.
>>
>> Hmm, interesting idea. I could (ab)use this at least in the short
>> term. Yet even then the file would further need including from
>> Rules.mk. Requiring all variables defined there to be exported isn't
>> a good idea, imo. Iirc not all make variables can usefully be
>> exported. For example, I have a local extension to CROSS_COMPILE in
>> place, which uses a variable with a dash in its name.
>>
>>>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
>>>> ---
>>>> RFC: The directory name based heuristic of course isn't nice. But I
>>>>      couldn't think of anything better. Suggestions?
>>>
>>> I can only thing of looking at a file that is in the build-tree rather
>>> than outside of it. A file named ".xenbuild-config" proposed early for
>>> example.
>>>
>>>> RFC: There also being a .config in the top-level source dir would be a
>>>>      little problematic: It would be included _after_ the one in the
>>>>      object tree. Yet if such a scenario is to be expected/supported at
>>>>      all, it makes more sense the other way around.
>>>
>>> Well, that would mean teaching Config.mk about out-of-tree build that
>>> part of the repository is capable of, or better would be to stop trying
>>> to read ".config" from Config.mk, and handle the file in the different
>>> sub-build systems.
>>
>> Hmm, is that a viable option? Or put differently: Wouldn't this mean doing
>> away with ./Config.mk altogether? Which in turn would mean no central
>> place anymore where XEN_TARGET_ARCH and friends could be overridden. (I
>> view this as a capability that we want to retain, if nothing else then for
>> the - see above - future option of allowing more than just xen/ to be
>> built out-of-tree.)
> 
> No, I'm not trying to get rid of Config.mk. There's a few thing in it
> that I'd like to remove from it, but not everything. I don't know how to
> deal with ".config" at the moment, but I guess that if Config.mk knew
> about out-of-tree build, it probably should only read one ".config", the
> one in the build tree.

And that (2nd) .config would then be placed where in that build tree,
according to what you're envisioning?

Just to mention it: Since a similar problem exists in Linux, for many
years I've been carrying private logic to record necessary overrides
in the Makefile that's generated into the build tree. But of course
that's hackery, i.e. doing just enough to fit my own needs. I'd like
to avoid having to carry similar hackery for Xen.

Jan

On 15.03.2023 15:58, Jan Beulich wrote:
> With in-tree builds Config.mk includes a .config file (if present) from
> the top-level directory. Similar functionality is wanted with out-of-
> tree builds. Yet the concept of "top-level directory" becomes fuzzy in
> that case, because there is not really a requirement to have identical
> top-level directory structure in the output tree; in fact there's no
> need for anything top-level-ish there. Look for such a .config, but only
> if the tree layout matches (read: if the directory we're building in is
> named "xen").
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> RFC: The directory name based heuristic of course isn't nice. But I
>      couldn't think of anything better. Suggestions?
> 
> RFC: There also being a .config in the top-level source dir would be a
>      little problematic: It would be included _after_ the one in the
>      object tree. Yet if such a scenario is to be expected/supported at
>      all, it makes more sense the other way around.

Anyone? I'm certainly okay for my approach to be rejected, but I'd like
to see out-of-tree builds to reach functional parity with in-tree ones.

Jan

> --- a/xen/Makefile
> +++ b/xen/Makefile
> @@ -236,8 +236,17 @@ endif
>  
>  include scripts/Kbuild.include
>  
> -# Don't break if the build process wasn't called from the top level
> -# we need XEN_TARGET_ARCH to generate the proper config
> +# Don't break if the build process wasn't called from the top level.  We need
> +# XEN_TARGET_ARCH to generate the proper config.  If building outside of the
> +# source tree also check whether we need to include a "top-level" .config:
> +# Config.mk, using $(XEN_ROOT)/.config, would look only in the source tree.
> +ifeq ($(building_out_of_srctree),1)
> +# Try to avoid including a random unrelated .config: Assume our parent dir
> +# is a "top-level" one only when the objtree is .../xen.
> +ifeq ($(patsubst %/xen,,$(abs_objtree)),)
> +-include ../.config
> +endif
> +endif
>  include $(XEN_ROOT)/Config.mk
>  
>  # Set ARCH/SUBARCH appropriately.
> --- a/xen/Rules.mk
> +++ b/xen/Rules.mk
> @@ -17,6 +17,13 @@ __build:
>  
>  -include $(objtree)/include/config/auto.conf
>  
> +# See commentary around the similar contruct in Makefile.
> +ifneq ($(abs_objtree),$(abs_srctree))
> +ifeq ($(patsubst %/xen,,$(abs_objtree)),)
> +../.config: ;
> +-include ../.config
> +endif
> +endif
>  include $(XEN_ROOT)/Config.mk
>  include $(srctree)/scripts/Kbuild.include
>  
>