Xenstored is using libxenctrl for only one purpose: to get information
about state of domains.

This patch series is removing that dependency by introducing a new
stable interface which can be used by xenstored instead.

There was a RFC series sent out 3 years ago, which I have taken as a
base and by addressing all comments from back then.

The main differences since that RFC series are:

- Instead of introducing an new main hypercall for a stable management
  interface I have just added a new domctl sub-op, as requested in 2021.

- I have added a new library libxenmanage for easy use of the new
  stable hypervisor interface. Main motivation for adding the library
  was the recent attempt to decouple oxenstored from the Xen git tree.
  By using the new library, oxenstored could benefit in the same way as
  xenstored from the new interface: it would be possible to rely on
  stable libraries only.

- Mini-OS has gained some more config options recently, so it was rather
  easy to make xenstore[pvh]-stubdom independent from libxenctrl, too.

Please note that the last 4 patches can be committed only after the
related Mini-OS patch "config: add support for libxenmanage" has gone in
AND the Mini-OS commit-id has been updated in Config.mk accordingly! The
Mini-OS patch has been Acked already, so it can go in as soon as patch
5 of this series (the one introducing libxenmanage) has been committed.

As patches 1 and 2 change current behavior, Jan didn't want to give his
Ack (he didn't reject the patch either). So I'm asking other "Rest"
maintainers to look at those patches specifically.

Changes in V2:
- new patch 1
- former patch 5 mover earlier, now patch 2 (can go in without the rest
  of the series)
- addressed comments

Changes in V3:
- addressed comments

Changes in V4:
- patches 1 and 3 of V3 dropped, as already committed
- addressed comments

Changes in V5:
- addressed comments

Changes in V6:
- patch 1 of V5 has been committed
- new patches 1-3 for fixing a race and avoiding new races with the
  added functionality (result of a comment by Jan Beulich)
- rework of locking in patch 4 (Jan Beulich)

Changes in V7:
- addressed comments
- rebase

Changes in V8:
- patch 1 of v7 has gone in
- addressed comments
- new patches 7-9 using the new unique_id and xenmanage_poll_changed_domain()

Juergen Gross (9):
  xen/events: don't allow binding a global virq from any domain
  xen/events: allow setting of global virq handler only for unbound
    virqs
  xen: add bitmap to indicate per-domain state changes
  xen: add new domctl get_changed_domain
  tools/libs: add a new libxenmanage library
  tools/xenstored: use new stable interface instead of libxenctrl
  docs: update xenstore migration stream definition
  tools/xenstored: use unique_id to identify new domain with same domid
  tools/xenstored: use xenmanage_poll_changed_domain()

 docs/designs/xenstore-migration.md     |  14 ++-
 stubdom/Makefile                       |   8 +-
 stubdom/mini-os.mk                     |   1 +
 tools/flask/policy/modules/dom0.te     |   1 +
 tools/flask/policy/modules/xen.if      |   5 +-
 tools/flask/policy/modules/xenstore.te |   1 +
 tools/include/xenmanage.h              |  92 ++++++++++++++
 tools/libs/Makefile                    |   1 +
 tools/libs/manage/Makefile             |  10 ++
 tools/libs/manage/Makefile.common      |   3 +
 tools/libs/manage/core.c               | 168 +++++++++++++++++++++++++
 tools/libs/manage/libxenmanage.map     |   8 ++
 tools/libs/uselibs.mk                  |   2 +
 tools/xenstored/Makefile               |   2 +-
 tools/xenstored/Makefile.common        |   2 +-
 tools/xenstored/core.h                 |   1 -
 tools/xenstored/domain.c               | 145 ++++++++++++++-------
 tools/xenstored/lu.c                   |   1 +
 tools/xenstored/lu_daemon.c            |   1 +
 tools/xenstored/xenstore_state.h       |   2 +-
 xen/common/domain.c                    | 125 ++++++++++++++++++
 xen/common/domctl.c                    |  18 ++-
 xen/common/event_channel.c             |  72 ++++++++++-
 xen/include/public/domctl.h            |  26 ++++
 xen/include/xen/event.h                |   4 +
 xen/include/xen/sched.h                |   5 +
 xen/include/xsm/dummy.h                |   8 ++
 xen/include/xsm/xsm.h                  |   6 +
 xen/xsm/dummy.c                        |   1 +
 xen/xsm/flask/hooks.c                  |   7 ++
 xen/xsm/flask/policy/access_vectors    |   2 +
 31 files changed, 677 insertions(+), 65 deletions(-)
 create mode 100644 tools/include/xenmanage.h
 create mode 100644 tools/libs/manage/Makefile
 create mode 100644 tools/libs/manage/Makefile.common
 create mode 100644 tools/libs/manage/core.c
 create mode 100644 tools/libs/manage/libxenmanage.map

-- 
2.43.0

Ping? Especially ...

On 04.02.25 12:33, Juergen Gross wrote:
> Xenstored is using libxenctrl for only one purpose: to get information
> about state of domains.
> 
> This patch series is removing that dependency by introducing a new
> stable interface which can be used by xenstored instead.
> 
> There was a RFC series sent out 3 years ago, which I have taken as a
> base and by addressing all comments from back then.
> 
> The main differences since that RFC series are:
> 
> - Instead of introducing an new main hypercall for a stable management
>    interface I have just added a new domctl sub-op, as requested in 2021.
> 
> - I have added a new library libxenmanage for easy use of the new
>    stable hypervisor interface. Main motivation for adding the library
>    was the recent attempt to decouple oxenstored from the Xen git tree.
>    By using the new library, oxenstored could benefit in the same way as
>    xenstored from the new interface: it would be possible to rely on
>    stable libraries only.
> 
> - Mini-OS has gained some more config options recently, so it was rather
>    easy to make xenstore[pvh]-stubdom independent from libxenctrl, too.
> 
> Please note that the last 4 patches can be committed only after the
> related Mini-OS patch "config: add support for libxenmanage" has gone in
> AND the Mini-OS commit-id has been updated in Config.mk accordingly! The
> Mini-OS patch has been Acked already, so it can go in as soon as patch
> 5 of this series (the one introducing libxenmanage) has been committed.
> 
> As patches 1 and 2 change current behavior, Jan didn't want to give his
> Ack (he didn't reject the patch either). So I'm asking other "Rest"
> maintainers to look at those patches specifically.

... patches 1 and 2 could need an additional opinion.


Juergen

On 25.02.25 12:10, Juergen Gross wrote:
> Ping? Especially ...
> 
> On 04.02.25 12:33, Juergen Gross wrote:
>> Xenstored is using libxenctrl for only one purpose: to get information
>> about state of domains.
>>
>> This patch series is removing that dependency by introducing a new
>> stable interface which can be used by xenstored instead.
>>
>> There was a RFC series sent out 3 years ago, which I have taken as a
>> base and by addressing all comments from back then.
>>
>> The main differences since that RFC series are:
>>
>> - Instead of introducing an new main hypercall for a stable management
>>    interface I have just added a new domctl sub-op, as requested in 2021.
>>
>> - I have added a new library libxenmanage for easy use of the new
>>    stable hypervisor interface. Main motivation for adding the library
>>    was the recent attempt to decouple oxenstored from the Xen git tree.
>>    By using the new library, oxenstored could benefit in the same way as
>>    xenstored from the new interface: it would be possible to rely on
>>    stable libraries only.
>>
>> - Mini-OS has gained some more config options recently, so it was rather
>>    easy to make xenstore[pvh]-stubdom independent from libxenctrl, too.
>>
>> Please note that the last 4 patches can be committed only after the
>> related Mini-OS patch "config: add support for libxenmanage" has gone in
>> AND the Mini-OS commit-id has been updated in Config.mk accordingly! The
>> Mini-OS patch has been Acked already, so it can go in as soon as patch
>> 5 of this series (the one introducing libxenmanage) has been committed.
>>
>> As patches 1 and 2 change current behavior, Jan didn't want to give his
>> Ack (he didn't reject the patch either). So I'm asking other "Rest"
>> maintainers to look at those patches specifically.
> 
> ... patches 1 and 2 could need an additional opinion.

And another ping.

One of Andrew, Stefano, Julien, Roger, Anthony, Mical: Please have a look.

The related discussion between Jan and me can be found here (patches 2 and 3):

https://lore.kernel.org/xen-devel/20250107101711.5980-1-jgross@suse.com/


Juergen

On Wed, 5 Mar 2025, Juergen Gross wrote:
> On 25.02.25 12:10, Juergen Gross wrote:
> > Ping? Especially ...
> > 
> > On 04.02.25 12:33, Juergen Gross wrote:
> > > Xenstored is using libxenctrl for only one purpose: to get information
> > > about state of domains.
> > > 
> > > This patch series is removing that dependency by introducing a new
> > > stable interface which can be used by xenstored instead.
> > > 
> > > There was a RFC series sent out 3 years ago, which I have taken as a
> > > base and by addressing all comments from back then.
> > > 
> > > The main differences since that RFC series are:
> > > 
> > > - Instead of introducing an new main hypercall for a stable management
> > >    interface I have just added a new domctl sub-op, as requested in 2021.
> > > 
> > > - I have added a new library libxenmanage for easy use of the new
> > >    stable hypervisor interface. Main motivation for adding the library
> > >    was the recent attempt to decouple oxenstored from the Xen git tree.
> > >    By using the new library, oxenstored could benefit in the same way as
> > >    xenstored from the new interface: it would be possible to rely on
> > >    stable libraries only.
> > > 
> > > - Mini-OS has gained some more config options recently, so it was rather
> > >    easy to make xenstore[pvh]-stubdom independent from libxenctrl, too.
> > > 
> > > Please note that the last 4 patches can be committed only after the
> > > related Mini-OS patch "config: add support for libxenmanage" has gone in
> > > AND the Mini-OS commit-id has been updated in Config.mk accordingly! The
> > > Mini-OS patch has been Acked already, so it can go in as soon as patch
> > > 5 of this series (the one introducing libxenmanage) has been committed.
> > > 
> > > As patches 1 and 2 change current behavior, Jan didn't want to give his
> > > Ack (he didn't reject the patch either). So I'm asking other "Rest"
> > > maintainers to look at those patches specifically.
> > 
> > ... patches 1 and 2 could need an additional opinion.
> 
> And another ping.
> 
> One of Andrew, Stefano, Julien, Roger, Anthony, Mical: Please have a look.
> 
> The related discussion between Jan and me can be found here (patches 2 and 3):
> 
> https://lore.kernel.org/xen-devel/20250107101711.5980-1-jgross@suse.com/

I didn't do an in-details review but based on Jan's comments and your
replies, I think they are an improvement. If someone else wants to do a
proper review, they would be welcome.

Acked-by: Stefano Stabellini <sstabellini@kernel.org>

On 06.03.2025 00:32, Stefano Stabellini wrote:
> On Wed, 5 Mar 2025, Juergen Gross wrote:
>> On 25.02.25 12:10, Juergen Gross wrote:
>>> Ping? Especially ...
>>>
>>> On 04.02.25 12:33, Juergen Gross wrote:
>>>> Xenstored is using libxenctrl for only one purpose: to get information
>>>> about state of domains.
>>>>
>>>> This patch series is removing that dependency by introducing a new
>>>> stable interface which can be used by xenstored instead.
>>>>
>>>> There was a RFC series sent out 3 years ago, which I have taken as a
>>>> base and by addressing all comments from back then.
>>>>
>>>> The main differences since that RFC series are:
>>>>
>>>> - Instead of introducing an new main hypercall for a stable management
>>>>    interface I have just added a new domctl sub-op, as requested in 2021.
>>>>
>>>> - I have added a new library libxenmanage for easy use of the new
>>>>    stable hypervisor interface. Main motivation for adding the library
>>>>    was the recent attempt to decouple oxenstored from the Xen git tree.
>>>>    By using the new library, oxenstored could benefit in the same way as
>>>>    xenstored from the new interface: it would be possible to rely on
>>>>    stable libraries only.
>>>>
>>>> - Mini-OS has gained some more config options recently, so it was rather
>>>>    easy to make xenstore[pvh]-stubdom independent from libxenctrl, too.
>>>>
>>>> Please note that the last 4 patches can be committed only after the
>>>> related Mini-OS patch "config: add support for libxenmanage" has gone in
>>>> AND the Mini-OS commit-id has been updated in Config.mk accordingly! The
>>>> Mini-OS patch has been Acked already, so it can go in as soon as patch
>>>> 5 of this series (the one introducing libxenmanage) has been committed.
>>>>
>>>> As patches 1 and 2 change current behavior, Jan didn't want to give his
>>>> Ack (he didn't reject the patch either). So I'm asking other "Rest"
>>>> maintainers to look at those patches specifically.
>>>
>>> ... patches 1 and 2 could need an additional opinion.
>>
>> And another ping.
>>
>> One of Andrew, Stefano, Julien, Roger, Anthony, Mical: Please have a look.
>>
>> The related discussion between Jan and me can be found here (patches 2 and 3):
>>
>> https://lore.kernel.org/xen-devel/20250107101711.5980-1-jgross@suse.com/
> 
> I didn't do an in-details review but based on Jan's comments and your
> replies, I think they are an improvement. If someone else wants to do a
> proper review, they would be welcome.
> 
> Acked-by: Stefano Stabellini <sstabellini@kernel.org>

I've taken the conservative approach and interpreted this as an ack for the
two patches in question only, rather than the entire series. Please indicate
if it was meant the other way around, as then the final 3 patches could also
go in.

Jan

On 06.03.25 14:13, Jan Beulich wrote:
> On 06.03.2025 00:32, Stefano Stabellini wrote:
>> On Wed, 5 Mar 2025, Juergen Gross wrote:
>>> On 25.02.25 12:10, Juergen Gross wrote:
>>>> Ping? Especially ...
>>>>
>>>> On 04.02.25 12:33, Juergen Gross wrote:
>>>>> Xenstored is using libxenctrl for only one purpose: to get information
>>>>> about state of domains.
>>>>>
>>>>> This patch series is removing that dependency by introducing a new
>>>>> stable interface which can be used by xenstored instead.
>>>>>
>>>>> There was a RFC series sent out 3 years ago, which I have taken as a
>>>>> base and by addressing all comments from back then.
>>>>>
>>>>> The main differences since that RFC series are:
>>>>>
>>>>> - Instead of introducing an new main hypercall for a stable management
>>>>>     interface I have just added a new domctl sub-op, as requested in 2021.
>>>>>
>>>>> - I have added a new library libxenmanage for easy use of the new
>>>>>     stable hypervisor interface. Main motivation for adding the library
>>>>>     was the recent attempt to decouple oxenstored from the Xen git tree.
>>>>>     By using the new library, oxenstored could benefit in the same way as
>>>>>     xenstored from the new interface: it would be possible to rely on
>>>>>     stable libraries only.
>>>>>
>>>>> - Mini-OS has gained some more config options recently, so it was rather
>>>>>     easy to make xenstore[pvh]-stubdom independent from libxenctrl, too.
>>>>>
>>>>> Please note that the last 4 patches can be committed only after the
>>>>> related Mini-OS patch "config: add support for libxenmanage" has gone in
>>>>> AND the Mini-OS commit-id has been updated in Config.mk accordingly! The
>>>>> Mini-OS patch has been Acked already, so it can go in as soon as patch
>>>>> 5 of this series (the one introducing libxenmanage) has been committed.
>>>>>
>>>>> As patches 1 and 2 change current behavior, Jan didn't want to give his
>>>>> Ack (he didn't reject the patch either). So I'm asking other "Rest"
>>>>> maintainers to look at those patches specifically.
>>>>
>>>> ... patches 1 and 2 could need an additional opinion.
>>>
>>> And another ping.
>>>
>>> One of Andrew, Stefano, Julien, Roger, Anthony, Mical: Please have a look.
>>>
>>> The related discussion between Jan and me can be found here (patches 2 and 3):
>>>
>>> https://lore.kernel.org/xen-devel/20250107101711.5980-1-jgross@suse.com/
>>
>> I didn't do an in-details review but based on Jan's comments and your
>> replies, I think they are an improvement. If someone else wants to do a
>> proper review, they would be welcome.
>>
>> Acked-by: Stefano Stabellini <sstabellini@kernel.org>
> 
> I've taken the conservative approach and interpreted this as an ack for the
> two patches in question only, rather than the entire series. Please indicate
> if it was meant the other way around, as then the final 3 patches could also
> go in.

Sorry, but please revert the last patch of this series you've committed
already. As stated in the cover letter AND that patch, a Mini-OS patch and
the bump of the Mini-OS commit in Config.mk are required in order to avoid
build failures when trying to build the Xenstore-stubdom binaries.


Juergen

On 06.03.2025 14:27, Jürgen Groß wrote:
> On 06.03.25 14:13, Jan Beulich wrote:
>> On 06.03.2025 00:32, Stefano Stabellini wrote:
>>> On Wed, 5 Mar 2025, Juergen Gross wrote:
>>>> On 25.02.25 12:10, Juergen Gross wrote:
>>>>> Ping? Especially ...
>>>>>
>>>>> On 04.02.25 12:33, Juergen Gross wrote:
>>>>>> Xenstored is using libxenctrl for only one purpose: to get information
>>>>>> about state of domains.
>>>>>>
>>>>>> This patch series is removing that dependency by introducing a new
>>>>>> stable interface which can be used by xenstored instead.
>>>>>>
>>>>>> There was a RFC series sent out 3 years ago, which I have taken as a
>>>>>> base and by addressing all comments from back then.
>>>>>>
>>>>>> The main differences since that RFC series are:
>>>>>>
>>>>>> - Instead of introducing an new main hypercall for a stable management
>>>>>>     interface I have just added a new domctl sub-op, as requested in 2021.
>>>>>>
>>>>>> - I have added a new library libxenmanage for easy use of the new
>>>>>>     stable hypervisor interface. Main motivation for adding the library
>>>>>>     was the recent attempt to decouple oxenstored from the Xen git tree.
>>>>>>     By using the new library, oxenstored could benefit in the same way as
>>>>>>     xenstored from the new interface: it would be possible to rely on
>>>>>>     stable libraries only.
>>>>>>
>>>>>> - Mini-OS has gained some more config options recently, so it was rather
>>>>>>     easy to make xenstore[pvh]-stubdom independent from libxenctrl, too.
>>>>>>
>>>>>> Please note that the last 4 patches can be committed only after the
>>>>>> related Mini-OS patch "config: add support for libxenmanage" has gone in
>>>>>> AND the Mini-OS commit-id has been updated in Config.mk accordingly! The
>>>>>> Mini-OS patch has been Acked already, so it can go in as soon as patch
>>>>>> 5 of this series (the one introducing libxenmanage) has been committed.
>>>>>>
>>>>>> As patches 1 and 2 change current behavior, Jan didn't want to give his
>>>>>> Ack (he didn't reject the patch either). So I'm asking other "Rest"
>>>>>> maintainers to look at those patches specifically.
>>>>>
>>>>> ... patches 1 and 2 could need an additional opinion.
>>>>
>>>> And another ping.
>>>>
>>>> One of Andrew, Stefano, Julien, Roger, Anthony, Mical: Please have a look.
>>>>
>>>> The related discussion between Jan and me can be found here (patches 2 and 3):
>>>>
>>>> https://lore.kernel.org/xen-devel/20250107101711.5980-1-jgross@suse.com/
>>>
>>> I didn't do an in-details review but based on Jan's comments and your
>>> replies, I think they are an improvement. If someone else wants to do a
>>> proper review, they would be welcome.
>>>
>>> Acked-by: Stefano Stabellini <sstabellini@kernel.org>
>>
>> I've taken the conservative approach and interpreted this as an ack for the
>> two patches in question only, rather than the entire series. Please indicate
>> if it was meant the other way around, as then the final 3 patches could also
>> go in.
> 
> Sorry, but please revert the last patch of this series you've committed
> already. As stated in the cover letter AND that patch, a Mini-OS patch and
> the bump of the Mini-OS commit in Config.mk are required in order to avoid
> build failures when trying to build the Xenstore-stubdom binaries.

Indeed, I overlooked this while preparing what to commit (while I remember
noticing it earlier on). Still it's probably sub-optimal to have a series
split in the middle like this.

Instead of reverting, let's bump the MiniOS ref in staging instead, as you
did suggest on Matrix.

Jan

On 06.03.25 14:53, Jan Beulich wrote:
> On 06.03.2025 14:27, Jürgen Groß wrote:
>> On 06.03.25 14:13, Jan Beulich wrote:
>>> On 06.03.2025 00:32, Stefano Stabellini wrote:
>>>> On Wed, 5 Mar 2025, Juergen Gross wrote:
>>>>> On 25.02.25 12:10, Juergen Gross wrote:
>>>>>> Ping? Especially ...
>>>>>>
>>>>>> On 04.02.25 12:33, Juergen Gross wrote:
>>>>>>> Xenstored is using libxenctrl for only one purpose: to get information
>>>>>>> about state of domains.
>>>>>>>
>>>>>>> This patch series is removing that dependency by introducing a new
>>>>>>> stable interface which can be used by xenstored instead.
>>>>>>>
>>>>>>> There was a RFC series sent out 3 years ago, which I have taken as a
>>>>>>> base and by addressing all comments from back then.
>>>>>>>
>>>>>>> The main differences since that RFC series are:
>>>>>>>
>>>>>>> - Instead of introducing an new main hypercall for a stable management
>>>>>>>      interface I have just added a new domctl sub-op, as requested in 2021.
>>>>>>>
>>>>>>> - I have added a new library libxenmanage for easy use of the new
>>>>>>>      stable hypervisor interface. Main motivation for adding the library
>>>>>>>      was the recent attempt to decouple oxenstored from the Xen git tree.
>>>>>>>      By using the new library, oxenstored could benefit in the same way as
>>>>>>>      xenstored from the new interface: it would be possible to rely on
>>>>>>>      stable libraries only.
>>>>>>>
>>>>>>> - Mini-OS has gained some more config options recently, so it was rather
>>>>>>>      easy to make xenstore[pvh]-stubdom independent from libxenctrl, too.
>>>>>>>
>>>>>>> Please note that the last 4 patches can be committed only after the
>>>>>>> related Mini-OS patch "config: add support for libxenmanage" has gone in
>>>>>>> AND the Mini-OS commit-id has been updated in Config.mk accordingly! The
>>>>>>> Mini-OS patch has been Acked already, so it can go in as soon as patch
>>>>>>> 5 of this series (the one introducing libxenmanage) has been committed.
>>>>>>>
>>>>>>> As patches 1 and 2 change current behavior, Jan didn't want to give his
>>>>>>> Ack (he didn't reject the patch either). So I'm asking other "Rest"
>>>>>>> maintainers to look at those patches specifically.
>>>>>>
>>>>>> ... patches 1 and 2 could need an additional opinion.
>>>>>
>>>>> And another ping.
>>>>>
>>>>> One of Andrew, Stefano, Julien, Roger, Anthony, Mical: Please have a look.
>>>>>
>>>>> The related discussion between Jan and me can be found here (patches 2 and 3):
>>>>>
>>>>> https://lore.kernel.org/xen-devel/20250107101711.5980-1-jgross@suse.com/
>>>>
>>>> I didn't do an in-details review but based on Jan's comments and your
>>>> replies, I think they are an improvement. If someone else wants to do a
>>>> proper review, they would be welcome.
>>>>
>>>> Acked-by: Stefano Stabellini <sstabellini@kernel.org>
>>>
>>> I've taken the conservative approach and interpreted this as an ack for the
>>> two patches in question only, rather than the entire series. Please indicate
>>> if it was meant the other way around, as then the final 3 patches could also
>>> go in.
>>
>> Sorry, but please revert the last patch of this series you've committed
>> already. As stated in the cover letter AND that patch, a Mini-OS patch and
>> the bump of the Mini-OS commit in Config.mk are required in order to avoid
>> build failures when trying to build the Xenstore-stubdom binaries.
> 
> Indeed, I overlooked this while preparing what to commit (while I remember
> noticing it earlier on). Still it's probably sub-optimal to have a series
> split in the middle like this.

In future I'll use 2 series in such a case.

> 
> Instead of reverting, let's bump the MiniOS ref in staging instead, as you
> did suggest on Matrix.

Yeah, patch has been sent already.


Juergen