tools/include/xenctrl.h | 26 +++++----------------- tools/libs/ctrl/xc_domain.c | 26 ++++++++++------------ tools/libs/light/libxl_sched.c | 9 ++++++++ xen/common/sched/core.c | 40 ++++++++-------------------------- xen/include/public/domctl.h | 9 ++------ 5 files changed, 38 insertions(+), 72 deletions(-)
Today the XEN_DOMCTL_setvcpuaffinity hypercall will not only change the affinity settings of a vcpu, it will return the new effective settings, too. In case an error happens during this hypercall, the expectation of the caller is that the affinity of the vcpu didn't change. This isn't true, however, if passing the new effective affinity back to the user is failing. Instead of making error handling in the hypervisor even more complex, just change the interface by NOT passing back the new affinity. It can easily be obtained by another XEN_DOMCTL_getvcpuaffinity call if needed. Juergen Gross (3): tools/libxl: don't rely on xc_vcpu_setaffinity() returned cpumaps xen: don't let XEN_DOMCTL_setvcpuaffinity return the new affinities tools/libxenctrl: rename parameters of xc_vcpu_setaffinity() tools/include/xenctrl.h | 26 +++++----------------- tools/libs/ctrl/xc_domain.c | 26 ++++++++++------------ tools/libs/light/libxl_sched.c | 9 ++++++++ xen/common/sched/core.c | 40 ++++++++-------------------------- xen/include/public/domctl.h | 9 ++------ 5 files changed, 38 insertions(+), 72 deletions(-) -- 2.54.0
On 01.07.2026 13:16, Juergen Gross wrote: > Today the XEN_DOMCTL_setvcpuaffinity hypercall will not only change the > affinity settings of a vcpu, it will return the new effective settings, > too. > > In case an error happens during this hypercall, the expectation of the > caller is that the affinity of the vcpu didn't change. This isn't true, > however, if passing the new effective affinity back to the user is > failing. That'll be the caller's fault though. Any -EFAULT coming back are an indication that the caller needs fixing. > Instead of making error handling in the hypervisor even more complex, > just change the interface by NOT passing back the new affinity. It can > easily be obtained by another XEN_DOMCTL_getvcpuaffinity call if > needed. At the expense of further increasing the pressure on the domctl lock. When making the recent error code change in this area, I was actually wondering in how far these two sub-ops might also be candidates for moving out of the locked region. Yet with vcpu_affinity_domctl() not using any locking that didn't look entirely straightforward to answer. Jan
On 01.07.26 13:39, Jan Beulich wrote: > On 01.07.2026 13:16, Juergen Gross wrote: >> Today the XEN_DOMCTL_setvcpuaffinity hypercall will not only change the >> affinity settings of a vcpu, it will return the new effective settings, >> too. >> >> In case an error happens during this hypercall, the expectation of the >> caller is that the affinity of the vcpu didn't change. This isn't true, >> however, if passing the new effective affinity back to the user is >> failing. > > That'll be the caller's fault though. Any -EFAULT coming back are an > indication that the caller needs fixing. The possible -ENOMEM isn't the callers fault, though. Juergen
On 01.07.2026 17:34, Juergen Gross wrote: > On 01.07.26 13:39, Jan Beulich wrote: >> On 01.07.2026 13:16, Juergen Gross wrote: >>> Today the XEN_DOMCTL_setvcpuaffinity hypercall will not only change the >>> affinity settings of a vcpu, it will return the new effective settings, >>> too. >>> >>> In case an error happens during this hypercall, the expectation of the >>> caller is that the affinity of the vcpu didn't change. This isn't true, >>> however, if passing the new effective affinity back to the user is >>> failing. >> >> That'll be the caller's fault though. Any -EFAULT coming back are an >> indication that the caller needs fixing. > > The possible -ENOMEM isn't the callers fault, though. Yet that'll happen ahead of any changes made, won't it? Jan
On 02.07.26 07:54, Jan Beulich wrote: > On 01.07.2026 17:34, Juergen Gross wrote: >> On 01.07.26 13:39, Jan Beulich wrote: >>> On 01.07.2026 13:16, Juergen Gross wrote: >>>> Today the XEN_DOMCTL_setvcpuaffinity hypercall will not only change the >>>> affinity settings of a vcpu, it will return the new effective settings, >>>> too. >>>> >>>> In case an error happens during this hypercall, the expectation of the >>>> caller is that the affinity of the vcpu didn't change. This isn't true, >>>> however, if passing the new effective affinity back to the user is >>>> failing. >>> >>> That'll be the caller's fault though. Any -EFAULT coming back are an >>> indication that the caller needs fixing. >> >> The possible -ENOMEM isn't the callers fault, though. > > Yet that'll happen ahead of any changes made, won't it? No, it can happen during the final call of cpumask_to_xenctl_bitmap(). Juergen
On 02.07.2026 08:13, Jürgen Groß wrote: > On 02.07.26 07:54, Jan Beulich wrote: >> On 01.07.2026 17:34, Juergen Gross wrote: >>> On 01.07.26 13:39, Jan Beulich wrote: >>>> On 01.07.2026 13:16, Juergen Gross wrote: >>>>> Today the XEN_DOMCTL_setvcpuaffinity hypercall will not only change the >>>>> affinity settings of a vcpu, it will return the new effective settings, >>>>> too. >>>>> >>>>> In case an error happens during this hypercall, the expectation of the >>>>> caller is that the affinity of the vcpu didn't change. This isn't true, >>>>> however, if passing the new effective affinity back to the user is >>>>> failing. >>>> >>>> That'll be the caller's fault though. Any -EFAULT coming back are an >>>> indication that the caller needs fixing. >>> >>> The possible -ENOMEM isn't the callers fault, though. >> >> Yet that'll happen ahead of any changes made, won't it? > > No, it can happen during the final call of cpumask_to_xenctl_bitmap(). Only hypothetically, on a big-endian architecture (which we don't currently have support for anywhere, hence why the little endian special case was added there). Jan
On 02.07.26 08:25, Jan Beulich wrote: > On 02.07.2026 08:13, Jürgen Groß wrote: >> On 02.07.26 07:54, Jan Beulich wrote: >>> On 01.07.2026 17:34, Juergen Gross wrote: >>>> On 01.07.26 13:39, Jan Beulich wrote: >>>>> On 01.07.2026 13:16, Juergen Gross wrote: >>>>>> Today the XEN_DOMCTL_setvcpuaffinity hypercall will not only change the >>>>>> affinity settings of a vcpu, it will return the new effective settings, >>>>>> too. >>>>>> >>>>>> In case an error happens during this hypercall, the expectation of the >>>>>> caller is that the affinity of the vcpu didn't change. This isn't true, >>>>>> however, if passing the new effective affinity back to the user is >>>>>> failing. >>>>> >>>>> That'll be the caller's fault though. Any -EFAULT coming back are an >>>>> indication that the caller needs fixing. >>>> >>>> The possible -ENOMEM isn't the callers fault, though. >>> >>> Yet that'll happen ahead of any changes made, won't it? >> >> No, it can happen during the final call of cpumask_to_xenctl_bitmap(). > > Only hypothetically, on a big-endian architecture (which we don't currently > have support for anywhere, hence why the little endian special case was > added there). Oh, indeed. Juergen
On 01.07.26 13:39, Jan Beulich wrote:
> On 01.07.2026 13:16, Juergen Gross wrote:
>> Today the XEN_DOMCTL_setvcpuaffinity hypercall will not only change the
>> affinity settings of a vcpu, it will return the new effective settings,
>> too.
>>
>> In case an error happens during this hypercall, the expectation of the
>> caller is that the affinity of the vcpu didn't change. This isn't true,
>> however, if passing the new effective affinity back to the user is
>> failing.
>
> That'll be the caller's fault though. Any -EFAULT coming back are an
> indication that the caller needs fixing.
>
>> Instead of making error handling in the hypervisor even more complex,
>> just change the interface by NOT passing back the new affinity. It can
>> easily be obtained by another XEN_DOMCTL_getvcpuaffinity call if
>> needed.
>
> At the expense of further increasing the pressure on the domctl lock.
> When making the recent error code change in this area, I was actually
> wondering in how far these two sub-ops might also be candidates for
> moving out of the locked region. Yet with vcpu_affinity_domctl() not
> using any locking that didn't look entirely straightforward to answer.
The calling hierarchy for the XEN_DOMCTL_setvcpuaffinity case is:
vcpu_affinity_domctl()
vcpu_set_hard_affinity()
vcpu_set_affinity()
vcpu_set_affinity()
vcpu_set_affinity() does the correct locking (takes the unit lock).
For not needing the domctl lock the XEN_DOMCTL_getvcpuaffinity path
would need to take the unit lock, too.
Juergen
On 01.07.2026 15:08, Jürgen Groß wrote: > On 01.07.26 13:39, Jan Beulich wrote: >> On 01.07.2026 13:16, Juergen Gross wrote: >>> Today the XEN_DOMCTL_setvcpuaffinity hypercall will not only change the >>> affinity settings of a vcpu, it will return the new effective settings, >>> too. >>> >>> In case an error happens during this hypercall, the expectation of the >>> caller is that the affinity of the vcpu didn't change. This isn't true, >>> however, if passing the new effective affinity back to the user is >>> failing. >> >> That'll be the caller's fault though. Any -EFAULT coming back are an >> indication that the caller needs fixing. >> >>> Instead of making error handling in the hypervisor even more complex, >>> just change the interface by NOT passing back the new affinity. It can >>> easily be obtained by another XEN_DOMCTL_getvcpuaffinity call if >>> needed. >> >> At the expense of further increasing the pressure on the domctl lock. >> When making the recent error code change in this area, I was actually >> wondering in how far these two sub-ops might also be candidates for >> moving out of the locked region. Yet with vcpu_affinity_domctl() not >> using any locking that didn't look entirely straightforward to answer. > > The calling hierarchy for the XEN_DOMCTL_setvcpuaffinity case is: > > vcpu_affinity_domctl() > vcpu_set_hard_affinity() > vcpu_set_affinity() > vcpu_set_affinity() > > vcpu_set_affinity() does the correct locking (takes the unit lock). But that's not sufficient: The two involved vcpu_set_affinity() calls want to stay together, i.e. not race with another domctl. Jan
On 01.07.26 15:48, Jan Beulich wrote: > On 01.07.2026 15:08, Jürgen Groß wrote: >> On 01.07.26 13:39, Jan Beulich wrote: >>> On 01.07.2026 13:16, Juergen Gross wrote: >>>> Today the XEN_DOMCTL_setvcpuaffinity hypercall will not only change the >>>> affinity settings of a vcpu, it will return the new effective settings, >>>> too. >>>> >>>> In case an error happens during this hypercall, the expectation of the >>>> caller is that the affinity of the vcpu didn't change. This isn't true, >>>> however, if passing the new effective affinity back to the user is >>>> failing. >>> >>> That'll be the caller's fault though. Any -EFAULT coming back are an >>> indication that the caller needs fixing. >>> >>>> Instead of making error handling in the hypervisor even more complex, >>>> just change the interface by NOT passing back the new affinity. It can >>>> easily be obtained by another XEN_DOMCTL_getvcpuaffinity call if >>>> needed. >>> >>> At the expense of further increasing the pressure on the domctl lock. >>> When making the recent error code change in this area, I was actually >>> wondering in how far these two sub-ops might also be candidates for >>> moving out of the locked region. Yet with vcpu_affinity_domctl() not >>> using any locking that didn't look entirely straightforward to answer. >> >> The calling hierarchy for the XEN_DOMCTL_setvcpuaffinity case is: >> >> vcpu_affinity_domctl() >> vcpu_set_hard_affinity() >> vcpu_set_affinity() >> vcpu_set_affinity() >> >> vcpu_set_affinity() does the correct locking (takes the unit lock). > > But that's not sufficient: The two involved vcpu_set_affinity() calls > want to stay together, i.e. not race with another domctl. Hmm, while I understand why this would be nice, I'm having a hard time imagining a sane scenario where two or more instances are issuing hypercalls for changing affinities. At least one of those instances will not be happy afterwards, in case the lock is split both might by unhappy. I don't see this being a substantial downside, as the main problem would be on the user side allowing multiple concurrent affinity changes for the same vcpu(s). OTOH it would be quite easy to merge both affinity settings into one locked region, as the underlying function sched_set_affinity() can be called with both affinity masks or with only one of them. This would require only a minor reorg of the affinity helpers. Juergen
On 01/07/2026 12:39 pm, Jan Beulich wrote: > On 01.07.2026 13:16, Juergen Gross wrote: >> Today the XEN_DOMCTL_setvcpuaffinity hypercall will not only change the >> affinity settings of a vcpu, it will return the new effective settings, >> too. >> >> In case an error happens during this hypercall, the expectation of the >> caller is that the affinity of the vcpu didn't change. This isn't true, >> however, if passing the new effective affinity back to the user is >> failing. > That'll be the caller's fault though. Any -EFAULT coming back are an > indication that the caller needs fixing. > >> Instead of making error handling in the hypervisor even more complex, >> just change the interface by NOT passing back the new affinity. It can >> easily be obtained by another XEN_DOMCTL_getvcpuaffinity call if >> needed. > At the expense of further increasing the pressure on the domctl lock. > When making the recent error code change in this area, I was actually > wondering in how far these two sub-ops might also be candidates for > moving out of the locked region. Yet with vcpu_affinity_domctl() not > using any locking that didn't look entirely straightforward to answer. How does the effective setting differ from what's passed in? I hadn't even realised that we pass data out of the set op, and it does smell of bad API design. While we do care about the domctl lock, I don't consider "extra pressure on it" as a valid argument to be used here. It's an internal detail, whereas the hypercall API is an external detail. ~Andrew
On 01.07.26 13:45, Andrew Cooper wrote: > On 01/07/2026 12:39 pm, Jan Beulich wrote: >> On 01.07.2026 13:16, Juergen Gross wrote: >>> Today the XEN_DOMCTL_setvcpuaffinity hypercall will not only change the >>> affinity settings of a vcpu, it will return the new effective settings, >>> too. >>> >>> In case an error happens during this hypercall, the expectation of the >>> caller is that the affinity of the vcpu didn't change. This isn't true, >>> however, if passing the new effective affinity back to the user is >>> failing. >> That'll be the caller's fault though. Any -EFAULT coming back are an >> indication that the caller needs fixing. >> >>> Instead of making error handling in the hypervisor even more complex, >>> just change the interface by NOT passing back the new affinity. It can >>> easily be obtained by another XEN_DOMCTL_getvcpuaffinity call if >>> needed. >> At the expense of further increasing the pressure on the domctl lock. >> When making the recent error code change in this area, I was actually >> wondering in how far these two sub-ops might also be candidates for >> moving out of the locked region. Yet with vcpu_affinity_domctl() not >> using any locking that didn't look entirely straightforward to answer. > > How does the effective setting differ from what's passed in? The hard affinity might differ in case the target domain is in a cpupool not covering the whole machine and e.g. "all" is passed on input. The effective setting will have only the CPUs of the cpupool set. > I hadn't even realised that we pass data out of the set op, and it does > smell of bad API design. Yes, especially as the data is used only by one caller for verification. > While we do care about the domctl lock, I don't consider "extra pressure > on it" as a valid argument to be used here. It's an internal detail, > whereas the hypercall API is an external detail. Juergen
© 2016 - 2026 Red Hat, Inc.