If run Xen with PVH dom0 and hvm domU, hvm will map a pirq for
a passthrough device by using gsi, see
xen_pt_realize->xc_physdev_map_pirq and
pci_add_dm_done->xc_physdev_map_pirq. Then xc_physdev_map_pirq
will call into Xen, but in hvm_physdev_op, PHYSDEVOP_map_pirq
is not allowed because currd is PVH dom0 and PVH has no
X86_EMU_USE_PIRQ flag, it will fail at has_pirq check.
So, allow PHYSDEVOP_map_pirq when currd is dom0 no matter if
dom0 has X86_EMU_USE_PIRQ flag and also allow
PHYSDEVOP_unmap_pirq for the failed path to unmap pirq.

What's more, in PVH dom0, the gsis don't get registered, but
the gsi of a passthrough device must be configured for it to
be able to be mapped into a hvm domU.
So, add PHYSDEVOP_setup_gsi for PVH dom0, because PVH dom0
will setup gsi during assigning a device to passthrough.

Co-developed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Jiqian Chen <Jiqian.Chen@amd.com>
---
 xen/arch/x86/hvm/hypercall.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/xen/arch/x86/hvm/hypercall.c b/xen/arch/x86/hvm/hypercall.c
index 6ad5b4d5f1..621d789bd3 100644
--- a/xen/arch/x86/hvm/hypercall.c
+++ b/xen/arch/x86/hvm/hypercall.c
@@ -72,8 +72,11 @@ long hvm_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
 
     switch ( cmd )
     {
+    case PHYSDEVOP_setup_gsi:
     case PHYSDEVOP_map_pirq:
     case PHYSDEVOP_unmap_pirq:
+        if ( is_hardware_domain(currd) )
+            break;
     case PHYSDEVOP_eoi:
     case PHYSDEVOP_irq_status_query:
     case PHYSDEVOP_get_free_pirq:
-- 
2.34.1

On Mon, Dec 11, 2023 at 12:40:08AM +0800, Jiqian Chen wrote:
> If run Xen with PVH dom0 and hvm domU, hvm will map a pirq for
> a passthrough device by using gsi, see
> xen_pt_realize->xc_physdev_map_pirq and
> pci_add_dm_done->xc_physdev_map_pirq. Then xc_physdev_map_pirq
> will call into Xen, but in hvm_physdev_op, PHYSDEVOP_map_pirq
> is not allowed because currd is PVH dom0 and PVH has no
> X86_EMU_USE_PIRQ flag, it will fail at has_pirq check.
> So, allow PHYSDEVOP_map_pirq when currd is dom0 no matter if
> dom0 has X86_EMU_USE_PIRQ flag and also allow
> PHYSDEVOP_unmap_pirq for the failed path to unmap pirq.
> 
> What's more, in PVH dom0, the gsis don't get registered, but
> the gsi of a passthrough device must be configured for it to
> be able to be mapped into a hvm domU.
> So, add PHYSDEVOP_setup_gsi for PVH dom0, because PVH dom0
> will setup gsi during assigning a device to passthrough.
> 
> Co-developed-by: Huang Rui <ray.huang@amd.com>
> Signed-off-by: Jiqian Chen <Jiqian.Chen@amd.com>
> ---
>  xen/arch/x86/hvm/hypercall.c | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/xen/arch/x86/hvm/hypercall.c b/xen/arch/x86/hvm/hypercall.c
> index 6ad5b4d5f1..621d789bd3 100644
> --- a/xen/arch/x86/hvm/hypercall.c
> +++ b/xen/arch/x86/hvm/hypercall.c
> @@ -72,8 +72,11 @@ long hvm_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>  
>      switch ( cmd )
>      {
> +    case PHYSDEVOP_setup_gsi:

I think given the new approach on the Linux side patches, where
pciback will configure the interrupt, there's no need to expose
setup_gsi anymore?

>      case PHYSDEVOP_map_pirq:
>      case PHYSDEVOP_unmap_pirq:
> +        if ( is_hardware_domain(currd) )
> +            break;

Also Jan already pointed this out in v2: this hypercall needs to be
limited so a PVH dom0 cannot execute it against itself.  IOW: refuse
the hypercall if DOMID_SELF or the passed domid matches the current
domain domid.

Thanks, Roger.

On 2023/12/11 23:31, Roger Pau Monné wrote:
> On Mon, Dec 11, 2023 at 12:40:08AM +0800, Jiqian Chen wrote:
>> If run Xen with PVH dom0 and hvm domU, hvm will map a pirq for
>> a passthrough device by using gsi, see
>> xen_pt_realize->xc_physdev_map_pirq and
>> pci_add_dm_done->xc_physdev_map_pirq. Then xc_physdev_map_pirq
>> will call into Xen, but in hvm_physdev_op, PHYSDEVOP_map_pirq
>> is not allowed because currd is PVH dom0 and PVH has no
>> X86_EMU_USE_PIRQ flag, it will fail at has_pirq check.
>> So, allow PHYSDEVOP_map_pirq when currd is dom0 no matter if
>> dom0 has X86_EMU_USE_PIRQ flag and also allow
>> PHYSDEVOP_unmap_pirq for the failed path to unmap pirq.
>>
>> What's more, in PVH dom0, the gsis don't get registered, but
>> the gsi of a passthrough device must be configured for it to
>> be able to be mapped into a hvm domU.
>> So, add PHYSDEVOP_setup_gsi for PVH dom0, because PVH dom0
>> will setup gsi during assigning a device to passthrough.
>>
>> Co-developed-by: Huang Rui <ray.huang@amd.com>
>> Signed-off-by: Jiqian Chen <Jiqian.Chen@amd.com>
>> ---
>>  xen/arch/x86/hvm/hypercall.c | 3 +++
>>  1 file changed, 3 insertions(+)
>>
>> diff --git a/xen/arch/x86/hvm/hypercall.c b/xen/arch/x86/hvm/hypercall.c
>> index 6ad5b4d5f1..621d789bd3 100644
>> --- a/xen/arch/x86/hvm/hypercall.c
>> +++ b/xen/arch/x86/hvm/hypercall.c
>> @@ -72,8 +72,11 @@ long hvm_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>>  
>>      switch ( cmd )
>>      {
>> +    case PHYSDEVOP_setup_gsi:
> 
> I think given the new approach on the Linux side patches, where
> pciback will configure the interrupt, there's no need to expose
> setup_gsi anymore?
The latest patch(the second patch of v3 on kernel side) does setup_gsi and map_pirq for passthrough device in pciback, so we need this and below.

> 
>>      case PHYSDEVOP_map_pirq:
>>      case PHYSDEVOP_unmap_pirq:
>> +        if ( is_hardware_domain(currd) )
>> +            break;
> 
> Also Jan already pointed this out in v2: this hypercall needs to be
> limited so a PVH dom0 cannot execute it against itself.  IOW: refuse
> the hypercall if DOMID_SELF or the passed domid matches the current
> domain domid.
Yes, I remember Jan's suggestion, but since the latest patch(the second patch of v3 on kernel side) has change the implementation, it does setup_gsi and map_pirq for dom0 itself, so I didn't add the DOMID_SELF check.

> 
> Thanks, Roger.

-- 
Best regards,
Jiqian Chen.

On 12.12.2023 07:49, Chen, Jiqian wrote:
> On 2023/12/11 23:31, Roger Pau Monné wrote:
>> On Mon, Dec 11, 2023 at 12:40:08AM +0800, Jiqian Chen wrote:
>>> --- a/xen/arch/x86/hvm/hypercall.c
>>> +++ b/xen/arch/x86/hvm/hypercall.c
>>> @@ -72,8 +72,11 @@ long hvm_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>>>  
>>>      switch ( cmd )
>>>      {
>>> +    case PHYSDEVOP_setup_gsi:
>>
>> I think given the new approach on the Linux side patches, where
>> pciback will configure the interrupt, there's no need to expose
>> setup_gsi anymore?
> The latest patch(the second patch of v3 on kernel side) does setup_gsi and map_pirq for passthrough device in pciback, so we need this and below.
> 
>>
>>>      case PHYSDEVOP_map_pirq:
>>>      case PHYSDEVOP_unmap_pirq:
>>> +        if ( is_hardware_domain(currd) )
>>> +            break;
>>
>> Also Jan already pointed this out in v2: this hypercall needs to be
>> limited so a PVH dom0 cannot execute it against itself.  IOW: refuse
>> the hypercall if DOMID_SELF or the passed domid matches the current
>> domain domid.
> Yes, I remember Jan's suggestion, but since the latest patch(the second patch of v3 on kernel side) has change the implementation, it does setup_gsi and map_pirq for dom0 itself, so I didn't add the DOMID_SELF check.

And why exactly would it do specifically the map_pirq? (Even the setup_gsi
looks questionable to me, but there might be reasons there.)

Jan

On 2023/12/12 17:30, Jan Beulich wrote:
> On 12.12.2023 07:49, Chen, Jiqian wrote:
>> On 2023/12/11 23:31, Roger Pau Monné wrote:
>>> On Mon, Dec 11, 2023 at 12:40:08AM +0800, Jiqian Chen wrote:
>>>> --- a/xen/arch/x86/hvm/hypercall.c
>>>> +++ b/xen/arch/x86/hvm/hypercall.c
>>>> @@ -72,8 +72,11 @@ long hvm_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>>>>  
>>>>      switch ( cmd )
>>>>      {
>>>> +    case PHYSDEVOP_setup_gsi:
>>>
>>> I think given the new approach on the Linux side patches, where
>>> pciback will configure the interrupt, there's no need to expose
>>> setup_gsi anymore?
>> The latest patch(the second patch of v3 on kernel side) does setup_gsi and map_pirq for passthrough device in pciback, so we need this and below.
>>
>>>
>>>>      case PHYSDEVOP_map_pirq:
>>>>      case PHYSDEVOP_unmap_pirq:
>>>> +        if ( is_hardware_domain(currd) )
>>>> +            break;
>>>
>>> Also Jan already pointed this out in v2: this hypercall needs to be
>>> limited so a PVH dom0 cannot execute it against itself.  IOW: refuse
>>> the hypercall if DOMID_SELF or the passed domid matches the current
>>> domain domid.
>> Yes, I remember Jan's suggestion, but since the latest patch(the second patch of v3 on kernel side) has change the implementation, it does setup_gsi and map_pirq for dom0 itself, so I didn't add the DOMID_SELF check.
> 
> And why exactly would it do specifically the map_pirq? (Even the setup_gsi
> looks questionable to me, but there might be reasons there.)
Map_pirq is to solve the check failure problem. (pci_add_dm_done-> xc_domain_irq_permission-> XEN_DOMCTL_irq_permission-> pirq_access_permitted->domain_pirq_to_irq->return irq is 0)
Setup_gsi is because the gsi is never be unmasked, so the gsi is never be registered( vioapic_hwdom_map_gsi-> mp_register_gsi is never be called).

> 
> Jan

-- 
Best regards,
Jiqian Chen.

On 13.12.2023 03:47, Chen, Jiqian wrote:
> On 2023/12/12 17:30, Jan Beulich wrote:
>> On 12.12.2023 07:49, Chen, Jiqian wrote:
>>> On 2023/12/11 23:31, Roger Pau Monné wrote:
>>>> On Mon, Dec 11, 2023 at 12:40:08AM +0800, Jiqian Chen wrote:
>>>>> --- a/xen/arch/x86/hvm/hypercall.c
>>>>> +++ b/xen/arch/x86/hvm/hypercall.c
>>>>> @@ -72,8 +72,11 @@ long hvm_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>>>>>  
>>>>>      switch ( cmd )
>>>>>      {
>>>>> +    case PHYSDEVOP_setup_gsi:
>>>>
>>>> I think given the new approach on the Linux side patches, where
>>>> pciback will configure the interrupt, there's no need to expose
>>>> setup_gsi anymore?
>>> The latest patch(the second patch of v3 on kernel side) does setup_gsi and map_pirq for passthrough device in pciback, so we need this and below.
>>>
>>>>
>>>>>      case PHYSDEVOP_map_pirq:
>>>>>      case PHYSDEVOP_unmap_pirq:
>>>>> +        if ( is_hardware_domain(currd) )
>>>>> +            break;
>>>>
>>>> Also Jan already pointed this out in v2: this hypercall needs to be
>>>> limited so a PVH dom0 cannot execute it against itself.  IOW: refuse
>>>> the hypercall if DOMID_SELF or the passed domid matches the current
>>>> domain domid.
>>> Yes, I remember Jan's suggestion, but since the latest patch(the second patch of v3 on kernel side) has change the implementation, it does setup_gsi and map_pirq for dom0 itself, so I didn't add the DOMID_SELF check.
>>
>> And why exactly would it do specifically the map_pirq? (Even the setup_gsi
>> looks questionable to me, but there might be reasons there.)
> Map_pirq is to solve the check failure problem. (pci_add_dm_done-> xc_domain_irq_permission-> XEN_DOMCTL_irq_permission-> pirq_access_permitted->domain_pirq_to_irq->return irq is 0)
> Setup_gsi is because the gsi is never be unmasked, so the gsi is never be registered( vioapic_hwdom_map_gsi-> mp_register_gsi is never be called).

And it was previously made pretty clear by Roger, I think, that doing a "map"
just for the purpose of granting permission is, well, at best a temporary
workaround in the early development phase. If there's presently no hypercall
to _only_ grant permission to IRQ, we need to add one. In fact "map" would
likely better not have done two things at a time from the very beginning ...

Jan

On 2023/12/13 15:03, Jan Beulich wrote:
> On 13.12.2023 03:47, Chen, Jiqian wrote:
>> On 2023/12/12 17:30, Jan Beulich wrote:
>>> On 12.12.2023 07:49, Chen, Jiqian wrote:
>>>> On 2023/12/11 23:31, Roger Pau Monné wrote:
>>>>> On Mon, Dec 11, 2023 at 12:40:08AM +0800, Jiqian Chen wrote:
>>>>>> --- a/xen/arch/x86/hvm/hypercall.c
>>>>>> +++ b/xen/arch/x86/hvm/hypercall.c
>>>>>> @@ -72,8 +72,11 @@ long hvm_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>>>>>>  
>>>>>>      switch ( cmd )
>>>>>>      {
>>>>>> +    case PHYSDEVOP_setup_gsi:
>>>>>
>>>>> I think given the new approach on the Linux side patches, where
>>>>> pciback will configure the interrupt, there's no need to expose
>>>>> setup_gsi anymore?
>>>> The latest patch(the second patch of v3 on kernel side) does setup_gsi and map_pirq for passthrough device in pciback, so we need this and below.
>>>>
>>>>>
>>>>>>      case PHYSDEVOP_map_pirq:
>>>>>>      case PHYSDEVOP_unmap_pirq:
>>>>>> +        if ( is_hardware_domain(currd) )
>>>>>> +            break;
>>>>>
>>>>> Also Jan already pointed this out in v2: this hypercall needs to be
>>>>> limited so a PVH dom0 cannot execute it against itself.  IOW: refuse
>>>>> the hypercall if DOMID_SELF or the passed domid matches the current
>>>>> domain domid.
>>>> Yes, I remember Jan's suggestion, but since the latest patch(the second patch of v3 on kernel side) has change the implementation, it does setup_gsi and map_pirq for dom0 itself, so I didn't add the DOMID_SELF check.
>>>
>>> And why exactly would it do specifically the map_pirq? (Even the setup_gsi
>>> looks questionable to me, but there might be reasons there.)
>> Map_pirq is to solve the check failure problem. (pci_add_dm_done-> xc_domain_irq_permission-> XEN_DOMCTL_irq_permission-> pirq_access_permitted->domain_pirq_to_irq->return irq is 0)
>> Setup_gsi is because the gsi is never be unmasked, so the gsi is never be registered( vioapic_hwdom_map_gsi-> mp_register_gsi is never be called).
> 
> And it was previously made pretty clear by Roger, I think, that doing a "map"
> just for the purpose of granting permission is, well, at best a temporary
> workaround in the early development phase. If there's presently no hypercall
> to _only_ grant permission to IRQ, we need to add one.
Could you please describe it in detail? Do you mean to add a new hypercall to grant irq access for dom0 or domU?
It seems XEN_DOMCTL_irq_permission is the hypercall to grant irq access from dom0 to domU(see XEN_DOMCTL_irq_permission-> irq_permit_access). There is no need to add hypercall to grant irq access.
We failed here (XEN_DOMCTL_irq_permission-> pirq_access_permitted->domain_pirq_to_irq->return irq is 0) is because the PVH dom0 didn't use PIRQ, so we can't get irq from pirq if "current" is PVH dom0.
So, it seems the logic of XEN_DOMCTL_irq_permission is not suitable when PVH dom0? Maybe it directly needs to get irq from the caller(domU) instead of "current" if the "current" has no PIRQ flag?

> In fact "map" would likely better not have done two things at a time from the very beginning ...
> 
> Jan

-- 
Best regards,
Jiqian Chen.

On Thu, Dec 14, 2023 at 08:55:45AM +0000, Chen, Jiqian wrote:
> On 2023/12/13 15:03, Jan Beulich wrote:
> > On 13.12.2023 03:47, Chen, Jiqian wrote:
> >> On 2023/12/12 17:30, Jan Beulich wrote:
> >>> On 12.12.2023 07:49, Chen, Jiqian wrote:
> >>>> On 2023/12/11 23:31, Roger Pau Monné wrote:
> >>>>> On Mon, Dec 11, 2023 at 12:40:08AM +0800, Jiqian Chen wrote:
> >>>>>> --- a/xen/arch/x86/hvm/hypercall.c
> >>>>>> +++ b/xen/arch/x86/hvm/hypercall.c
> >>>>>> @@ -72,8 +72,11 @@ long hvm_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
> >>>>>>  
> >>>>>>      switch ( cmd )
> >>>>>>      {
> >>>>>> +    case PHYSDEVOP_setup_gsi:
> >>>>>
> >>>>> I think given the new approach on the Linux side patches, where
> >>>>> pciback will configure the interrupt, there's no need to expose
> >>>>> setup_gsi anymore?
> >>>> The latest patch(the second patch of v3 on kernel side) does setup_gsi and map_pirq for passthrough device in pciback, so we need this and below.
> >>>>
> >>>>>
> >>>>>>      case PHYSDEVOP_map_pirq:
> >>>>>>      case PHYSDEVOP_unmap_pirq:
> >>>>>> +        if ( is_hardware_domain(currd) )
> >>>>>> +            break;
> >>>>>
> >>>>> Also Jan already pointed this out in v2: this hypercall needs to be
> >>>>> limited so a PVH dom0 cannot execute it against itself.  IOW: refuse
> >>>>> the hypercall if DOMID_SELF or the passed domid matches the current
> >>>>> domain domid.
> >>>> Yes, I remember Jan's suggestion, but since the latest patch(the second patch of v3 on kernel side) has change the implementation, it does setup_gsi and map_pirq for dom0 itself, so I didn't add the DOMID_SELF check.
> >>>
> >>> And why exactly would it do specifically the map_pirq? (Even the setup_gsi
> >>> looks questionable to me, but there might be reasons there.)
> >> Map_pirq is to solve the check failure problem. (pci_add_dm_done-> xc_domain_irq_permission-> XEN_DOMCTL_irq_permission-> pirq_access_permitted->domain_pirq_to_irq->return irq is 0)
> >> Setup_gsi is because the gsi is never be unmasked, so the gsi is never be registered( vioapic_hwdom_map_gsi-> mp_register_gsi is never be called).
> > 
> > And it was previously made pretty clear by Roger, I think, that doing a "map"
> > just for the purpose of granting permission is, well, at best a temporary
> > workaround in the early development phase. If there's presently no hypercall
> > to _only_ grant permission to IRQ, we need to add one.
> Could you please describe it in detail? Do you mean to add a new hypercall to grant irq access for dom0 or domU?
> It seems XEN_DOMCTL_irq_permission is the hypercall to grant irq access from dom0 to domU(see XEN_DOMCTL_irq_permission-> irq_permit_access). There is no need to add hypercall to grant irq access.
> We failed here (XEN_DOMCTL_irq_permission-> pirq_access_permitted->domain_pirq_to_irq->return irq is 0) is because the PVH dom0 didn't use PIRQ, so we can't get irq from pirq if "current" is PVH dom0.

One way to bodge this would be to detect whether the caller of
XEN_DOMCTL_irq_permission is a PV or an HVM domain, and in case of HVM
assume the pirq field is a GSI.  I'm unsure however how that will work
with non-x86 architectures.

It would  be better to introduce a new XEN_DOMCTL_gsi_permission, or
maybe XEN_DOMCTL_intr_permission that can take a struct we can use to
accommodate GSIs and other arch specific interrupt identifiers.

I'm also wondering whether the hypercall should be in a stable
interface so it could be easily used from QEMU if needed.

> So, it seems the logic of XEN_DOMCTL_irq_permission is not suitable when PVH dom0? Maybe it directly needs to get irq from the caller(domU) instead of "current" if the "current" has no PIRQ flag?

Hm, I'm kind of confused by this last sentence, as you mention "the
caller(domU)".  The caller of XEN_DOMCTL_irq_permission will always be
dom0 or the hardware domain.

Thanks, Roger.

On 14.12.2023 10:55, Roger Pau Monné wrote:
> On Thu, Dec 14, 2023 at 08:55:45AM +0000, Chen, Jiqian wrote:
>> On 2023/12/13 15:03, Jan Beulich wrote:
>>> On 13.12.2023 03:47, Chen, Jiqian wrote:
>>>> On 2023/12/12 17:30, Jan Beulich wrote:
>>>>> On 12.12.2023 07:49, Chen, Jiqian wrote:
>>>>>> On 2023/12/11 23:31, Roger Pau Monné wrote:
>>>>>>> On Mon, Dec 11, 2023 at 12:40:08AM +0800, Jiqian Chen wrote:
>>>>>>>> --- a/xen/arch/x86/hvm/hypercall.c
>>>>>>>> +++ b/xen/arch/x86/hvm/hypercall.c
>>>>>>>> @@ -72,8 +72,11 @@ long hvm_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>>>>>>>>  
>>>>>>>>      switch ( cmd )
>>>>>>>>      {
>>>>>>>> +    case PHYSDEVOP_setup_gsi:
>>>>>>>
>>>>>>> I think given the new approach on the Linux side patches, where
>>>>>>> pciback will configure the interrupt, there's no need to expose
>>>>>>> setup_gsi anymore?
>>>>>> The latest patch(the second patch of v3 on kernel side) does setup_gsi and map_pirq for passthrough device in pciback, so we need this and below.
>>>>>>
>>>>>>>
>>>>>>>>      case PHYSDEVOP_map_pirq:
>>>>>>>>      case PHYSDEVOP_unmap_pirq:
>>>>>>>> +        if ( is_hardware_domain(currd) )
>>>>>>>> +            break;
>>>>>>>
>>>>>>> Also Jan already pointed this out in v2: this hypercall needs to be
>>>>>>> limited so a PVH dom0 cannot execute it against itself.  IOW: refuse
>>>>>>> the hypercall if DOMID_SELF or the passed domid matches the current
>>>>>>> domain domid.
>>>>>> Yes, I remember Jan's suggestion, but since the latest patch(the second patch of v3 on kernel side) has change the implementation, it does setup_gsi and map_pirq for dom0 itself, so I didn't add the DOMID_SELF check.
>>>>>
>>>>> And why exactly would it do specifically the map_pirq? (Even the setup_gsi
>>>>> looks questionable to me, but there might be reasons there.)
>>>> Map_pirq is to solve the check failure problem. (pci_add_dm_done-> xc_domain_irq_permission-> XEN_DOMCTL_irq_permission-> pirq_access_permitted->domain_pirq_to_irq->return irq is 0)
>>>> Setup_gsi is because the gsi is never be unmasked, so the gsi is never be registered( vioapic_hwdom_map_gsi-> mp_register_gsi is never be called).
>>>
>>> And it was previously made pretty clear by Roger, I think, that doing a "map"
>>> just for the purpose of granting permission is, well, at best a temporary
>>> workaround in the early development phase. If there's presently no hypercall
>>> to _only_ grant permission to IRQ, we need to add one.
>> Could you please describe it in detail? Do you mean to add a new hypercall to grant irq access for dom0 or domU?
>> It seems XEN_DOMCTL_irq_permission is the hypercall to grant irq access from dom0 to domU(see XEN_DOMCTL_irq_permission-> irq_permit_access). There is no need to add hypercall to grant irq access.
>> We failed here (XEN_DOMCTL_irq_permission-> pirq_access_permitted->domain_pirq_to_irq->return irq is 0) is because the PVH dom0 didn't use PIRQ, so we can't get irq from pirq if "current" is PVH dom0.
> 
> One way to bodge this would be to detect whether the caller of
> XEN_DOMCTL_irq_permission is a PV or an HVM domain, and in case of HVM
> assume the pirq field is a GSI.  I'm unsure however how that will work
> with non-x86 architectures.
> 
> It would  be better to introduce a new XEN_DOMCTL_gsi_permission, or
> maybe XEN_DOMCTL_intr_permission that can take a struct we can use to
> accommodate GSIs and other arch specific interrupt identifiers.

How would you see MSI being handled then?

Jan

On 14.12.2023 09:55, Chen, Jiqian wrote:
> On 2023/12/13 15:03, Jan Beulich wrote:
>> On 13.12.2023 03:47, Chen, Jiqian wrote:
>>> On 2023/12/12 17:30, Jan Beulich wrote:
>>>> On 12.12.2023 07:49, Chen, Jiqian wrote:
>>>>> On 2023/12/11 23:31, Roger Pau Monné wrote:
>>>>>> On Mon, Dec 11, 2023 at 12:40:08AM +0800, Jiqian Chen wrote:
>>>>>>> --- a/xen/arch/x86/hvm/hypercall.c
>>>>>>> +++ b/xen/arch/x86/hvm/hypercall.c
>>>>>>> @@ -72,8 +72,11 @@ long hvm_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
>>>>>>>  
>>>>>>>      switch ( cmd )
>>>>>>>      {
>>>>>>> +    case PHYSDEVOP_setup_gsi:
>>>>>>
>>>>>> I think given the new approach on the Linux side patches, where
>>>>>> pciback will configure the interrupt, there's no need to expose
>>>>>> setup_gsi anymore?
>>>>> The latest patch(the second patch of v3 on kernel side) does setup_gsi and map_pirq for passthrough device in pciback, so we need this and below.
>>>>>
>>>>>>
>>>>>>>      case PHYSDEVOP_map_pirq:
>>>>>>>      case PHYSDEVOP_unmap_pirq:
>>>>>>> +        if ( is_hardware_domain(currd) )
>>>>>>> +            break;
>>>>>>
>>>>>> Also Jan already pointed this out in v2: this hypercall needs to be
>>>>>> limited so a PVH dom0 cannot execute it against itself.  IOW: refuse
>>>>>> the hypercall if DOMID_SELF or the passed domid matches the current
>>>>>> domain domid.
>>>>> Yes, I remember Jan's suggestion, but since the latest patch(the second patch of v3 on kernel side) has change the implementation, it does setup_gsi and map_pirq for dom0 itself, so I didn't add the DOMID_SELF check.
>>>>
>>>> And why exactly would it do specifically the map_pirq? (Even the setup_gsi
>>>> looks questionable to me, but there might be reasons there.)
>>> Map_pirq is to solve the check failure problem. (pci_add_dm_done-> xc_domain_irq_permission-> XEN_DOMCTL_irq_permission-> pirq_access_permitted->domain_pirq_to_irq->return irq is 0)
>>> Setup_gsi is because the gsi is never be unmasked, so the gsi is never be registered( vioapic_hwdom_map_gsi-> mp_register_gsi is never be called).
>>
>> And it was previously made pretty clear by Roger, I think, that doing a "map"
>> just for the purpose of granting permission is, well, at best a temporary
>> workaround in the early development phase. If there's presently no hypercall
>> to _only_ grant permission to IRQ, we need to add one.
> Could you please describe it in detail? Do you mean to add a new hypercall to grant irq access for dom0 or domU?
> It seems XEN_DOMCTL_irq_permission is the hypercall to grant irq access from dom0 to domU(see XEN_DOMCTL_irq_permission-> irq_permit_access). There is no need to add hypercall to grant irq access.

Hmm, yes and no. May I turn your attention to
https://lists.xen.org/archives/html/xen-devel/2023-07/msg02056.html
and its earlier version
https://lists.xen.org/archives/html/xen-devel/2023-05/msg00301.html
(it's imo a shame that this series continues to be stuck)?

Both make pretty clear that without pIRQ, this domctl cannot be used in
its present shape anyway, for ...

> We failed here (XEN_DOMCTL_irq_permission-> pirq_access_permitted->domain_pirq_to_irq->return irq is 0) is because the PVH dom0 didn't use PIRQ, so we can't get irq from pirq if "current" is PVH dom0.

... this very reason. Addressing this one way or another is a necessary
part of making passthrough work with PVH Dom0. So _effectively_ there is
no hypercall allowing PVH Dom0 to grant IRQ permission.

> So, it seems the logic of XEN_DOMCTL_irq_permission is not suitable when PVH dom0?

That's my view, yes.

> Maybe it directly needs to get irq from the caller(domU) instead of "current" if the "current" has no PIRQ flag?

I don't think the IRQ mapping in the DomU is necessary to be known here.
What we want to grant is access to a host resource. That host resource is
therefore all that should need specifying for the operation to be carried
out. It just so happens that a PV Dom0 would specify the host IRQ by way
of supplying its own equivalent pIRQ.

Things are more "interesting" for MSI, though: The (Xen) IRQ may not be
known early enough. There wants to be a way of indicating that when such
an IRQ is created, permission should be granted to the domain that is
going to use that IRQ (by way of being assigned the respective device).
(This aspect may be part of why "map" presently also grants permission,
yet I continue to think that was wrong from the start. The more that
access there is [likely needlessly] granted to the domain requesting the
mapping, just for it to then further grant access to the DomU.)

Jan