>-----Original Message-----
>From: Clément MATHIEU--DRIF <clement.mathieu--drif@bull.com>
>Subject: Re: [PATCH v3 0/6] intel_iommu: Enable PRQ support for passthrough
>device
>
>
>On Fri, 2026-08-21 at 02:44 +0000, Duan, Zhenzhong wrote:
>> Caution: External email. Do not open attachments or click links, unless this email
>comes from a known sender and you know the content is safe.
>>
>>
>> Hi Clement,
>>
>>
>> > -----Original Message-----
>> > From: Clément MATHIEU--DRIF <[clement.mathieu--
>drif@bull.com](mailto:clement.mathieu--drif@bull.com)>
>> > Subject: Re: [PATCH v3 0/6] intel_iommu: Enable PRQ support for passthrough
>> > device
>> >
>> >
>> > On Wed, 2026-08-12 at 13:26 +0000, Duan, Zhenzhong wrote:
>> >
>> > > Caution: External email. Do not open attachments or click links, unless this
>email
>> >
>> > comes from a known sender and you know the content is safe.
>> >
>> > >
>> > >
>> > >
>> > >
>> > > > -----Original Message-----
>> > > > From: Clément MATHIEU--DRIF <[clement.mathieu--
>> > >
>> >
>> > [drif@bull.com](mailto:drif@bull.com)](mailto:[clement.mathieu--
>drif@bull.com](mailto:clement.mathieu--drif@bull.com))>
>> >
>> > >
>> > > > Subject: Re: [PATCH v3 0/6] intel_iommu: Enable PRQ support for
>passthrough
>> > > > device
>> > > >
>> > > >
>> > > > On Mon, 2026-08-10 at 10:49 +0000, Duan, Zhenzhong wrote:
>> > > >
>> > > >
>> > > > > Caution: External email. Do not open attachments or click links, unless
>this
>> > > >
>> > >
>> >
>> > email
>> >
>> > >
>> > > >
>> > > > comes from a known sender and you know the content is safe.
>> > > >
>> > > >
>> > > > >
>> > > > >
>> > > > > Hi Clement,
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > > > -----Original Message-----
>> > > > > > From: Clément MATHIEU--DRIF <[clement.mathieu--
>> > > > >
>> > > > >
>> > > >
>> > > >
>> > > >
>[[drif@bull.com](mailto:drif@bull.com)](mailto:[drif@bull.com](mailto:drif@bull.c
>om))](mailto:[clement.mathieu--
>> > >
>> >
>> > [drif@bull.com](mailto:drif@bull.com)](mailto:[clement.mathieu--
>drif@bull.com](mailto:clement.mathieu--drif@bull.com)))>
>> >
>> > >
>> > > >
>> > > >
>> > > > >
>> > > > >
>> > > > > > Subject: Re: [PATCH v3 0/6] intel_iommu: Enable PRQ support for
>> > > > >
>> > > >
>> > >
>> >
>> > passthrough
>> >
>> > >
>> > > >
>> > > > >
>> > > > > > device
>> > > > > >
>> > > > > > Hi Zhenzhong,
>> > > > > >
>> > > > > > I will read this series early next week.
>> > > > > > thanks for the respin!
>> > > > > >
>> > > > > > btw, related to this work: have ever seen
>> > > > > > vfio_device_get_aw_bits returning 47 whereas
>> > > > > > the iommu supports 48? This leads the following
>> > > > > > aw test to fail:
>> > > > > >
>> > > > > > ```
>> > > > > > ret = hiodc->get_cap(hiod, HOST_IOMMU_DEVICE_CAP_AW_BITS,
>errp);
>> > > > > > if (ret < 0) {
>> > > > > >    return false;
>> > > > > > }
>> > > > > > if (s->aw_bits > ret) {
>> > > > > >    error_setg(errp, "aw-bits %d > host aw-bits %d", s->aw_bits, ret);
>> > > > > >    return false;
>> > > > > > }
>> > > > > > ```
>> > > > > >
>> > > > > > I got this when trying to nest VMs.
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > > I am trying to reproduce, what device did you passthrough?
>> > > >
>> > > >
>> > > >
>> > > > I used edu for testing.
>> > > > An I had aw-bits=48 on both vms.
>> > >
>> > >
>> > >
>> > > Reproduced the same with L1(fsts=on) and L2(fsts=off).
>> >
>> >
>> > My config has fsts=on on both VMs.
>> > Which kernel are you using?
>> > Mine is RHEL10.2 (6.12.0-211.7.3.el10_2.x86_64)
>>
>>
>> In theory we will not run into that check if fsts=on on both VMs, on my env I see:
>>
>> "qemu-system-x86_64: -device vfio-pci,host=00:04.0,iommufd=iommufd0: vfio
>0000:00:04.0: Failed to allocate hwpt: Operation not supported"
>>
>> which is expected as we want nesting support in L1 but it only supports fsts.
>
>Oh yep, I forgot to mention that I'm not attaching iommufd for this test.
>I agree with what you say when iommufd is present, but without it (weird config,
>I admit, but interesting for testing), I get:
>
>```
>qemu-system-x86_64: -device vfio-pci,host=00:01.0: vfio 0000:00:01.0: Failed to
>set vIOMMU: aw-bits 48 > host aw-bits 47

Oh, I see, I reproduced the same.

>```
>
>Just wondering why such a 1bit difference.

host aw-bits 47 is what host kernel supports, not the pure hw limit.
For first stage page table, PT_FEAT_SIGN_EXTEND is set,
kernel limited max iova ranges to 47bit even though hw supports 48 bit, see below:

static __always_inline struct pt_range _pt_top_range(struct pt_common *common,
                                                     uintptr_t top_of_table)
{
...
        /*
         * The top range will default to the lower region only with sign extend.
         */
        range.max_vasz_lg2 = max_vasz_lg2;
        if (pt_feature(common, PT_FEAT_SIGN_EXTEND))
                max_vasz_lg2--;
...
}

So qemu gets host aw-bits 47. This is strictly following vtd spec:

Software using first-stage translation structures to translate an IO Virtual Address (IOVA) must use 
canonical addresses. Additionally, software must limit addresses to less than the minimum of MGAW 
and the lower canonical address width implied by FSPM (i.e., 47-bit when FSPM is 4-level and 56-bit 
when FSPM is 5-level).

BRs,
Zhenzhong