| 1 | 1 | ||
|---|---|---|---|
| 2 | > -----Original Message----- | 2 | > -----Original Message----- |
| 3 | > From: Jason Gunthorpe <jgg@nvidia.com> | 3 | > From: Nicolin Chen <nicolinc@nvidia.com> |
| 4 | > Sent: Thursday, February 6, 2025 5:47 PM | 4 | > Sent: Thursday, February 6, 2025 8:33 PM |
| 5 | > To: Daniel P. Berrangé <berrange@redhat.com> | 5 | > To: Shameerali Kolothum Thodi |
| 6 | > Cc: Shameerali Kolothum Thodi | 6 | > <shameerali.kolothum.thodi@huawei.com>; Daniel P. Berrangé |
| 7 | > <shameerali.kolothum.thodi@huawei.com>; qemu-arm@nongnu.org; | 7 | > <berrange@redhat.com>; Jason Gunthorpe <jgg@nvidia.com> |
| 8 | > qemu-devel@nongnu.org; eric.auger@redhat.com; | 8 | > Cc: qemu-arm@nongnu.org; qemu-devel@nongnu.org; |
| 9 | > peter.maydell@linaro.org; nicolinc@nvidia.com; ddutile@redhat.com; | 9 | > eric.auger@redhat.com; peter.maydell@linaro.org; ddutile@redhat.com; |
| 10 | > Linuxarm <linuxarm@huawei.com>; Wangzhou (B) | 10 | > Linuxarm <linuxarm@huawei.com>; Wangzhou (B) |
| 11 | > <wangzhou1@hisilicon.com>; jiangkunkun <jiangkunkun@huawei.com>; | 11 | > <wangzhou1@hisilicon.com>; jiangkunkun <jiangkunkun@huawei.com>; |
| 12 | > Jonathan Cameron <jonathan.cameron@huawei.com>; | 12 | > Jonathan Cameron <jonathan.cameron@huawei.com>; |
| 13 | > zhangfei.gao@linaro.org; nathanc@nvidia.com | 13 | > zhangfei.gao@linaro.org; nathanc@nvidia.com |
| 14 | > Subject: Re: [RFC PATCH 0/5] hw/arm/virt: Add support for user-creatable | 14 | > Subject: Re: [RFC PATCH 0/5] hw/arm/virt: Add support for user-creatable |
| 15 | > nested SMMUv3 | 15 | > nested SMMUv3 |
| 16 | > | 16 | > |
| 17 | > On Thu, Feb 06, 2025 at 05:10:32PM +0000, Daniel P. Berrangé wrote: | 17 | > On Thu, Feb 06, 2025 at 02:22:01PM -0400, Jason Gunthorpe wrote: |
| 18 | > > On Thu, Feb 06, 2025 at 01:02:38PM -0400, Jason Gunthorpe wrote: | 18 | > > On Thu, Feb 06, 2025 at 06:18:14PM +0000, Shameerali Kolothum Thodi |
| 19 | > > > On Thu, Feb 06, 2025 at 03:07:06PM +0000, Shameerali Kolothum Thodi | ||
| 20 | > wrote: | 19 | > wrote: |
| 21 | > > > > > If we set the physical/guest SMMU relationship directly, then at the | 20 | > > |
| 22 | > > > > > time the VFIO device is plugged, we can diagnose the incorrectly | 21 | > > > > So even if you invent an iommu ID we cannot accept it as a handle to |
| 23 | > > > > > placed VFIO device, and better reason about behaviour. | 22 | > > > > create viommu in iommufd. |
| 24 | > > > > | ||
| 25 | > > > > Agree. | ||
| 26 | > > > | 23 | > > > |
| 27 | > > > Can you just take in a VFIO cdev FD reference on this command line: | 24 | > > > Creating the vIOMMU only happens when the user does a cold/hot |
| 28 | > > > | 25 | > plug of |
| 29 | > > > -device arm-smmuv3-accel,id=smmuv2,bus=pcie.2 | 26 | > > > a VFIO device. At that time Qemu checks whether the assigned id |
| 30 | > > > | 27 | > matches |
| 31 | > > > And that will lock the pSMMU/vSMMU relationship? | 28 | > > > with whatever the kernel tell it. |
| 32 | > > | 29 | > > |
| 33 | > > We shouldn't assume any VFIO device exists in the QEMU cnofig at the | 30 | > > This is not hard up until the guest is started. If you boot a guest |
| 34 | > time | 31 | > > without a backing viommu iommufd object then there will be some more |
| 35 | > > we realize the virtual ssmu. I expect the SMMU may be cold plugged, | 32 | > > complexities. |
| 36 | > while | ||
| 37 | > > the VFIO devices may be hot plugged arbitrarly later, and we should have | ||
| 38 | > > the association initialized the SMMU is realized. | ||
| 39 | > | 33 | > |
| 40 | > This is not supported kernel side, you can't instantiate a vIOMMU | 34 | > Yea, I imagined that things would be complicated with hotplugs.. |
| 41 | > without a VFIO device that uses it. For security. | 35 | > |
| 36 | > On one hand, I got the part that we need some fixed link forehand | ||
| 37 | > to ease migration/hotplugs. | ||
| 38 | > | ||
| 39 | > On the other hand, all IOMMUFD ioctls need a VFIO device FD, which | ||
| 40 | > brings the immediate attention that we cannot even decide vSMMU's | ||
| 41 | > capabilities being reflected in its IDR/IIDR registers, without a | ||
| 42 | > coldplug device -- if we boot a VM (one vSMMU<->pSMMU) with only a | ||
| 43 | > hotplug device, the IOMMU_GET_HW_INFO cannot be done during guest | ||
| 42 | 44 | ||
| 43 | I think that is fine if Qemu knows about association beforehand. During | 45 | Right. I forgot about the call to smmu_dev_get_info() during the reset. |
| 44 | vIOMMU instantiation it can cross check whether the user specified | 46 | That means we need at least one dev per Guest SMMU during Guest |
| 45 | pSMMU <->vSMMU is correct for the device. | 47 | boot :( |
| 46 | |||
| 47 | Also how do we do it with multiple VF devices under a pSUMMU ? Which | ||
| 48 | cdev fd in that case? | ||
| 49 | 48 | ||
| 50 | Thanks, | 49 | Thanks, |
| 51 | Shameer | 50 | Shameer | diff view generated by jsdifflib |