At the very beginning, the x86 vIOMMUs are created via "-M iommu=on".
We moved one step further a year ago to have the vIOMMUs just like a
general device, so that we can init them with far more specific
parameters with "-device" interface.

However, gradually we found that problem starts to occur due to this.
The main issue is that the DMA address space of any PCI device is
postponed to be init after device realization, while some devices'
realizations would depend on this address space. That looks like a
dead lock. We have patches and solutions for different single problem,
but, maybe it's time we can consider to solve the root cause this
time, of course after 2.9 release.

This series tries to solve the root cause, and move vIOMMU inits back
to machine init for x86 platforms. Then, we'll have solid DMA address
space for each device even during realization.

Please kindly review. Thanks.

Peter Xu (5):
  QemuOpts: introduce qemu_opts_extract()
  util: export device_init_func()
  util: propagate error for device_func_init()
  q35: init vIOMMU during machine init
  pci: move dma_as init back to bus realize

 hw/pci-host/q35.c        | 45 +++++++++++++++++++++++++++++++++++++++++++++
 hw/pci/pci.c             | 21 +--------------------
 include/hw/pci/pci_bus.h |  2 --
 include/qemu-common.h    |  1 +
 include/qemu/option.h    |  2 ++
 include/sysemu/sysemu.h  |  1 -
 util/qemu-option.c       | 24 ++++++++++++++++++++++++
 vl.c                     |  9 ++-------
 8 files changed, 75 insertions(+), 30 deletions(-)

-- 
2.7.4

On Fri, Mar 31, 2017 at 03:36:28PM +0800, Peter Xu wrote:
> At the very beginning, the x86 vIOMMUs are created via "-M iommu=on".
> We moved one step further a year ago to have the vIOMMUs just like a
> general device, so that we can init them with far more specific
> parameters with "-device" interface.
> 
> However, gradually we found that problem starts to occur due to this.
> The main issue is that the DMA address space of any PCI device is
> postponed to be init after device realization, while some devices'
> realizations would depend on this address space. That looks like a
> dead lock. We have patches and solutions for different single problem,
> but, maybe it's time we can consider to solve the root cause this
> time, of course after 2.9 release.
> 
> This series tries to solve the root cause, and move vIOMMU inits back
> to machine init for x86 platforms. Then, we'll have solid DMA address
> space for each device even during realization.
> 
> Please kindly review. Thanks.

I think it's a clean way to do it at a high level.
However I would like to set a tag in the class
rather than listing specific devices.
Also, init order should be consistent for all machines
not just q35.



> Peter Xu (5):
>   QemuOpts: introduce qemu_opts_extract()
>   util: export device_init_func()
>   util: propagate error for device_func_init()
>   q35: init vIOMMU during machine init
>   pci: move dma_as init back to bus realize
> 
>  hw/pci-host/q35.c        | 45 +++++++++++++++++++++++++++++++++++++++++++++
>  hw/pci/pci.c             | 21 +--------------------
>  include/hw/pci/pci_bus.h |  2 --
>  include/qemu-common.h    |  1 +
>  include/qemu/option.h    |  2 ++
>  include/sysemu/sysemu.h  |  1 -
>  util/qemu-option.c       | 24 ++++++++++++++++++++++++
>  vl.c                     |  9 ++-------
>  8 files changed, 75 insertions(+), 30 deletions(-)
> 
> -- 
> 2.7.4

On Fri, Mar 31, 2017 at 07:17:34PM +0300, Michael S. Tsirkin wrote:
> On Fri, Mar 31, 2017 at 03:36:28PM +0800, Peter Xu wrote:
> > At the very beginning, the x86 vIOMMUs are created via "-M iommu=on".
> > We moved one step further a year ago to have the vIOMMUs just like a
> > general device, so that we can init them with far more specific
> > parameters with "-device" interface.
> > 
> > However, gradually we found that problem starts to occur due to this.
> > The main issue is that the DMA address space of any PCI device is
> > postponed to be init after device realization, while some devices'
> > realizations would depend on this address space. That looks like a
> > dead lock. We have patches and solutions for different single problem,
> > but, maybe it's time we can consider to solve the root cause this
> > time, of course after 2.9 release.
> > 
> > This series tries to solve the root cause, and move vIOMMU inits back
> > to machine init for x86 platforms. Then, we'll have solid DMA address
> > space for each device even during realization.
> > 
> > Please kindly review. Thanks.
> 
> I think it's a clean way to do it at a high level.
> However I would like to set a tag in the class
> rather than listing specific devices.
> Also, init order should be consistent for all machines
> not just q35.

I agree that we may finally need a tag if we want to have a general
solution for device init ordering. However I was thinking maybe for
x86 IOMMUs we should even move the init earlier than an ordered device
list, considering the integrated devices are created during machine
init, and it's before the general device init loop.

Actually, iiuc that's also following how the real hardware works -
since the IOMMU unit (now we only have the root vIOMMU) belongs to
north bridge, so imho in emulation codes we'd better follow how the
hardware works if possible (I believe in hardware IOMMUs should be
inited along with north bridge).

(btw, maybe I should mark at least the first patch as RFC...)

Thanks,

-- peterx