This RFC v2 is a follow-up to RFC v1 [1].

On some platforms, peer-to-peer (P2P) DMA between PCIe devices requires
guest physical addresses (GPAs) assigned to device BARs to match their
corresponding host physical addresses (HPAs).

RFC v1 proposed moving PCI enumeration and BAR assignment into QEMU
before firmware execution, with EDK2 operating in discovery-only mode
via PcdPciDisableBusEnumeration.

The feedback was that PCI enumeration and resource assignment should
remain in firmware rather than being performed by the VMM. The suggested
EDK2 mechanism was EFI_INCOMPATIBLE_PCI_DEVICE_SUPPORT_PROTOCOL, which
provides a per-device interception point for PCI resource assignment.

Following the feedback, RFC v2 keeps PCI enumeration and resource
assignment in firmware. QEMU only validates the user-provided fixed
BAR configuration and provides the required metadata to firmware
through the "etc/fixed-bars" fw_cfg file.

Two new PCI properties are introduced:

* fixed-bar=on on a PCIe root port marks its subordinate hierarchy for
  fixed BAR placement. Every device with memory BARs in that hierarchy
  must provide a complete pci-bars= configuration.

* pci-bars=barN@<addr>[,barM@<addr>]... on a PCI endpoint specifies the
  required address for each memory BAR. All memory BARs on the device
  must have an explicitly assigned address.

QEMU validates the user-specified configuration before passing it to
firmware. Validation checks BAR alignment, verifies that addresses are
within the configured PCIe MMIO apertures, and ensures that fixed BAR
ranges do not overlap. QEMU performs neither PCI enumeration nor BAR
allocation.

On the firmware side, a new DXE driver, QemuFixedBarsDxe, installs
EFI_INCOMPATIBLE_PCI_DEVICE_SUPPORT_PROTOCOL before PciBusDxe starts.
When PciBusDxe calls CheckDevice() for a discovered PCI function, the
driver returns ACPI address descriptors with _MIF|_MAF set for fixed
BARs. Two small changes to PciBusDxe preserve these fixed addresses and
program them into the BAR registers during BAR programming.

After PciEnumerationComplete, QemuFixedBarsDxe walks each fixed
root-port hierarchy and programs the bridge memory windows to cover
the fixed BAR ranges assigned to endpoint devices.

Example 1: VFIO passthrough devices under a fixed BAR root port.

    -device pcie-root-port,id=pcie.port1,bus=pcie.1,chassis=1,io-reserve=0,fixed-bar=on
    -device x3130-upstream,id=upstream1,bus=pcie.port1
    -device xio3130-downstream,id=downstream1_1,bus=upstream1,chassis=1,slot=2
    -device vfio-pci-nohotplug,host=0018:06:00.0,bus=downstream1_1,id=dev0
    -set device.dev0.pci-bars=bar0@0x48000000000,bar2@0x50000000000,bar4@0x58000000000

Example 2: Extending the PCIe MMIO aperture with highmem-mmio-base
(and highmem-mmio-size), using a mix of emulated and VFIO devices.

    -machine virt,...,highmem-mmio-base=0x400000000000,highmem-mmio-size=0x400000000000
    -device e1000,netdev=net0,bus=pcie.0,pci-bars=bar0@0x10000000
    -device pcie-root-port,id=pcie.port9,bus=pcie.9,chassis=4,io-reserve=0,fixed-bar=on
    -device x3130-upstream,id=upstream9,bus=pcie.port9
    -device xio3130-downstream,id=downstream9_1,bus=upstream9,chassis=4,slot=1
    -device vfio-pci,host=0012:03:00.1,bus=downstream9_1,id=nic1,pci-bars=bar0@0x7000c0000000
    -device xio3130-downstream,id=downstream9_2,bus=upstream9,chassis=4,slot=2
    -device vfio-pci-nohotplug,host=0019:06:00.0,bus=downstream9_2,id=dev1
    -set device.dev1.pci-bars=bar0@0x6d4000000000,bar2@0x6d8000000000,bar4@0x6e0000000000

The pci-bars= property is generic and may be used with any PCI endpoint
device, including emulated devices and devices with 32-bit memory BARs.

The optional highmem-mmio-base machine property allows the operator to
reposition the high PCIe MMIO window so that the requested BAR
addresses can be placed within the configured address space.

Limitations:
- I/O BARs and the expansion ROM BAR are not covered by pci-bars=.
- This has only been tested on the AArch64 virt machine.
- SR-IOV VF BARs are not covered.

Testing:

The series was tested on the AArch64 virt machine using multiple PCI
topologies, including emulated devices, VFIO passthrough devices, PCIe
switches, and fixed BAR root ports.

A git branch with this series applied is available at:
https://github.com/tdavenvidia/upstream-qemu/tree/RFC-v2-fixed-bar-upstream

[1] RFC v1 upstream thread:
https://lore.kernel.org/qemu-devel/20260508183717.193630-1-tdave@nvidia.com/

Tushar Dave (5):
  hw/pci: add fixed-bar and pci-bars properties
  pci: add validation for fixed BAR configuration
  pci: add fixed BAR fw_cfg blob export
  hw/arm/virt: export fixed BAR metadata via fw_cfg
  hw/arm/virt: add highmem-mmio-base property

 hw/arm/virt.c                   |  78 ++++++++-
 hw/pci-bridge/pcie_root_port.c  |   1 +
 hw/pci/meson.build              |   2 +
 hw/pci/pci-fixed-bar-blob.c     | 291 ++++++++++++++++++++++++++++++++
 hw/pci/pci-fixed-bar-validate.c | 279 ++++++++++++++++++++++++++++++
 hw/pci/pci-fixed-bar-validate.h |  21 +++
 hw/pci/pci-fixed-bar.h          |  60 +++++++
 hw/pci/pci.c                    | 129 ++++++++++++++
 include/hw/pci/pci_device.h     |  10 ++
 include/hw/pci/pcie_port.h      |   1 +
 10 files changed, 871 insertions(+), 1 deletion(-)
 create mode 100644 hw/pci/pci-fixed-bar-blob.c
 create mode 100644 hw/pci/pci-fixed-bar-validate.c
 create mode 100644 hw/pci/pci-fixed-bar-validate.h
 create mode 100644 hw/pci/pci-fixed-bar.h

-- 
2.34.1

  Hi,

> Following the feedback, RFC v2 keeps PCI enumeration and resource
> assignment in firmware. QEMU only validates the user-provided fixed
> BAR configuration and provides the required metadata to firmware
> through the "etc/fixed-bars" fw_cfg file.

qemu already has vendor-specific pci capabilities.  They are used to
pass hints for the bridge window sizes of pci bridges (including pcie
root ports) with hotplug support.  See OvmfPkg/PciHotPlugInitDxe/ for
the firmware side support.

I'd strongly recommend to do the same for the fixed bars:  Add a pci
capability to pass that information.  All the logic you have today to
link the information in the fw_cfg file to the correct pci device is
simply not needed any more then.

> * fixed-bar=on on a PCIe root port marks its subordinate hierarchy for
>   fixed BAR placement. Every device with memory BARs in that hierarchy
>   must provide a complete pci-bars= configuration.

Why is this needed?

> * pci-bars=barN@<addr>[,barM@<addr>]... on a PCI endpoint specifies the
>   required address for each memory BAR. All memory BARs on the device
>   must have an explicitly assigned address.

fixed-bar-<nr>=<addr> ?

> On the firmware side, a new DXE driver, QemuFixedBarsDxe, installs
> EFI_INCOMPATIBLE_PCI_DEVICE_SUPPORT_PROTOCOL before PciBusDxe starts.
> When PciBusDxe calls CheckDevice() for a discovered PCI function, the
> driver returns ACPI address descriptors with _MIF|_MAF set for fixed
> BARs. Two small changes to PciBusDxe preserve these fixed addresses and
> program them into the BAR registers during BAR programming.

Expecting PciBusDxe respecting AddrRangeMin looks sensible to me ...

> After PciEnumerationComplete, QemuFixedBarsDxe walks each fixed
> root-port hierarchy and programs the bridge memory windows to cover
> the fixed BAR ranges assigned to endpoint devices.

... but changing things after-the-fact in platform code is a complete
non-starter.  PciBusDxe needs to do that, i.e. take care that the bridge
window assigned actually cover the fixed pci bars.

take care,
  Gerd

On Thu, 27 Aug 2026 09:18:50 +0200
Gerd Hoffmann <kraxel@redhat.com> wrote:

>   Hi,
> 
> > Following the feedback, RFC v2 keeps PCI enumeration and resource
> > assignment in firmware. QEMU only validates the user-provided fixed
> > BAR configuration and provides the required metadata to firmware
> > through the "etc/fixed-bars" fw_cfg file.  
> 
> qemu already has vendor-specific pci capabilities.  They are used to
> pass hints for the bridge window sizes of pci bridges (including pcie
> root ports) with hotplug support.  See OvmfPkg/PciHotPlugInitDxe/ for
> the firmware side support.
> 
> I'd strongly recommend to do the same for the fixed bars:  Add a pci
> capability to pass that information.  All the logic you have today to
> link the information in the fw_cfg file to the correct pci device is
> simply not needed any more then.

Placement of a VMM defined capability into a vfio-pci device is not
such a trivial problem as it is for emulated devices.  Space may not be
readily available and the capability may mask non-architected registers.

Does this suggestion relate to fixing the gap between mapping fw_cfg
entries by vendor/device IDs or is there something fundamentally
undesirable about using fw_cfg here?
 
> > * fixed-bar=on on a PCIe root port marks its subordinate hierarchy for
> >   fixed BAR placement. Every device with memory BARs in that hierarchy
> >   must provide a complete pci-bars= configuration.  
> 
> Why is this needed?

AIUI, the problem space is greatly expanded if we mix user provided
fixed-bars with firmware assigned BARs and it's possible that there is
no solution that meets the requirements.  This option both simplifies
the problem space and allows the resource windows to be audited to
generate user actionable errors in QEMU.

> > * pci-bars=barN@<addr>[,barM@<addr>]... on a PCI endpoint specifies the
> >   required address for each memory BAR. All memory BARs on the device
> >   must have an explicitly assigned address.  
> 
> fixed-bar-<nr>=<addr> ?

Could be a reasonable alternative.  I'll let Tushar or others wrestle
with the deeper edk2 comments below ;)  Thanks,

Alex

> > On the firmware side, a new DXE driver, QemuFixedBarsDxe, installs
> > EFI_INCOMPATIBLE_PCI_DEVICE_SUPPORT_PROTOCOL before PciBusDxe starts.
> > When PciBusDxe calls CheckDevice() for a discovered PCI function, the
> > driver returns ACPI address descriptors with _MIF|_MAF set for fixed
> > BARs. Two small changes to PciBusDxe preserve these fixed addresses and
> > program them into the BAR registers during BAR programming.  
> 
> Expecting PciBusDxe respecting AddrRangeMin looks sensible to me ...
> 
> > After PciEnumerationComplete, QemuFixedBarsDxe walks each fixed
> > root-port hierarchy and programs the bridge memory windows to cover
> > the fixed BAR ranges assigned to endpoint devices.  
> 
> ... but changing things after-the-fact in platform code is a complete
> non-starter.  PciBusDxe needs to do that, i.e. take care that the bridge
> window assigned actually cover the fixed pci bars.
> 
> take care,
>   Gerd
> 
>

On Thu, Aug 27, 2026 at 07:47:33AM -0600, Alex Williamson wrote:
> On Thu, 27 Aug 2026 09:18:50 +0200
> Gerd Hoffmann <kraxel@redhat.com> wrote:
> 
> > I'd strongly recommend to do the same for the fixed bars:  Add a pci
> > capability to pass that information.  All the logic you have today to
> > link the information in the fw_cfg file to the correct pci device is
> > simply not needed any more then.
> 
> Placement of a VMM defined capability into a vfio-pci device is not
> such a trivial problem as it is for emulated devices.  Space may not be
> readily available and the capability may mask non-architected registers.
> 
> Does this suggestion relate to fixing the gap between mapping fw_cfg
> entries by vendor/device IDs or is there something fundamentally
> undesirable about using fw_cfg here?

Well, fw_cfg is the fallback option if we don't have any better way.
Attaching the information directly to the device by placing it in a
pci capability is at very minimum worth exploring.  If this is not
working for vfio devices, ok, we have to accept that I guess.

And, yes, the logic to match entries in the fw_cfg file with the correct
device using vendor and device id looks somewhat fragile to me too.

Existing code in qemu+firmware (for example bootorder) uses the location
in the physical device tree to identify devices, like this:

/pci@i0cf8/pci-bridge@3/*@0/*@0/*@0,0
 ^^^^^^^^^                             pcie root bus
           ^^^^^^^^^^^^                pcie root port @ slot 3
                        ^^^            virtio-scsi-pci @ slot 0
                            ^^^        scsi controller bus #0
                                ^^^^^  scsi device target 0, lun 0

> > > * fixed-bar=on on a PCIe root port marks its subordinate hierarchy for
> > >   fixed BAR placement. Every device with memory BARs in that hierarchy
> > >   must provide a complete pci-bars= configuration.  
> > 
> > Why is this needed?
> 
> AIUI, the problem space is greatly expanded if we mix user provided
> fixed-bars with firmware assigned BARs and it's possible that there is
> no solution that meets the requirements.  This option both simplifies
> the problem space and allows the resource windows to be audited to
> generate user actionable errors in QEMU.

I can see that allowing fixed and non-fixed bars mix is much harder to
handle.  Do we need to ask the user to manually set that though?  I'd
prefer pci devices propagating automatically to the parent bus that they
have fixed bars and additional constrains apply.

Also: if the main use case for this is to map vfio devices with guest
physical address == host physical address, is there a need to specify
this manually at all?  Shouldn't we have a 'vfio-pci-fixed' device which
handles this automatically?

> > > * pci-bars=barN@<addr>[,barM@<addr>]... on a PCI endpoint specifies the
> > >   required address for each memory BAR. All memory BARs on the device
> > >   must have an explicitly assigned address.  
> > 
> > fixed-bar-<nr>=<addr> ?
> 
> Could be a reasonable alternative.

Parsing (and quoting) property strings with commas in the middle is a
PITA, also when using numerical properties you can use the 'size'
property type which accepts things like '16G'.

take care,
  Gerd

On 8/31/2026 8:24 AM, Gerd Hoffmann wrote:
> On Thu, Aug 27, 2026 at 07:47:33AM -0600, Alex Williamson wrote:
>> On Thu, 27 Aug 2026 09:18:50 +0200
>> Gerd Hoffmann <kraxel@redhat.com> wrote:
>>
>>> I'd strongly recommend to do the same for the fixed bars:  Add a pci
>>> capability to pass that information.  All the logic you have today to
>>> link the information in the fw_cfg file to the correct pci device is
>>> simply not needed any more then.
>>
>> Placement of a VMM defined capability into a vfio-pci device is not
>> such a trivial problem as it is for emulated devices.  Space may not be
>> readily available and the capability may mask non-architected registers.
>>
>> Does this suggestion relate to fixing the gap between mapping fw_cfg
>> entries by vendor/device IDs or is there something fundamentally
>> undesirable about using fw_cfg here?
> 
> Well, fw_cfg is the fallback option if we don't have any better way.
> Attaching the information directly to the device by placing it in a
> pci capability is at very minimum worth exploring.  If this is not
> working for vfio devices, ok, we have to accept that I guess.
> 
> And, yes, the logic to match entries in the fw_cfg file with the correct
> device using vendor and device id looks somewhat fragile to me too.
> 
> Existing code in qemu+firmware (for example bootorder) uses the location
> in the physical device tree to identify devices, like this:
> 
> /pci@i0cf8/pci-bridge@3/*@0/*@0/*@0,0
>  ^^^^^^^^^                             pcie root bus
>            ^^^^^^^^^^^^                pcie root port @ slot 3
>                         ^^^            virtio-scsi-pci @ slot 0
>                             ^^^        scsi controller bus #0
>                                 ^^^^^  scsi device target 0, lun 0

Good point but the problem is CheckDevice()'s own signature, which is
fixed by UEFI PI spec (only passes
VendorId/DeviceId/RevisionId/SubsystemVendorId/SubsystemDeviceId). Even
though the path exists internally, the standard protocol interface
doesn't pass it to the callback. Therefore, we prepare the blob entries
in the same order PciBusDxe discovers devices, so matching by VID:DID
inherently works.

> 
>>>> * fixed-bar=on on a PCIe root port marks its subordinate hierarchy for
>>>>   fixed BAR placement. Every device with memory BARs in that hierarchy
>>>>   must provide a complete pci-bars= configuration.  
>>>
>>> Why is this needed?
>>
>> AIUI, the problem space is greatly expanded if we mix user provided
>> fixed-bars with firmware assigned BARs and it's possible that there is
>> no solution that meets the requirements.  This option both simplifies
>> the problem space and allows the resource windows to be audited to
>> generate user actionable errors in QEMU.
> 
> I can see that allowing fixed and non-fixed bars mix is much harder to
> handle.  Do we need to ask the user to manually set that though?  I'd
> prefer pci devices propagating automatically to the parent bus that they
> have fixed bars and additional constrains apply.

I looked at this again, and technically nothing actually needs the flag
to exist. The real reason I kept it is closer to a usability one; it's
meant to be a visible signal in the launch script itself, so anyone
reading or writing the qemu command line sees up front that every device
under that root port is expected to have pci-bars= configured, rather
than that requirement only surfacing as a runtime error if something's
missing.

I would be okay to drop it but that was the reasoning. Let me know.

> 
> Also: if the main use case for this is to map vfio devices with guest
> physical address == host physical address, is there a need to specify
> this manually at all?  Shouldn't we have a 'vfio-pci-fixed' device which
> handles this automatically?

VFIO GPA == HPA is the primary motivation, but I don't think fixed-bar
should be tied to VFIO or automatically derive guest addresses from the
host. For the VFIO use case, the admin can choose to specify the host
BAR addresses as the fixed-bar configuration to get GPA == HPA, but the
mechanism itself doesn't assume or enforce that -- the desired guest
layout isn't always just a copy of the host's, so having fixed-bar
auto-derive it on its own would be incorrect in some cases, not just
less general. The mechanism remains a generic way to explicitly specify
PCI BAR addresses.

> 
>>>> * pci-bars=barN@<addr>[,barM@<addr>]... on a PCI endpoint specifies the
>>>>   required address for each memory BAR. All memory BARs on the device
>>>>   must have an explicitly assigned address.  
>>>
>>> fixed-bar-<nr>=<addr> ?
>>
>> Could be a reasonable alternative.
> 
> Parsing (and quoting) property strings with commas in the middle is a
> PITA, also when using numerical properties you can use the 'size'
> property type which accepts things like '16G'.

Fair point.

-device some-device,pci-bars=bar0@0x1000000000,bar1@0x2000000000

would become:

-device some-device,fixed-bar-0=0x1000000000,fixed-bar-1=0x2000000000

and, with the 'size' property type, the same addresses could also be
expressed as:

-device some-device,fixed-bar-0=64G,fixed-bar-1=128G

> 
> take care,
>   Gerd

Thanks.
-Tushar

  Hi,

> > And, yes, the logic to match entries in the fw_cfg file with the correct
> > device using vendor and device id looks somewhat fragile to me too.
> > 
> > Existing code in qemu+firmware (for example bootorder) uses the location
> > in the physical device tree to identify devices, like this:
> > 
> > /pci@i0cf8/pci-bridge@3/*@0/*@0/*@0,0
> >  ^^^^^^^^^                             pcie root bus
> >            ^^^^^^^^^^^^                pcie root port @ slot 3
> >                         ^^^            virtio-scsi-pci @ slot 0
> >                             ^^^        scsi controller bus #0
> >                                 ^^^^^  scsi device target 0, lun 0
> 
> Good point but the problem is CheckDevice()'s own signature, which is
> fixed by UEFI PI spec (only passes
> VendorId/DeviceId/RevisionId/SubsystemVendorId/SubsystemDeviceId). Even
> though the path exists internally, the standard protocol interface
> doesn't pass it to the callback.

Hmm, yes.  Seems to be designed to apply quirks to device classes, not
individual devices.

Also note that OVMF already has an incompatible pci device driver and
there can be only one instance, so the code must be merged into the
existing driver instead of adding a second.

> Therefore, we prepare the blob entries
> in the same order PciBusDxe discovers devices, so matching by VID:DID
> inherently works.

Question is whenever we want have that edk2 limitation and the knowledge
about edk2 internals (pci scan order) encoded in the qemu <-> firmware
protocol.  I think it makes sense to (additionally) pass the complete
device path even if the current edk2 implementation doesn't use it, so
we have the option to improve things later on without having to change
the qemu <-> firmware protocolS for that.

> > I can see that allowing fixed and non-fixed bars mix is much harder to
> > handle.  Do we need to ask the user to manually set that though?  I'd
> > prefer pci devices propagating automatically to the parent bus that they
> > have fixed bars and additional constrains apply.
> 
> I looked at this again, and technically nothing actually needs the flag
> to exist. The real reason I kept it is closer to a usability one; it's
> meant to be a visible signal in the launch script itself, so anyone
> reading or writing the qemu command line sees up front that every device
> under that root port is expected to have pci-bars= configured, rather
> than that requirement only surfacing as a runtime error if something's
> missing.

I'm not sure how much of a usability win that actually is, if you forget
to set the flag you still get a runtime error.

In general I like things which can be done automatically actually happen
automatically as this simplifies things for the user in most cases.

> > Also: if the main use case for this is to map vfio devices with guest
> > physical address == host physical address, is there a need to specify
> > this manually at all?  Shouldn't we have a 'vfio-pci-fixed' device which
> > handles this automatically?
> 
> VFIO GPA == HPA is the primary motivation, but I don't think fixed-bar
> should be tied to VFIO or automatically derive guest addresses from the
> host.

Why not?  It is a great usability improvement IMHO.

> For the VFIO use case, the admin can choose to specify the host
> BAR addresses as the fixed-bar configuration to get GPA == HPA, but the
> mechanism itself doesn't assume or enforce that -- the desired guest
> layout isn't always just a copy of the host's, so having fixed-bar
> auto-derive it on its own would be incorrect in some cases, not just
> less general.

You still can have fixed-bar-<nr>=<addr> properties to override the
auto-discovered address for some or all pci bars.

> The mechanism remains a generic way to explicitly specify
> PCI BAR addresses.

Yes, the code which creates the fw_cfg files is generic and it makes
sense to have that in the core pci code, so it can be used for every pci
device.

Nevertheless I'd tend to only expose the properties for devices where an
actual use case exists.  Which is obviously vfio-pci(-fixed).  Also
pci-testdev for development / testing / CI.  I can't see much beyond
that though.

take care,
  Gerd

On 9/2/2026 1:15 AM, Gerd Hoffmann wrote:
>   Hi,
> 
>>> And, yes, the logic to match entries in the fw_cfg file with the correct
>>> device using vendor and device id looks somewhat fragile to me too.
>>>
>>> Existing code in qemu+firmware (for example bootorder) uses the location
>>> in the physical device tree to identify devices, like this:
>>>
>>> /pci@i0cf8/pci-bridge@3/*@0/*@0/*@0,0
>>>  ^^^^^^^^^                             pcie root bus
>>>            ^^^^^^^^^^^^                pcie root port @ slot 3
>>>                         ^^^            virtio-scsi-pci @ slot 0
>>>                             ^^^        scsi controller bus #0
>>>                                 ^^^^^  scsi device target 0, lun 0
>>
>> Good point but the problem is CheckDevice()'s own signature, which is
>> fixed by UEFI PI spec (only passes
>> VendorId/DeviceId/RevisionId/SubsystemVendorId/SubsystemDeviceId). Even
>> though the path exists internally, the standard protocol interface
>> doesn't pass it to the callback.
> 
> Hmm, yes.  Seems to be designed to apply quirks to device classes, not
> individual devices.
> 
> Also note that OVMF already has an incompatible pci device driver and
> there can be only one instance, so the code must be merged into the
> existing driver instead of adding a second.

I checked OvmfPkg/IncompatiblePciDeviceSupportDxe -- its CheckDevice()
is unconditional, it returns the same 64-bit-MMIO-preference descriptor
for every device regardless of VendorId/DeviceId. Merging Fixed BAR
design in would make it a simple dispatch: if the device has an entry in
the fw_cfg blob we export, return our descriptor; otherwise fall through
to the existing behavior unchanged. Does that match what you had in
mind, or is there a different integration point you'd prefer?

> 
>> Therefore, we prepare the blob entries
>> in the same order PciBusDxe discovers devices, so matching by VID:DID
>> inherently works.
> 
> Question is whenever we want have that edk2 limitation and the knowledge
> about edk2 internals (pci scan order) encoded in the qemu <-> firmware
> protocol.  I think it makes sense to (additionally) pass the complete
> device path even if the current edk2 implementation doesn't use it, so
> we have the option to improve things later on without having to change
> the qemu <-> firmware protocolS for that.

I see your point. Sure thing, I'll add it.

> 
>>> I can see that allowing fixed and non-fixed bars mix is much harder to
>>> handle.  Do we need to ask the user to manually set that though?  I'd
>>> prefer pci devices propagating automatically to the parent bus that they
>>> have fixed bars and additional constrains apply.
>>
>> I looked at this again, and technically nothing actually needs the flag
>> to exist. The real reason I kept it is closer to a usability one; it's
>> meant to be a visible signal in the launch script itself, so anyone
>> reading or writing the qemu command line sees up front that every device
>> under that root port is expected to have pci-bars= configured, rather
>> than that requirement only surfacing as a runtime error if something's
>> missing.
> 
> I'm not sure how much of a usability win that actually is, if you forget
> to set the flag you still get a runtime error.

Fair point. I will drop 'fixed-bar=on' from RP property.

Thanks.
-Tushar
> 
> In general I like things which can be done automatically actually happen
> automatically as this simplifies things for the user in most cases.
> 
>>> Also: if the main use case for this is to map vfio devices with guest
>>> physical address == host physical address, is there a need to specify
>>> this manually at all?  Shouldn't we have a 'vfio-pci-fixed' device which
>>> handles this automatically?
>>
>> VFIO GPA == HPA is the primary motivation, but I don't think fixed-bar
>> should be tied to VFIO or automatically derive guest addresses from the
>> host.
> 
> Why not?  It is a great usability improvement IMHO.
> 
>> For the VFIO use case, the admin can choose to specify the host
>> BAR addresses as the fixed-bar configuration to get GPA == HPA, but the
>> mechanism itself doesn't assume or enforce that -- the desired guest
>> layout isn't always just a copy of the host's, so having fixed-bar
>> auto-derive it on its own would be incorrect in some cases, not just
>> less general.
> 
> You still can have fixed-bar-<nr>=<addr> properties to override the
> auto-discovered address for some or all pci bars.
> 
>> The mechanism remains a generic way to explicitly specify
>> PCI BAR addresses.
> 
> Yes, the code which creates the fw_cfg files is generic and it makes
> sense to have that in the core pci code, so it can be used for every pci
> device.
> 
> Nevertheless I'd tend to only expose the properties for devices where an
> actual use case exists.  Which is obviously vfio-pci(-fixed).  Also
> pci-testdev for development / testing / CI.  I can't see much beyond
> that though.
> 
> take care,
>   Gerd

  Hi,

> > Also note that OVMF already has an incompatible pci device driver and
> > there can be only one instance, so the code must be merged into the
> > existing driver instead of adding a second.
> 
> I checked OvmfPkg/IncompatiblePciDeviceSupportDxe -- its CheckDevice()
> is unconditional, it returns the same 64-bit-MMIO-preference descriptor
> for every device regardless of VendorId/DeviceId. Merging Fixed BAR
> design in would make it a simple dispatch: if the device has an entry in
> the fw_cfg blob we export, return our descriptor; otherwise fall through
> to the existing behavior unchanged. Does that match what you had in
> mind, or is there a different integration point you'd prefer?

For the most part yes.

I'd suggest to keep the fixed-bars code in a separate source file,
then just add a small dispatch hook to the existing CheckDevice
function.  In case there is a fixed-bars entry use that instead of the
hardcoded template.

The option rom tweak for confidential VMs should be applied to the
fixed-bars entries too, so don't return early.

take care,
  Gerd

On 9/3/2026 4:47 AM, Gerd Hoffmann wrote:
>   Hi,
> 
>>> Also note that OVMF already has an incompatible pci device driver and
>>> there can be only one instance, so the code must be merged into the
>>> existing driver instead of adding a second.
>>
>> I checked OvmfPkg/IncompatiblePciDeviceSupportDxe -- its CheckDevice()
>> is unconditional, it returns the same 64-bit-MMIO-preference descriptor
>> for every device regardless of VendorId/DeviceId. Merging Fixed BAR
>> design in would make it a simple dispatch: if the device has an entry in
>> the fw_cfg blob we export, return our descriptor; otherwise fall through
>> to the existing behavior unchanged. Does that match what you had in
>> mind, or is there a different integration point you'd prefer?
> 
> For the most part yes.
> 
> I'd suggest to keep the fixed-bars code in a separate source file,
> then just add a small dispatch hook to the existing CheckDevice
> function.  In case there is a fixed-bars entry use that instead of the
> hardcoded template.
> 
> The option rom tweak for confidential VMs should be applied to the
> fixed-bars entries too, so don't return early.

Okay, thanks. I will incorporate all the changes and send the non-RFC
patch series.

Thanks.
-Tushar
> 
> take care,
>   Gerd

On Wed, 2 Sep 2026 08:15:30 +0200
Gerd Hoffmann <kraxel@redhat.com> wrote:

>   Hi,
> 
> > > And, yes, the logic to match entries in the fw_cfg file with the correct
> > > device using vendor and device id looks somewhat fragile to me too.
> > > 
> > > Existing code in qemu+firmware (for example bootorder) uses the location
> > > in the physical device tree to identify devices, like this:
> > > 
> > > /pci@i0cf8/pci-bridge@3/*@0/*@0/*@0,0
> > >  ^^^^^^^^^                             pcie root bus
> > >            ^^^^^^^^^^^^                pcie root port @ slot 3
> > >                         ^^^            virtio-scsi-pci @ slot 0
> > >                             ^^^        scsi controller bus #0
> > >                                 ^^^^^  scsi device target 0, lun 0  

Yeah, I wish such path-based device identification where available here.

> > 
> > Good point but the problem is CheckDevice()'s own signature, which is
> > fixed by UEFI PI spec (only passes
> > VendorId/DeviceId/RevisionId/SubsystemVendorId/SubsystemDeviceId). Even
> > though the path exists internally, the standard protocol interface
> > doesn't pass it to the callback.  
> 
> Hmm, yes.  Seems to be designed to apply quirks to device classes, not
> individual devices.
> 
> Also note that OVMF already has an incompatible pci device driver and
> there can be only one instance, so the code must be merged into the
> existing driver instead of adding a second.
> 
> > Therefore, we prepare the blob entries
> > in the same order PciBusDxe discovers devices, so matching by VID:DID
> > inherently works.  
> 
> Question is whenever we want have that edk2 limitation and the knowledge
> about edk2 internals (pci scan order) encoded in the qemu <-> firmware
> protocol.  I think it makes sense to (additionally) pass the complete
> device path even if the current edk2 implementation doesn't use it, so
> we have the option to improve things later on without having to change
> the qemu <-> firmware protocolS for that.
> 
> > > I can see that allowing fixed and non-fixed bars mix is much harder to
> > > handle.  Do we need to ask the user to manually set that though?  I'd
> > > prefer pci devices propagating automatically to the parent bus that they
> > > have fixed bars and additional constrains apply.  
> > 
> > I looked at this again, and technically nothing actually needs the flag
> > to exist. The real reason I kept it is closer to a usability one; it's
> > meant to be a visible signal in the launch script itself, so anyone
> > reading or writing the qemu command line sees up front that every device
> > under that root port is expected to have pci-bars= configured, rather
> > than that requirement only surfacing as a runtime error if something's
> > missing.  
> 
> I'm not sure how much of a usability win that actually is, if you forget
> to set the flag you still get a runtime error.
> 
> In general I like things which can be done automatically actually happen
> automatically as this simplifies things for the user in most cases.
> 
> > > Also: if the main use case for this is to map vfio devices with guest
> > > physical address == host physical address, is there a need to specify
> > > this manually at all?  Shouldn't we have a 'vfio-pci-fixed' device which
> > > handles this automatically?  
> > 
> > VFIO GPA == HPA is the primary motivation, but I don't think fixed-bar
> > should be tied to VFIO or automatically derive guest addresses from the
> > host.  
> 
> Why not?  It is a great usability improvement IMHO.

We thought about whether to make a vfio-pci shortcut to allow the HPA to
be pushed to the fixed BAR address, but decided that it's also easy for
a userspace script to to collect the physical BAR addresses and
construct the QEMU device options, while maintaining compatibility with
emulated devices and therefore enabling more comprehensive testing.  I
don't think we want to be limited by the physical devices available when
we're testing this.

> > For the VFIO use case, the admin can choose to specify the host
> > BAR addresses as the fixed-bar configuration to get GPA == HPA, but the
> > mechanism itself doesn't assume or enforce that -- the desired guest
> > layout isn't always just a copy of the host's, so having fixed-bar
> > auto-derive it on its own would be incorrect in some cases, not just
> > less general.  
> 
> You still can have fixed-bar-<nr>=<addr> properties to override the
> auto-discovered address for some or all pci bars.

If QEMU is willing to accept both a generic PCI mechanism to specify
this, AND a vfio-pci shortcut, sure, we can create the shortcut.  As
above though, it's also something the caller can construct relatively
easily (maybe not by hand, but with a trivial script) and increases the
test surface for QEMU.
 
> > The mechanism remains a generic way to explicitly specify
> > PCI BAR addresses.  
> 
> Yes, the code which creates the fw_cfg files is generic and it makes
> sense to have that in the core pci code, so it can be used for every pci
> device.
> 
> Nevertheless I'd tend to only expose the properties for devices where an
> actual use case exists.  Which is obviously vfio-pci(-fixed).  Also
> pci-testdev for development / testing / CI.  I can't see much beyond
> that though.

I always imagined the properties would live on the core PCI device and
at best vfio-pci would have a shortcut to prefill those properties
based on physical BAR address.  pci-testdev is pretty limited and we
can't fully test arbitrary device functionality with it.  We'd also
lose the ability to diverge from the host programming if we need to
debug a layout generated on another system.

IMO, the artificial restriction isn't worth it, especially in the
proposed environment where we enforce and validate fixed BAR
configurations for an entire PCI sub-tree.  I think that already
eliminates the most common usage failures we'd see otherwise.
Thanks,

Alex

  Hi,

> > You still can have fixed-bar-<nr>=<addr> properties to override the
> > auto-discovered address for some or all pci bars.
> 
> If QEMU is willing to accept both a generic PCI mechanism to specify
> this, AND a vfio-pci shortcut, sure, we can create the shortcut.

I think this makes sense, but at the end of the day I'm not the pci
maintainer, so this is not my call.

> As above though, it's also something the caller can construct
> relatively easily (maybe not by hand, but with a trivial script)

Then everybody who wants / needs this reinvents such a script.
Do we really want that?

> > Nevertheless I'd tend to only expose the properties for devices where an
> > actual use case exists.  Which is obviously vfio-pci(-fixed).  Also
> > pci-testdev for development / testing / CI.  I can't see much beyond
> > that though.
> 
> I always imagined the properties would live on the core PCI device and
> at best vfio-pci would have a shortcut to prefill those properties
> based on physical BAR address.  pci-testdev is pretty limited and we
> can't fully test arbitrary device functionality with it.  We'd also
> lose the ability to diverge from the host programming if we need to
> debug a layout generated on another system.
> 
> IMO, the artificial restriction isn't worth it, especially in the
> proposed environment where we enforce and validate fixed BAR
> configurations for an entire PCI sub-tree.  I think that already
> eliminates the most common usage failures we'd see otherwise.

Fair enough.

take care,
  Gerd

On Thu, 27 Aug 2026, at 15:47, Alex Williamson via groups.io wrote:
> On Thu, 27 Aug 2026 09:18:50 +0200
> Gerd Hoffmann <kraxel@redhat.com> wrote:
>
>>   Hi,
>> 
>> > Following the feedback, RFC v2 keeps PCI enumeration and resource
>> > assignment in firmware. QEMU only validates the user-provided fixed
>> > BAR configuration and provides the required metadata to firmware
>> > through the "etc/fixed-bars" fw_cfg file.  
>> 
>> qemu already has vendor-specific pci capabilities.  They are used to
>> pass hints for the bridge window sizes of pci bridges (including pcie
>> root ports) with hotplug support.  See OvmfPkg/PciHotPlugInitDxe/ for
>> the firmware side support.
>> 
>> I'd strongly recommend to do the same for the fixed bars:  Add a pci
>> capability to pass that information.  All the logic you have today to
>> link the information in the fw_cfg file to the correct pci device is
>> simply not needed any more then.
>
> Placement of a VMM defined capability into a vfio-pci device is not
> such a trivial problem as it is for emulated devices.  Space may not be
> readily available and the capability may mask non-architected registers.
>

Is this the reason we cannot rely on the Enhanced Allocation (EA)
capability here?

> Does this suggestion relate to fixing the gap between mapping fw_cfg
> entries by vendor/device IDs or is there something fundamentally
> undesirable about using fw_cfg here?
> 

I much prefer this approach over the previous one, as the PCI resource
allocation logic remains in the firmware where it belongs.

However, the OS may still re-assign/re-balance things in some cases, and
so using a non-standard mechanism here means that the OS needs to learn
that these devices are special.

...
>
>> > On the firmware side, a new DXE driver, QemuFixedBarsDxe, installs
>> > EFI_INCOMPATIBLE_PCI_DEVICE_SUPPORT_PROTOCOL before PciBusDxe starts.
>> > When PciBusDxe calls CheckDevice() for a discovered PCI function, the
>> > driver returns ACPI address descriptors with _MIF|_MAF set for fixed
>> > BARs. Two small changes to PciBusDxe preserve these fixed addresses and
>> > program them into the BAR registers during BAR programming.  
>> 
>> Expecting PciBusDxe respecting AddrRangeMin looks sensible to me ...
>> 
>> > After PciEnumerationComplete, QemuFixedBarsDxe walks each fixed
>> > root-port hierarchy and programs the bridge memory windows to cover
>> > the fixed BAR ranges assigned to endpoint devices.  
>> 
>> ... but changing things after-the-fact in platform code is a complete
>> non-starter.  PciBusDxe needs to do that, i.e. take care that the bridge
>> window assigned actually cover the fixed pci bars.
>> 

Agreed - if the bridge windows are not programmed correctly on the first
pass, there is something in the code that needs to be fixed. I don't think
papering over it like this is the right approach.

On 8/27/2026 9:38 AM, Ard Biesheuvel wrote:
> 
> On Thu, 27 Aug 2026, at 15:47, Alex Williamson via groups.io wrote:
>> On Thu, 27 Aug 2026 09:18:50 +0200
>> Gerd Hoffmann <kraxel@redhat.com> wrote:
>>
>>>   Hi,
>>>
>>>> Following the feedback, RFC v2 keeps PCI enumeration and resource
>>>> assignment in firmware. QEMU only validates the user-provided fixed
>>>> BAR configuration and provides the required metadata to firmware
>>>> through the "etc/fixed-bars" fw_cfg file.  
>>>
>>> qemu already has vendor-specific pci capabilities.  They are used to
>>> pass hints for the bridge window sizes of pci bridges (including pcie
>>> root ports) with hotplug support.  See OvmfPkg/PciHotPlugInitDxe/ for
>>> the firmware side support.
>>>
>>> I'd strongly recommend to do the same for the fixed bars:  Add a pci
>>> capability to pass that information.  All the logic you have today to
>>> link the information in the fw_cfg file to the correct pci device is
>>> simply not needed any more then.
>>
>> Placement of a VMM defined capability into a vfio-pci device is not
>> such a trivial problem as it is for emulated devices.  Space may not be
>> readily available and the capability may mask non-architected registers.
>>
> 
> Is this the reason we cannot rely on the Enhanced Allocation (EA)
> capability here?

AFAICT, this was explored on RFC v1.
EFI_INCOMPATIBLE_PCI_DEVICE_SUPPORT_PROTOCOL was suggested as the
shorter path at the time, which is what this series uses.
> 
>> Does this suggestion relate to fixing the gap between mapping fw_cfg
>> entries by vendor/device IDs or is there something fundamentally
>> undesirable about using fw_cfg here?
>>
> 
> I much prefer this approach over the previous one, as the PCI resource
> allocation logic remains in the firmware where it belongs.
> 
> However, the OS may still re-assign/re-balance things in some cases, and
> so using a non-standard mechanism here means that the OS needs to learn
> that these devices are special.

This fixed-bar design already accounted for this using ACPI _DSM in
patch 4/5 — fixed BAR placement requires ACPI and emits the standard
_DSM function 5 (Preserve PCI Boot Configuration) for the hierarchy, so
the OS is told not to reassign these BARs.

> 
> ...
>>
>>>> On the firmware side, a new DXE driver, QemuFixedBarsDxe, installs
>>>> EFI_INCOMPATIBLE_PCI_DEVICE_SUPPORT_PROTOCOL before PciBusDxe starts.
>>>> When PciBusDxe calls CheckDevice() for a discovered PCI function, the
>>>> driver returns ACPI address descriptors with _MIF|_MAF set for fixed
>>>> BARs. Two small changes to PciBusDxe preserve these fixed addresses and
>>>> program them into the BAR registers during BAR programming.  
>>>
>>> Expecting PciBusDxe respecting AddrRangeMin looks sensible to me ...
>>>
>>>> After PciEnumerationComplete, QemuFixedBarsDxe walks each fixed
>>>> root-port hierarchy and programs the bridge memory windows to cover
>>>> the fixed BAR ranges assigned to endpoint devices.  
>>>
>>> ... but changing things after-the-fact in platform code is a complete
>>> non-starter.  PciBusDxe needs to do that, i.e. take care that the bridge
>>> window assigned actually cover the fixed pci bars.
>>>
> 
> Agreed - if the bridge windows are not programmed correctly on the first
> pass, there is something in the code that needs to be fixed. I don't think
> papering over it like this is the right approach.

That's a fair point. I did the window-sizing after PciBusDxe because I
didn't want to touch existing PciBusDxe code too much.

As per my understanding, PciBusDxe's enumeration splits into three phases:

Phase 1 (PciHostBridgeEnumerator) walks the whole tree and calls
CheckDevice() per device — by the time this phase finishes, every fixed
BAR address is already known and cached on the device
(PciBar[Bar].FixedBaseAddress).

Phase 2 (PciHostBridgeResourceAllocator) prepares an address for every
resource node in the tree — both individual BARs and bridge windows
alike — purely from size and alignment; PCI_RESOURCE_NODE has no address
field at all, so this is entirely blind to whether a fixed address was
already required.

Phase 3 (ProgramResource) then writes the actual PCI config-space
registers, and by default it just writes whatever address Phase 2
prepared, for both BARs and bridge windows. The one exception is
ProgramBar() — it specifically checks whether that particular BAR was
marked fixed back in Phase 1, and if so, overrides Phase 2's prepared
base address with the real fixed one. However, ProgramPpbApperture(),
which writes the bridge's own window registers, has no equivalent
override — it always writes whatever Phase 2 prepared, with no awareness
of a fixed BAR anywhere underneath it. And that needs fixing, and for
that I have to change the existing code.

I think the fix would be to extend Phase 2's own sizing step
(CalculateResourceAperture() in PciResourceSupport.c) to check for the
already-known fixed address on each child and, when present, size and
position the window as the exact union of those addresses instead of the
blind size-only sum. Does that match the direction you had in mind, or
is there a different integration point you'd suggest?


Thanks.
-Tushar

  Hi,

> > Is this the reason we cannot rely on the Enhanced Allocation (EA)
> > capability here?
> 
> AFAICT, this was explored on RFC v1.

> EFI_INCOMPATIBLE_PCI_DEVICE_SUPPORT_PROTOCOL was suggested as the
> shorter path at the time, which is what this series uses.

Essentially we have *two* problems to solve here.  The first is how do
we get the fixed bar information from qemu to the firmware, and the
second is how we integrate that into edk2.

So we could have a driver which parses EA and passes along the
information found to PciDxe using
EFI_INCOMPATIBLE_PCI_DEVICE_SUPPORT_PROTOCOL.

Not sure how much of a win that would be compared to adding EA support
to PciDxe directly given that the PciDxe bridge window logic needs
enhancements to properly handle fixed bars (as discussed below).

> > Agreed - if the bridge windows are not programmed correctly on the first
> > pass, there is something in the code that needs to be fixed. I don't think
> > papering over it like this is the right approach.
> 
> That's a fair point. I did the window-sizing after PciBusDxe because I
> didn't want to touch existing PciBusDxe code too much.
> 
> As per my understanding, PciBusDxe's enumeration splits into three phases:
> 
> Phase 1 (PciHostBridgeEnumerator) walks the whole tree and calls
> CheckDevice() per device — by the time this phase finishes, every fixed
> BAR address is already known and cached on the device
> (PciBar[Bar].FixedBaseAddress).
> 
> Phase 2 (PciHostBridgeResourceAllocator) prepares an address for every
> resource node in the tree — both individual BARs and bridge windows
> alike — purely from size and alignment; PCI_RESOURCE_NODE has no address
> field at all, so this is entirely blind to whether a fixed address was
> already required.
> 
> Phase 3 (ProgramResource) then writes the actual PCI config-space
> registers, and by default it just writes whatever address Phase 2
> prepared, for both BARs and bridge windows. The one exception is
> ProgramBar() — it specifically checks whether that particular BAR was
> marked fixed back in Phase 1, and if so, overrides Phase 2's prepared
> base address with the real fixed one. However, ProgramPpbApperture(),
> which writes the bridge's own window registers, has no equivalent
> override — it always writes whatever Phase 2 prepared, with no awareness
> of a fixed BAR anywhere underneath it. And that needs fixing, and for
> that I have to change the existing code.
> 
> I think the fix would be to extend Phase 2's own sizing step
> (CalculateResourceAperture() in PciResourceSupport.c) to check for the
> already-known fixed address on each child and, when present, size and
> position the window as the exact union of those addresses instead of the
> blind size-only sum. Does that match the direction you had in mind, or
> is there a different integration point you'd suggest?

Sounds about right, when propagating resource requirements up from
devices to bridges looking only at the size is not enough if we want
properly support pci bars at fixed locations.

take care,
  Gerd

On 8/31/2026 8:42 AM, Gerd Hoffmann wrote:
>   Hi,
> 
>>> Is this the reason we cannot rely on the Enhanced Allocation (EA)
>>> capability here?
>>
>> AFAICT, this was explored on RFC v1.
> 
>> EFI_INCOMPATIBLE_PCI_DEVICE_SUPPORT_PROTOCOL was suggested as the
>> shorter path at the time, which is what this series uses.
> 
> Essentially we have *two* problems to solve here.  The first is how do
> we get the fixed bar information from qemu to the firmware, and the
> second is how we integrate that into edk2.
> 
> So we could have a driver which parses EA and passes along the
> information found to PciDxe using
> EFI_INCOMPATIBLE_PCI_DEVICE_SUPPORT_PROTOCOL.
> 
> Not sure how much of a win that would be compared to adding EA support
> to PciDxe directly given that the PciDxe bridge window logic needs
> enhancements to properly handle fixed bars (as discussed below).

Agreed, EA doesn't buy us much here; the bridge-window fix is needed
either way.

> 
>>> Agreed - if the bridge windows are not programmed correctly on the first
>>> pass, there is something in the code that needs to be fixed. I don't think
>>> papering over it like this is the right approach.
>>
>> That's a fair point. I did the window-sizing after PciBusDxe because I
>> didn't want to touch existing PciBusDxe code too much.
>>
>> As per my understanding, PciBusDxe's enumeration splits into three phases:
>>
>> Phase 1 (PciHostBridgeEnumerator) walks the whole tree and calls
>> CheckDevice() per device — by the time this phase finishes, every fixed
>> BAR address is already known and cached on the device
>> (PciBar[Bar].FixedBaseAddress).
>>
>> Phase 2 (PciHostBridgeResourceAllocator) prepares an address for every
>> resource node in the tree — both individual BARs and bridge windows
>> alike — purely from size and alignment; PCI_RESOURCE_NODE has no address
>> field at all, so this is entirely blind to whether a fixed address was
>> already required.
>>
>> Phase 3 (ProgramResource) then writes the actual PCI config-space
>> registers, and by default it just writes whatever address Phase 2
>> prepared, for both BARs and bridge windows. The one exception is
>> ProgramBar() — it specifically checks whether that particular BAR was
>> marked fixed back in Phase 1, and if so, overrides Phase 2's prepared
>> base address with the real fixed one. However, ProgramPpbApperture(),
>> which writes the bridge's own window registers, has no equivalent
>> override — it always writes whatever Phase 2 prepared, with no awareness
>> of a fixed BAR anywhere underneath it. And that needs fixing, and for
>> that I have to change the existing code.
>>
>> I think the fix would be to extend Phase 2's own sizing step
>> (CalculateResourceAperture() in PciResourceSupport.c) to check for the
>> already-known fixed address on each child and, when present, size and
>> position the window as the exact union of those addresses instead of the
>> blind size-only sum. Does that match the direction you had in mind, or
>> is there a different integration point you'd suggest?
> 
> Sounds about right, when propagating resource requirements up from
> devices to bridges looking only at the size is not enough if we want
> properly support pci bars at fixed locations.

Thanks for confirming the direction. I'll implement this and post it as
part of the next round.

> 
> take care,
>   Gerd


Thanks.
-Tushar