[RFC PATCH v4 0/6] xl: Allow PCI devices to be passthrough'd via Qemu command line

Thierry Escande posted 6 patches 1 week, 4 days ago
Patches applied successfully (tree, apply log)
git fetch https://gitlab.com/xen-project/patchew/xen tags/patchew/1776955586.8631fc262581453bbf619ec5b2062170.19dbace7684000f373@vates.tech
docs/man/xl-pci-configuration.5.pod | 17 +++++++++
docs/man/xl.cfg.5.pod.in            |  6 +++
tools/include/libxl.h               |  1 +
tools/libs/light/libxl_dm.c         | 33 +++++++++++++++++
tools/libs/light/libxl_internal.h   | 12 +++++-
tools/libs/light/libxl_json.c       | 11 +++++-
tools/libs/light/libxl_pci.c        | 57 ++++++++++++++++++++---------
tools/libs/light/libxl_types.idl    |  1 +
tools/libs/util/libxlu_pci.c        |  2 +
tools/xl/xl_parse.c                 |  5 +++
10 files changed, 123 insertions(+), 22 deletions(-)
[RFC PATCH v4 0/6] xl: Allow PCI devices to be passthrough'd via Qemu command line
Posted by Thierry Escande 1 week, 4 days ago
This series adds new 'hotplug' option for PCI device passthrough.

The current mechanism for device passthrough uses QMP device_add command
to hot-plug PCI devices to the guest Qemu instance. This is an issue for
guests running on Q35 chipset (preliminary support posted at [1]) since the
Q35 PCI root bus does not support hotplug. Devices could be hotplugged to
a secondary PCI bus but Xen only support 1 PCI bus for now.

The 'hotplug' option allows to control how devices are attached to the
guest, either by using the legacy QMP mechanism (this is the default) or by
passing them directly to the Qemu command line using xen-pci-passthrough
device.

Example usage in cfg file:
 pci = [ "00:03.0,seize=1,hotplug=0" ]

Since Qemu -device option accepts parameters in json format, this
patchset adds a new internal function libxl__device_pci_get_qapi_json() that
generates the device json object used for both QMP hotplug and command line
passing.

Also, to handle the assignable PCI device list, the function
libxl_pci_assignable() is renamed as libxl_device_pci_assignable() and is
made available from libxl.h for access in libxl_dm.c. 2 other functions are
also exported internally: libxl__pciback_dev_is_assigned() and
libxl__device_pci_assignable_add().

This has been successfully tested on Xen 4.22-dev and Qemu 10.0 running
Debian VMs in both Bios and UEFI mode with a passthrough'd nvme disk.
Disabling the hotplug mechanism might be made mandatory for Q35 machines
later, once Q35 support is merged upstream.

[1] https://lore.kernel.org/xen-devel/20260313163455.790692-1-thierry.escande@vates.tech/

Changes in v2:
 - Add support for YAJL json parser

Changes in v3:
 - Move code block of device command line parameters creation
 - Better handling of pci device assignation

Changes in v4:
 - Split the refactoring parts of the 2nd patch
 - Extend libxl__json_object_to_json() and use it to generate the qapi json
   string

Thierry Escande (6):
  xl: Add an hotplug option for PCI device passthrough
  libxl: Refactor some PCI device handling functions
  libxl: Add libxl__device_pci_get_qapi_json() internal API
  libxl: Extend libxl__json_object_to_json() prototype
  libxl: Allow PCI device passthrough using -device Qemu command line
  docs: provide description for pci hotplug option

 docs/man/xl-pci-configuration.5.pod | 17 +++++++++
 docs/man/xl.cfg.5.pod.in            |  6 +++
 tools/include/libxl.h               |  1 +
 tools/libs/light/libxl_dm.c         | 33 +++++++++++++++++
 tools/libs/light/libxl_internal.h   | 12 +++++-
 tools/libs/light/libxl_json.c       | 11 +++++-
 tools/libs/light/libxl_pci.c        | 57 ++++++++++++++++++++---------
 tools/libs/light/libxl_types.idl    |  1 +
 tools/libs/util/libxlu_pci.c        |  2 +
 tools/xl/xl_parse.c                 |  5 +++
 10 files changed, 123 insertions(+), 22 deletions(-)

-- 
2.53.0



-- 
Thierry Escande | Vates XCP-ng Developer

XCP-ng & Xen Orchestra - Vates solutions

web: https://vates.tech
Re: [RFC PATCH v4 0/6] xl: Allow PCI devices to be passthrough'd via Qemu command line
Posted by Anthony PERARD 1 week ago
On Thu, Apr 23, 2026 at 04:46:10PM +0200, Thierry Escande wrote:
> This series adds new 'hotplug' option for PCI device passthrough.
> 
> The current mechanism for device passthrough uses QMP device_add command
> to hot-plug PCI devices to the guest Qemu instance. This is an issue for
> guests running on Q35 chipset (preliminary support posted at [1]) since the
> Q35 PCI root bus does not support hotplug. Devices could be hotplugged to
> a secondary PCI bus but Xen only support 1 PCI bus for now.
> 
> The 'hotplug' option allows to control how devices are attached to the
> guest, either by using the legacy QMP mechanism (this is the default) or by
> passing them directly to the Qemu command line using xen-pci-passthrough
> device.

Hi Thierry,

I don't like this new "hotplug" option. It is just an implementation
detail on how the device is added to QEMU. It is just going to be
confusing for users. Because in both case, the device is "cold-plugged",
it just happen to be inserted after QEMU thing it is fully configured,
but the guest hasn't really started yet.

It is likely that we implemented the pci passthrough this way because it
worked and because we could reuse the same code path for real hotplug as
well.

So, for the patch that is really needed (adding the device on the
command line), we could do just for the Q35 machine, or we could do that
for all guests. For the second option, probably the best one, but we are
going to try to find out if everything work as expected, especially
Linux stubdomain.

> Example usage in cfg file:
>  pci = [ "00:03.0,seize=1,hotplug=0" ]
> 
> Since Qemu -device option accepts parameters in json format, this
> patchset adds a new internal function libxl__device_pci_get_qapi_json() that
> generates the device json object used for both QMP hotplug and command line
> passing.
> 
> Also, to handle the assignable PCI device list, the function
> libxl_pci_assignable() is renamed as libxl_device_pci_assignable() and is

I though you were renaming a public API function, but the original one
was static with the wrong name. (function prefixed with "libxl_", one
underscore, are in the public API, private one to libxl are prefixed
with "libxl__".)

> made available from libxl.h for access in libxl_dm.c. 2 other functions are

If you only mean to use a function internally, add it to
libxl_internal.h, I don't want to expose more functions to the API
without a good reason. `xl pci-assignable-list` already work, so I don't
think exposing a new function is useful.

Cheers,


-- 
Anthony Perard | Vates XCP-ng Developer

XCP-ng & Xen Orchestra - Vates solutions

web: https://vates.tech
Re: [RFC PATCH v4 0/6] xl: Allow PCI devices to be passthrough'd via Qemu command line
Posted by Oleksii Kurochko 1 week, 4 days ago
On 4/23/26 4:46 PM, Thierry Escande wrote:
> This series adds new 'hotplug' option for PCI device passthrough.
> 
> The current mechanism for device passthrough uses QMP device_add command
> to hot-plug PCI devices to the guest Qemu instance. This is an issue for
> guests running on Q35 chipset (preliminary support posted at [1]) since the
> Q35 PCI root bus does not support hotplug. Devices could be hotplugged to
> a secondary PCI bus but Xen only support 1 PCI bus for now.
> 
> The 'hotplug' option allows to control how devices are attached to the
> guest, either by using the legacy QMP mechanism (this is the default) or by
> passing them directly to the Qemu command line using xen-pci-passthrough
> device.
> 
> Example usage in cfg file:
>   pci = [ "00:03.0,seize=1,hotplug=0" ]
> 
> Since Qemu -device option accepts parameters in json format, this
> patchset adds a new internal function libxl__device_pci_get_qapi_json() that
> generates the device json object used for both QMP hotplug and command line
> passing.
> 
> Also, to handle the assignable PCI device list, the function
> libxl_pci_assignable() is renamed as libxl_device_pci_assignable() and is
> made available from libxl.h for access in libxl_dm.c. 2 other functions are
> also exported internally: libxl__pciback_dev_is_assigned() and
> libxl__device_pci_assignable_add().
> 
> This has been successfully tested on Xen 4.22-dev and Qemu 10.0 running
> Debian VMs in both Bios and UEFI mode with a passthrough'd nvme disk.
> Disabling the hotplug mechanism might be made mandatory for Q35 machines
> later, once Q35 support is merged upstream.
> 
> [1] https://lore.kernel.org/xen-devel/20260313163455.790692-1-thierry.escande@vates.tech/
> 
> Changes in v2:
>   - Add support for YAJL json parser
> 
> Changes in v3:
>   - Move code block of device command line parameters creation
>   - Better handling of pci device assignation
> 
> Changes in v4:
>   - Split the refactoring parts of the 2nd patch
>   - Extend libxl__json_object_to_json() and use it to generate the qapi json
>     string
> 
> Thierry Escande (6):
>    xl: Add an hotplug option for PCI device passthrough
>    libxl: Refactor some PCI device handling functions
>    libxl: Add libxl__device_pci_get_qapi_json() internal API
>    libxl: Extend libxl__json_object_to_json() prototype
>    libxl: Allow PCI device passthrough using -device Qemu command line
>    docs: provide description for pci hotplug option
> 
>   docs/man/xl-pci-configuration.5.pod | 17 +++++++++
>   docs/man/xl.cfg.5.pod.in            |  6 +++
>   tools/include/libxl.h               |  1 +
>   tools/libs/light/libxl_dm.c         | 33 +++++++++++++++++
>   tools/libs/light/libxl_internal.h   | 12 +++++-
>   tools/libs/light/libxl_json.c       | 11 +++++-
>   tools/libs/light/libxl_pci.c        | 57 ++++++++++++++++++++---------
>   tools/libs/light/libxl_types.idl    |  1 +
>   tools/libs/util/libxlu_pci.c        |  2 +
>   tools/xl/xl_parse.c                 |  5 +++
>   10 files changed, 123 insertions(+), 22 deletions(-)
> 

I think we also want to add an item to CHANGELOG.md that it is possible 
to passthrough PCI device via QEMU command line.

Thanks.

~ Oleksii
Re: [RFC PATCH v4 0/6] xl: Allow PCI devices to be passthrough'd via Qemu command line
Posted by Thierry Escande 1 week, 4 days ago

On 4/23/26 17:20, Oleksii Kurochko wrote:
> 
> On 4/23/26 4:46 PM, Thierry Escande wrote:
>> This series adds new 'hotplug' option for PCI device passthrough.
>>
>> The current mechanism for device passthrough uses QMP device_add command
>> to hot-plug PCI devices to the guest Qemu instance. This is an issue for
>> guests running on Q35 chipset (preliminary support posted at [1])
>> since the
>> Q35 PCI root bus does not support hotplug. Devices could be hotplugged to
>> a secondary PCI bus but Xen only support 1 PCI bus for now.
>>
>> The 'hotplug' option allows to control how devices are attached to the
>> guest, either by using the legacy QMP mechanism (this is the default)
>> or by
>> passing them directly to the Qemu command line using xen-pci-passthrough
>> device.
>>
>> Example usage in cfg file:
>>   pci = [ "00:03.0,seize=1,hotplug=0" ]
>>
>> Since Qemu -device option accepts parameters in json format, this
>> patchset adds a new internal function
>> libxl__device_pci_get_qapi_json() that
>> generates the device json object used for both QMP hotplug and command
>> line
>> passing.
>>
>> Also, to handle the assignable PCI device list, the function
>> libxl_pci_assignable() is renamed as libxl_device_pci_assignable() and is
>> made available from libxl.h for access in libxl_dm.c. 2 other
>> functions are
>> also exported internally: libxl__pciback_dev_is_assigned() and
>> libxl__device_pci_assignable_add().
>>
>> This has been successfully tested on Xen 4.22-dev and Qemu 10.0 running
>> Debian VMs in both Bios and UEFI mode with a passthrough'd nvme disk.
>> Disabling the hotplug mechanism might be made mandatory for Q35 machines
>> later, once Q35 support is merged upstream.
>>
>> [1] https://lore.kernel.org/xen-devel/20260313163455.790692-1-
>> thierry.escande@vates.tech/
>>
>> Changes in v2:
>>   - Add support for YAJL json parser
>>
>> Changes in v3:
>>   - Move code block of device command line parameters creation
>>   - Better handling of pci device assignation
>>
>> Changes in v4:
>>   - Split the refactoring parts of the 2nd patch
>>   - Extend libxl__json_object_to_json() and use it to generate the
>> qapi json
>>     string
>>
>> Thierry Escande (6):
>>    xl: Add an hotplug option for PCI device passthrough
>>    libxl: Refactor some PCI device handling functions
>>    libxl: Add libxl__device_pci_get_qapi_json() internal API
>>    libxl: Extend libxl__json_object_to_json() prototype
>>    libxl: Allow PCI device passthrough using -device Qemu command line
>>    docs: provide description for pci hotplug option
>>
>>   docs/man/xl-pci-configuration.5.pod | 17 +++++++++
>>   docs/man/xl.cfg.5.pod.in            |  6 +++
>>   tools/include/libxl.h               |  1 +
>>   tools/libs/light/libxl_dm.c         | 33 +++++++++++++++++
>>   tools/libs/light/libxl_internal.h   | 12 +++++-
>>   tools/libs/light/libxl_json.c       | 11 +++++-
>>   tools/libs/light/libxl_pci.c        | 57 ++++++++++++++++++++---------
>>   tools/libs/light/libxl_types.idl    |  1 +
>>   tools/libs/util/libxlu_pci.c        |  2 +
>>   tools/xl/xl_parse.c                 |  5 +++
>>   10 files changed, 123 insertions(+), 22 deletions(-)
>>
> 
> I think we also want to add an item to CHANGELOG.md that it is possible
> to passthrough PCI device via QEMU command line.

Sure, will do.

Thanks,
Thierry

> Thanks.
> 
> ~ Oleksii



-- 
Thierry Escande | Vates XCP-ng Developer

XCP-ng & Xen Orchestra - Vates solutions

web: https://vates.tech