:p
atchew
Login
launching a guest with a TPM-CRB device and VFIO-PCI devices. The CRB command buffer currently is a RAM MemoryRegion and given its base address alignment, it causes an error report on vfio_listener_region_add(). This series proposes to use a ram-device region instead which helps in better assessing the dma map error failure severity on VFIO side. Best Regards Eric This series can be found at: https://github.com/eauger/qemu/tree/tpm-crb-ram-device-v1 Eric Auger (2): tpm: CRB: Use ram_device for "tpm-crb-cmd" region hw/vfio/common: Silence ram device offset alignment error traces hw/tpm/meson.build | 2 +- hw/tpm/tpm_crb.c | 10 ++++++++-- hw/vfio/common.c | 15 ++++++++++++++- hw/vfio/trace-events | 1 + 4 files changed, 24 insertions(+), 4 deletions(-) -- 2.26.3
Representing the CRB cmd/response buffer as a standard RAM region causes some trouble when the device is used with VFIO. Indeed VFIO attempts to DMA_MAP this region as usual RAM but this latter does not have a valid page size alignment causing such an error report: "vfio_listener_region_add received unaligned region". To allow VFIO to detect that failing dma mapping this region is not an issue, let's use a ram_device memory region type instead. The change in meson.build is required to include the cpu.h header. Signed-off-by: Eric Auger <eric.auger@redhat.com> --- hw/tpm/meson.build | 2 +- hw/tpm/tpm_crb.c | 10 ++++++++-- 2 files changed, 9 insertions(+), 3 deletions(-) diff --git a/hw/tpm/meson.build b/hw/tpm/meson.build index XXXXXXX..XXXXXXX 100644 --- a/hw/tpm/meson.build +++ b/hw/tpm/meson.build @@ -XXX,XX +XXX,XX @@ softmmu_ss.add(when: 'CONFIG_TPM_TIS', if_true: files('tpm_tis_common.c')) softmmu_ss.add(when: 'CONFIG_TPM_TIS_ISA', if_true: files('tpm_tis_isa.c')) softmmu_ss.add(when: 'CONFIG_TPM_TIS_SYSBUS', if_true: files('tpm_tis_sysbus.c')) -softmmu_ss.add(when: 'CONFIG_TPM_CRB', if_true: files('tpm_crb.c')) +specific_ss.add(when: 'CONFIG_TPM_CRB', if_true: files('tpm_crb.c')) specific_ss.add(when: ['CONFIG_SOFTMMU', 'CONFIG_TPM_TIS'], if_true: files('tpm_ppi.c')) specific_ss.add(when: ['CONFIG_SOFTMMU', 'CONFIG_TPM_CRB'], if_true: files('tpm_ppi.c')) specific_ss.add(when: 'CONFIG_TPM_SPAPR', if_true: files('tpm_spapr.c')) diff --git a/hw/tpm/tpm_crb.c b/hw/tpm/tpm_crb.c index XXXXXXX..XXXXXXX 100644 --- a/hw/tpm/tpm_crb.c +++ b/hw/tpm/tpm_crb.c @@ -XXX,XX +XXX,XX @@ #include "sysemu/tpm_backend.h" #include "sysemu/tpm_util.h" #include "sysemu/reset.h" +#include "cpu.h" #include "tpm_prop.h" #include "tpm_ppi.h" #include "trace.h" @@ -XXX,XX +XXX,XX @@ struct CRBState { bool ppi_enabled; TPMPPI ppi; + uint8_t *crb_cmd_buf; }; typedef struct CRBState CRBState; @@ -XXX,XX +XXX,XX @@ static void tpm_crb_realize(DeviceState *dev, Error **errp) return; } + s->crb_cmd_buf = qemu_memalign(qemu_real_host_page_size, + HOST_PAGE_ALIGN(CRB_CTRL_CMD_SIZE)); + memory_region_init_io(&s->mmio, OBJECT(s), &tpm_crb_memory_ops, s, "tpm-crb-mmio", sizeof(s->regs)); - memory_region_init_ram(&s->cmdmem, OBJECT(s), - "tpm-crb-cmd", CRB_CTRL_CMD_SIZE, errp); + memory_region_init_ram_device_ptr(&s->cmdmem, OBJECT(s), "tpm-crb-cmd", + CRB_CTRL_CMD_SIZE, s->crb_cmd_buf); + vmstate_register_ram(&s->cmdmem, DEVICE(s)); memory_region_add_subregion(get_system_memory(), TPM_CRB_ADDR_BASE, &s->mmio); -- 2.26.3
Failing to DMA MAP a ram_device should not cause an error message. This is currently happening with the TPM CRB command region and this is causing confusion. We may want to keep the trace for debug purpose though. Signed-off-by: Eric Auger <eric.auger@redhat.com> --- I am not totally clear why we do not fail on the non RAM device case though. --- hw/vfio/common.c | 15 ++++++++++++++- hw/vfio/trace-events | 1 + 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index XXXXXXX..XXXXXXX 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -XXX,XX +XXX,XX @@ static void vfio_listener_region_add(MemoryListener *listener, if (unlikely((section->offset_within_address_space & ~qemu_real_host_page_mask) != (section->offset_within_region & ~qemu_real_host_page_mask))) { - error_report("%s received unaligned region", __func__); + if (memory_region_is_ram_device(section->mr)) { /* just debug purpose */ + trace_vfio_listener_region_add_bad_offset_alignment( + memory_region_name(section->mr), + section->offset_within_address_space, + section->offset_within_region, qemu_real_host_page_size); + } else { /* error case we don't want to be fatal */ + error_report("%s received unaligned region %s iova=0x%"PRIx64 + " offset_within_region=0x%"PRIx64 + " qemu_real_host_page_mask=0x%"PRIx64, + __func__, memory_region_name(section->mr), + section->offset_within_address_space, + section->offset_within_region, + qemu_real_host_page_mask); + } return; } diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index XXXXXXX..XXXXXXX 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -XXX,XX +XXX,XX @@ vfio_listener_region_add_skip(uint64_t start, uint64_t end) "SKIPPING region_add vfio_spapr_group_attach(int groupfd, int tablefd) "Attached groupfd %d to liobn fd %d" vfio_listener_region_add_iommu(uint64_t start, uint64_t end) "region_add [iommu] 0x%"PRIx64" - 0x%"PRIx64 vfio_listener_region_add_ram(uint64_t iova_start, uint64_t iova_end, void *vaddr) "region_add [ram] 0x%"PRIx64" - 0x%"PRIx64" [%p]" +vfio_listener_region_add_bad_offset_alignment(const char *name, uint64_t iova, uint64_t offset_within_region, uint64_t page_size) "Region \"%s\" @0x%"PRIx64", offset_within_region=0x%"PRIx64", qemu_real_host_page_mask=0x%"PRIx64 " cannot be mapped for DMA" vfio_listener_region_add_no_dma_map(const char *name, uint64_t iova, uint64_t size, uint64_t page_size) "Region \"%s\" 0x%"PRIx64" size=0x%"PRIx64" is not aligned to 0x%"PRIx64" and cannot be mapped for DMA" vfio_listener_region_del_skip(uint64_t start, uint64_t end) "SKIPPING region_del 0x%"PRIx64" - 0x%"PRIx64 vfio_listener_region_del(uint64_t start, uint64_t end) "region_del 0x%"PRIx64" - 0x%"PRIx64 -- 2.26.3
This series aims at removing a spurious error message we get when launching a guest with a TPM-CRB device and VFIO-PCI devices. The CRB command buffer currently is a RAM MemoryRegion and given its base address alignment, it causes an error report on vfio_listener_region_add(). This series proposes to use a ram-device region instead which helps in better assessing the dma map error failure on VFIO side. Best Regards Eric This series can be found at: https://github.com/eauger/qemu/tree/tpm-crb-ram-device-v2 History: v1 -> v2: - added tpm_crb_unrealize (dared to keep Stefan's T-b though) Eric Auger (2): tpm: CRB: Use ram_device for "tpm-crb-cmd" region hw/vfio/common: Silence ram device offset alignment error traces hw/tpm/meson.build | 2 +- hw/tpm/tpm_crb.c | 22 ++++++++++++++++++++-- hw/vfio/common.c | 15 ++++++++++++++- hw/vfio/trace-events | 1 + 4 files changed, 36 insertions(+), 4 deletions(-) -- 2.26.3
Representing the CRB cmd/response buffer as a standard RAM region causes some trouble when the device is used with VFIO. Indeed VFIO attempts to DMA_MAP this region as usual RAM but this latter does not have a valid page size alignment causing such an error report: "vfio_listener_region_add received unaligned region". To allow VFIO to detect that failing dma mapping this region is not an issue, let's use a ram_device memory region type instead. The change in meson.build is required to include the cpu.h header. Signed-off-by: Eric Auger <eric.auger@redhat.com> Tested-by: Stefan Berger <stefanb@linux.ibm.com> --- v1 -> v2: - Add tpm_crb_unrealize --- hw/tpm/meson.build | 2 +- hw/tpm/tpm_crb.c | 22 ++++++++++++++++++++-- 2 files changed, 21 insertions(+), 3 deletions(-) diff --git a/hw/tpm/meson.build b/hw/tpm/meson.build index XXXXXXX..XXXXXXX 100644 --- a/hw/tpm/meson.build +++ b/hw/tpm/meson.build @@ -XXX,XX +XXX,XX @@ softmmu_ss.add(when: 'CONFIG_TPM_TIS', if_true: files('tpm_tis_common.c')) softmmu_ss.add(when: 'CONFIG_TPM_TIS_ISA', if_true: files('tpm_tis_isa.c')) softmmu_ss.add(when: 'CONFIG_TPM_TIS_SYSBUS', if_true: files('tpm_tis_sysbus.c')) -softmmu_ss.add(when: 'CONFIG_TPM_CRB', if_true: files('tpm_crb.c')) +specific_ss.add(when: 'CONFIG_TPM_CRB', if_true: files('tpm_crb.c')) specific_ss.add(when: ['CONFIG_SOFTMMU', 'CONFIG_TPM_TIS'], if_true: files('tpm_ppi.c')) specific_ss.add(when: ['CONFIG_SOFTMMU', 'CONFIG_TPM_CRB'], if_true: files('tpm_ppi.c')) specific_ss.add(when: 'CONFIG_TPM_SPAPR', if_true: files('tpm_spapr.c')) diff --git a/hw/tpm/tpm_crb.c b/hw/tpm/tpm_crb.c index XXXXXXX..XXXXXXX 100644 --- a/hw/tpm/tpm_crb.c +++ b/hw/tpm/tpm_crb.c @@ -XXX,XX +XXX,XX @@ #include "sysemu/tpm_backend.h" #include "sysemu/tpm_util.h" #include "sysemu/reset.h" +#include "cpu.h" #include "tpm_prop.h" #include "tpm_ppi.h" #include "trace.h" @@ -XXX,XX +XXX,XX @@ struct CRBState { bool ppi_enabled; TPMPPI ppi; + uint8_t *crb_cmd_buf; }; typedef struct CRBState CRBState; @@ -XXX,XX +XXX,XX @@ static void tpm_crb_realize(DeviceState *dev, Error **errp) return; } + s->crb_cmd_buf = qemu_memalign(qemu_real_host_page_size, + HOST_PAGE_ALIGN(CRB_CTRL_CMD_SIZE)); + memory_region_init_io(&s->mmio, OBJECT(s), &tpm_crb_memory_ops, s, "tpm-crb-mmio", sizeof(s->regs)); - memory_region_init_ram(&s->cmdmem, OBJECT(s), - "tpm-crb-cmd", CRB_CTRL_CMD_SIZE, errp); + memory_region_init_ram_device_ptr(&s->cmdmem, OBJECT(s), "tpm-crb-cmd", + CRB_CTRL_CMD_SIZE, s->crb_cmd_buf); + vmstate_register_ram(&s->cmdmem, DEVICE(s)); memory_region_add_subregion(get_system_memory(), TPM_CRB_ADDR_BASE, &s->mmio); @@ -XXX,XX +XXX,XX @@ static void tpm_crb_realize(DeviceState *dev, Error **errp) qemu_register_reset(tpm_crb_reset, dev); } +static void tpm_crb_unrealize(DeviceState *dev) +{ + CRBState *s = CRB(dev); + + qemu_vfree(s->crb_cmd_buf); + + if (s->ppi_enabled) { + qemu_vfree(s->ppi.buf); + } +} + static void tpm_crb_class_init(ObjectClass *klass, void *data) { DeviceClass *dc = DEVICE_CLASS(klass); TPMIfClass *tc = TPM_IF_CLASS(klass); dc->realize = tpm_crb_realize; + dc->unrealize = tpm_crb_unrealize; device_class_set_props(dc, tpm_crb_properties); dc->vmsd = &vmstate_tpm_crb; dc->user_creatable = true; -- 2.26.3
Failing to DMA MAP a ram_device should not cause an error message. This is currently happening with the TPM CRB command region and this is causing confusion. We may want to keep the trace for debug purpose though. Signed-off-by: Eric Auger <eric.auger@redhat.com> Tested-by: Stefan Berger <stefanb@linux.ibm.com> --- hw/vfio/common.c | 15 ++++++++++++++- hw/vfio/trace-events | 1 + 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index XXXXXXX..XXXXXXX 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -XXX,XX +XXX,XX @@ static void vfio_listener_region_add(MemoryListener *listener, if (unlikely((section->offset_within_address_space & ~qemu_real_host_page_mask) != (section->offset_within_region & ~qemu_real_host_page_mask))) { - error_report("%s received unaligned region", __func__); + if (memory_region_is_ram_device(section->mr)) { /* just debug purpose */ + trace_vfio_listener_region_add_bad_offset_alignment( + memory_region_name(section->mr), + section->offset_within_address_space, + section->offset_within_region, qemu_real_host_page_size); + } else { /* error case we don't want to be fatal */ + error_report("%s received unaligned region %s iova=0x%"PRIx64 + " offset_within_region=0x%"PRIx64 + " qemu_real_host_page_mask=0x%"PRIx64, + __func__, memory_region_name(section->mr), + section->offset_within_address_space, + section->offset_within_region, + qemu_real_host_page_mask); + } return; } diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index XXXXXXX..XXXXXXX 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -XXX,XX +XXX,XX @@ vfio_listener_region_add_skip(uint64_t start, uint64_t end) "SKIPPING region_add vfio_spapr_group_attach(int groupfd, int tablefd) "Attached groupfd %d to liobn fd %d" vfio_listener_region_add_iommu(uint64_t start, uint64_t end) "region_add [iommu] 0x%"PRIx64" - 0x%"PRIx64 vfio_listener_region_add_ram(uint64_t iova_start, uint64_t iova_end, void *vaddr) "region_add [ram] 0x%"PRIx64" - 0x%"PRIx64" [%p]" +vfio_listener_region_add_bad_offset_alignment(const char *name, uint64_t iova, uint64_t offset_within_region, uint64_t page_size) "Region \"%s\" @0x%"PRIx64", offset_within_region=0x%"PRIx64", qemu_real_host_page_mask=0x%"PRIx64 " cannot be mapped for DMA" vfio_listener_region_add_no_dma_map(const char *name, uint64_t iova, uint64_t size, uint64_t page_size) "Region \"%s\" 0x%"PRIx64" size=0x%"PRIx64" is not aligned to 0x%"PRIx64" and cannot be mapped for DMA" vfio_listener_region_del_skip(uint64_t start, uint64_t end) "SKIPPING region_del 0x%"PRIx64" - 0x%"PRIx64 vfio_listener_region_del(uint64_t start, uint64_t end) "region_del 0x%"PRIx64" - 0x%"PRIx64 -- 2.26.3