The impetus for this patch set is to get TPM 2.0 working on Windows 11 ARM64.
Windows' tpm.sys does not seem to work on a TPM TIS device (as verified with
VMWare's implementation). However, the current TPM CRB device uses a fixed
system bus address that is reserved for RAM in ARM64 Virt machines.

In the process of adding the TPM CRB SysBus device, we also went ahead and
cleaned up some of the existing TPM hardware code and fixed some bugs. We used
the TPM TIS devices as a template for the TPM CRB devices and refactored out
common code. We moved the ACPI DSDT generation to the device in order to handle
dynamic base address requirements as well as reduce redundent code in different
machine ACPI generation. We also changed the tpm_crb device to use the ISA bus
instead of depending on the default system bus as the device only was built for
the PC configuration.

Another change is that the TPM CRB registers are now mapped in the same way that
the pflash ROM devices are mapped. It is a memory region whose writes are
trapped as MMIO accesses. This was needed because Apple Silicon does not decode
LDP (AARCH64 load pair of registers) caused page faults. @agraf suggested that
we do this to avoid having to do AARCH64 decoding in the HVF backend's fault
handler.

Unfortunately, it seems like the LDP fault still happens on HVF but the issue
seems to be in the HVF backend which needs to be fixed in a separate patch.

One last thing that's needed to get Windows 11 to recognize the TPM 2.0 device
is for the OVMF firmware to setup the TPM device. Currently, OVMF for ARM64 Virt
only recognizes the TPM TIS device through a FDT entry. A workaround is to
falsely identify the TPM CRB device as a TPM TIS device in the FDT node but this
causes issues for Linux. A proper fix would involve adding an ACPI device driver
in OVMF.

This has been tested on ARM64 with `tpm-crb-device` and on x86_64 with
`tpm-crb`. Additional testing should be performed on other architectures (RISCV
and Loongarch for example) as well as migration cases.

v5:
- Fixed a typo in "tpm_crb: use a single read-as-mem/write-as-mmio mapping"
- Fixed ACPI tables not being created for pc CRB device

v4:
- Fixed broken test blobs

v3:
- Support backwards and forwards migration of existing tpm-crb device
- Dropped patch which moved tpm-crb to ISA bus due to migration concerns
- Unified `tpm_sysbus_plug` handler for ARM and Loongarch
- Added ACPI table tests for tpm-crb-device
- Refactored TPM CRB tests to run on tpm-crb-device for ARM Virt

v2:
- Fixed an issue where VMstate restore from an older version failed due to name
  collision of the memory block.
- In the ACPI table generation for CRB devices, the check for TPM 2.0 backend is
  moved to the device realize as CRB does not support TPM 1.0. It will error in
  that case.
- Dropped the patch to fix crash when PPI is enabled on TIS SysBus device since
  a separate patch submitted by Stefan Berger disables such an option.
- Fixed an issue where we default tpmEstablished=0 when it should be 1.
- In TPM CRB SysBus's ACPI entry, we accidently changed _UID from 0 to 1. This
  shouldn't be an issue but we changed it back just in case.
- Added a patch to migrate saved VMstate from an older version with the regs
  saved separately instead of as a RAM block.

Joelle van Dyne (14):
  tpm_crb: refactor common code
  tpm_crb: CTRL_RSP_ADDR is 64-bits wide
  tpm_ppi: refactor memory space initialization
  tpm_crb: use a single read-as-mem/write-as-mmio mapping
  tpm_crb: move ACPI table building to device interface
  tpm-sysbus: add plug handler for TPM on SysBus
  hw/arm/virt: connect TPM to platform bus
  hw/loongarch/virt: connect TPM to platform bus
  tpm_tis_sysbus: move DSDT AML generation to device
  tests: acpi: prepare for TPM CRB tests
  tpm_crb_sysbus: introduce TPM CRB SysBus device
  tests: acpi: implement TPM CRB tests for ARM virt
  tests: acpi: updated expected blobs for TPM CRB
  tests: add TPM-CRB sysbus tests for aarch64

 docs/specs/tpm.rst                        |   2 +
 hw/tpm/tpm_crb.h                          |  79 ++++++
 hw/tpm/tpm_ppi.h                          |  10 +-
 include/hw/acpi/tpm.h                     |   3 +-
 include/sysemu/tpm.h                      |   7 +
 tests/qtest/tpm-tests.h                   |   2 +
 tests/qtest/tpm-util.h                    |   4 +-
 hw/acpi/aml-build.c                       |   7 +-
 hw/arm/virt-acpi-build.c                  |  38 +--
 hw/arm/virt.c                             |   8 +
 hw/core/sysbus-fdt.c                      |   1 +
 hw/i386/acpi-build.c                      |  16 +-
 hw/loongarch/acpi-build.c                 |  38 +--
 hw/loongarch/virt.c                       |   8 +
 hw/riscv/virt.c                           |   1 +
 hw/tpm/tpm-sysbus.c                       |  47 ++++
 hw/tpm/tpm_crb.c                          | 302 ++++++----------------
 hw/tpm/tpm_crb_common.c                   | 262 +++++++++++++++++++
 hw/tpm/tpm_crb_sysbus.c                   | 162 ++++++++++++
 hw/tpm/tpm_ppi.c                          |   5 +-
 hw/tpm/tpm_tis_isa.c                      |   5 +-
 hw/tpm/tpm_tis_sysbus.c                   |  37 +++
 tests/qtest/bios-tables-test.c            |  47 +++-
 tests/qtest/tpm-crb-device-swtpm-test.c   |  72 ++++++
 tests/qtest/tpm-crb-device-test.c         |  71 +++++
 tests/qtest/tpm-crb-swtpm-test.c          |   2 +
 tests/qtest/tpm-crb-test.c                | 121 +--------
 tests/qtest/tpm-tests.c                   | 121 +++++++++
 tests/qtest/tpm-tis-device-swtpm-test.c   |   2 +-
 tests/qtest/tpm-tis-device-test.c         |   2 +-
 tests/qtest/tpm-tis-i2c-test.c            |   3 +
 tests/qtest/tpm-tis-swtpm-test.c          |   2 +-
 tests/qtest/tpm-tis-test.c                |   2 +-
 tests/qtest/tpm-util.c                    |  16 +-
 hw/arm/Kconfig                            |   1 +
 hw/loongarch/Kconfig                      |   2 +
 hw/riscv/Kconfig                          |   1 +
 hw/tpm/Kconfig                            |   5 +
 hw/tpm/meson.build                        |   5 +
 hw/tpm/trace-events                       |   2 +-
 tests/data/acpi/q35/DSDT.crb.tpm2         | Bin 0 -> 8355 bytes
 tests/data/acpi/q35/TPM2.crb.tpm2         | Bin 0 -> 76 bytes
 tests/data/acpi/virt/DSDT.crb-device.tpm2 | Bin 0 -> 5276 bytes
 tests/data/acpi/virt/TPM2.crb-device.tpm2 | Bin 0 -> 76 bytes
 tests/qtest/meson.build                   |   4 +
 45 files changed, 1057 insertions(+), 468 deletions(-)
 create mode 100644 hw/tpm/tpm_crb.h
 create mode 100644 hw/tpm/tpm-sysbus.c
 create mode 100644 hw/tpm/tpm_crb_common.c
 create mode 100644 hw/tpm/tpm_crb_sysbus.c
 create mode 100644 tests/qtest/tpm-crb-device-swtpm-test.c
 create mode 100644 tests/qtest/tpm-crb-device-test.c
 create mode 100644 tests/data/acpi/q35/DSDT.crb.tpm2
 create mode 100644 tests/data/acpi/q35/TPM2.crb.tpm2
 create mode 100644 tests/data/acpi/virt/DSDT.crb-device.tpm2
 create mode 100644 tests/data/acpi/virt/TPM2.crb-device.tpm2

-- 
2.41.0

Hi

On Tue, Nov 14, 2023 at 6:10 AM Joelle van Dyne <j@getutm.app> wrote:
>
> The impetus for this patch set is to get TPM 2.0 working on Windows 11 ARM64.
> Windows' tpm.sys does not seem to work on a TPM TIS device (as verified with
> VMWare's implementation). However, the current TPM CRB device uses a fixed
> system bus address that is reserved for RAM in ARM64 Virt machines.
>
> In the process of adding the TPM CRB SysBus device, we also went ahead and
> cleaned up some of the existing TPM hardware code and fixed some bugs. We used
> the TPM TIS devices as a template for the TPM CRB devices and refactored out
> common code. We moved the ACPI DSDT generation to the device in order to handle
> dynamic base address requirements as well as reduce redundent code in different
> machine ACPI generation. We also changed the tpm_crb device to use the ISA bus
> instead of depending on the default system bus as the device only was built for
> the PC configuration.
>
> Another change is that the TPM CRB registers are now mapped in the same way that
> the pflash ROM devices are mapped. It is a memory region whose writes are
> trapped as MMIO accesses. This was needed because Apple Silicon does not decode
> LDP (AARCH64 load pair of registers) caused page faults. @agraf suggested that
> we do this to avoid having to do AARCH64 decoding in the HVF backend's fault
> handler.
>
> Unfortunately, it seems like the LDP fault still happens on HVF but the issue
> seems to be in the HVF backend which needs to be fixed in a separate patch.
>
> One last thing that's needed to get Windows 11 to recognize the TPM 2.0 device
> is for the OVMF firmware to setup the TPM device. Currently, OVMF for ARM64 Virt
> only recognizes the TPM TIS device through a FDT entry. A workaround is to
> falsely identify the TPM CRB device as a TPM TIS device in the FDT node but this
> causes issues for Linux. A proper fix would involve adding an ACPI device driver
> in OVMF.
>
> This has been tested on ARM64 with `tpm-crb-device` and on x86_64 with
> `tpm-crb`. Additional testing should be performed on other architectures (RISCV
> and Loongarch for example) as well as migration cases.
>
> v5:
> - Fixed a typo in "tpm_crb: use a single read-as-mem/write-as-mmio mapping"
> - Fixed ACPI tables not being created for pc CRB device
>
> v4:
> - Fixed broken test blobs
>
> v3:
> - Support backwards and forwards migration of existing tpm-crb device
> - Dropped patch which moved tpm-crb to ISA bus due to migration concerns
> - Unified `tpm_sysbus_plug` handler for ARM and Loongarch
> - Added ACPI table tests for tpm-crb-device
> - Refactored TPM CRB tests to run on tpm-crb-device for ARM Virt
>
> v2:
> - Fixed an issue where VMstate restore from an older version failed due to name
>   collision of the memory block.
> - In the ACPI table generation for CRB devices, the check for TPM 2.0 backend is
>   moved to the device realize as CRB does not support TPM 1.0. It will error in
>   that case.
> - Dropped the patch to fix crash when PPI is enabled on TIS SysBus device since
>   a separate patch submitted by Stefan Berger disables such an option.
> - Fixed an issue where we default tpmEstablished=0 when it should be 1.
> - In TPM CRB SysBus's ACPI entry, we accidently changed _UID from 0 to 1. This
>   shouldn't be an issue but we changed it back just in case.
> - Added a patch to migrate saved VMstate from an older version with the regs
>   saved separately instead of as a RAM block.
>
> Joelle van Dyne (14):
>   tpm_crb: refactor common code
>   tpm_crb: CTRL_RSP_ADDR is 64-bits wide
>   tpm_ppi: refactor memory space initialization
>   tpm_crb: use a single read-as-mem/write-as-mmio mapping
>   tpm_crb: move ACPI table building to device interface
>   tpm-sysbus: add plug handler for TPM on SysBus
>   hw/arm/virt: connect TPM to platform bus
>   hw/loongarch/virt: connect TPM to platform bus
>   tpm_tis_sysbus: move DSDT AML generation to device
>   tests: acpi: prepare for TPM CRB tests
>   tpm_crb_sysbus: introduce TPM CRB SysBus device
>   tests: acpi: implement TPM CRB tests for ARM virt
>   tests: acpi: updated expected blobs for TPM CRB
>   tests: add TPM-CRB sysbus tests for aarch64

The series looks good to me.
Have you checked there are no regressions with Windows HLK?

thanks

>
>  docs/specs/tpm.rst                        |   2 +
>  hw/tpm/tpm_crb.h                          |  79 ++++++
>  hw/tpm/tpm_ppi.h                          |  10 +-
>  include/hw/acpi/tpm.h                     |   3 +-
>  include/sysemu/tpm.h                      |   7 +
>  tests/qtest/tpm-tests.h                   |   2 +
>  tests/qtest/tpm-util.h                    |   4 +-
>  hw/acpi/aml-build.c                       |   7 +-
>  hw/arm/virt-acpi-build.c                  |  38 +--
>  hw/arm/virt.c                             |   8 +
>  hw/core/sysbus-fdt.c                      |   1 +
>  hw/i386/acpi-build.c                      |  16 +-
>  hw/loongarch/acpi-build.c                 |  38 +--
>  hw/loongarch/virt.c                       |   8 +
>  hw/riscv/virt.c                           |   1 +
>  hw/tpm/tpm-sysbus.c                       |  47 ++++
>  hw/tpm/tpm_crb.c                          | 302 ++++++----------------
>  hw/tpm/tpm_crb_common.c                   | 262 +++++++++++++++++++
>  hw/tpm/tpm_crb_sysbus.c                   | 162 ++++++++++++
>  hw/tpm/tpm_ppi.c                          |   5 +-
>  hw/tpm/tpm_tis_isa.c                      |   5 +-
>  hw/tpm/tpm_tis_sysbus.c                   |  37 +++
>  tests/qtest/bios-tables-test.c            |  47 +++-
>  tests/qtest/tpm-crb-device-swtpm-test.c   |  72 ++++++
>  tests/qtest/tpm-crb-device-test.c         |  71 +++++
>  tests/qtest/tpm-crb-swtpm-test.c          |   2 +
>  tests/qtest/tpm-crb-test.c                | 121 +--------
>  tests/qtest/tpm-tests.c                   | 121 +++++++++
>  tests/qtest/tpm-tis-device-swtpm-test.c   |   2 +-
>  tests/qtest/tpm-tis-device-test.c         |   2 +-
>  tests/qtest/tpm-tis-i2c-test.c            |   3 +
>  tests/qtest/tpm-tis-swtpm-test.c          |   2 +-
>  tests/qtest/tpm-tis-test.c                |   2 +-
>  tests/qtest/tpm-util.c                    |  16 +-
>  hw/arm/Kconfig                            |   1 +
>  hw/loongarch/Kconfig                      |   2 +
>  hw/riscv/Kconfig                          |   1 +
>  hw/tpm/Kconfig                            |   5 +
>  hw/tpm/meson.build                        |   5 +
>  hw/tpm/trace-events                       |   2 +-
>  tests/data/acpi/q35/DSDT.crb.tpm2         | Bin 0 -> 8355 bytes
>  tests/data/acpi/q35/TPM2.crb.tpm2         | Bin 0 -> 76 bytes
>  tests/data/acpi/virt/DSDT.crb-device.tpm2 | Bin 0 -> 5276 bytes
>  tests/data/acpi/virt/TPM2.crb-device.tpm2 | Bin 0 -> 76 bytes
>  tests/qtest/meson.build                   |   4 +
>  45 files changed, 1057 insertions(+), 468 deletions(-)
>  create mode 100644 hw/tpm/tpm_crb.h
>  create mode 100644 hw/tpm/tpm-sysbus.c
>  create mode 100644 hw/tpm/tpm_crb_common.c
>  create mode 100644 hw/tpm/tpm_crb_sysbus.c
>  create mode 100644 tests/qtest/tpm-crb-device-swtpm-test.c
>  create mode 100644 tests/qtest/tpm-crb-device-test.c
>  create mode 100644 tests/data/acpi/q35/DSDT.crb.tpm2
>  create mode 100644 tests/data/acpi/q35/TPM2.crb.tpm2
>  create mode 100644 tests/data/acpi/virt/DSDT.crb-device.tpm2
>  create mode 100644 tests/data/acpi/virt/TPM2.crb-device.tpm2
>
> --
> 2.41.0
>
>


-- 
Marc-André Lureau

On Tue, Nov 14, 2023 at 1:38 AM Marc-André Lureau
<marcandre.lureau@gmail.com> wrote:
>
> Hi
>
> On Tue, Nov 14, 2023 at 6:10 AM Joelle van Dyne <j@getutm.app> wrote:
> >
> > The impetus for this patch set is to get TPM 2.0 working on Windows 11 ARM64.
> > Windows' tpm.sys does not seem to work on a TPM TIS device (as verified with
> > VMWare's implementation). However, the current TPM CRB device uses a fixed
> > system bus address that is reserved for RAM in ARM64 Virt machines.
> >
> > In the process of adding the TPM CRB SysBus device, we also went ahead and
> > cleaned up some of the existing TPM hardware code and fixed some bugs. We used
> > the TPM TIS devices as a template for the TPM CRB devices and refactored out
> > common code. We moved the ACPI DSDT generation to the device in order to handle
> > dynamic base address requirements as well as reduce redundent code in different
> > machine ACPI generation. We also changed the tpm_crb device to use the ISA bus
> > instead of depending on the default system bus as the device only was built for
> > the PC configuration.
> >
> > Another change is that the TPM CRB registers are now mapped in the same way that
> > the pflash ROM devices are mapped. It is a memory region whose writes are
> > trapped as MMIO accesses. This was needed because Apple Silicon does not decode
> > LDP (AARCH64 load pair of registers) caused page faults. @agraf suggested that
> > we do this to avoid having to do AARCH64 decoding in the HVF backend's fault
> > handler.
> >
> > Unfortunately, it seems like the LDP fault still happens on HVF but the issue
> > seems to be in the HVF backend which needs to be fixed in a separate patch.
> >
> > One last thing that's needed to get Windows 11 to recognize the TPM 2.0 device
> > is for the OVMF firmware to setup the TPM device. Currently, OVMF for ARM64 Virt
> > only recognizes the TPM TIS device through a FDT entry. A workaround is to
> > falsely identify the TPM CRB device as a TPM TIS device in the FDT node but this
> > causes issues for Linux. A proper fix would involve adding an ACPI device driver
> > in OVMF.
> >
> > This has been tested on ARM64 with `tpm-crb-device` and on x86_64 with
> > `tpm-crb`. Additional testing should be performed on other architectures (RISCV
> > and Loongarch for example) as well as migration cases.
> >
> > v5:
> > - Fixed a typo in "tpm_crb: use a single read-as-mem/write-as-mmio mapping"
> > - Fixed ACPI tables not being created for pc CRB device
> >
> > v4:
> > - Fixed broken test blobs
> >
> > v3:
> > - Support backwards and forwards migration of existing tpm-crb device
> > - Dropped patch which moved tpm-crb to ISA bus due to migration concerns
> > - Unified `tpm_sysbus_plug` handler for ARM and Loongarch
> > - Added ACPI table tests for tpm-crb-device
> > - Refactored TPM CRB tests to run on tpm-crb-device for ARM Virt
> >
> > v2:
> > - Fixed an issue where VMstate restore from an older version failed due to name
> >   collision of the memory block.
> > - In the ACPI table generation for CRB devices, the check for TPM 2.0 backend is
> >   moved to the device realize as CRB does not support TPM 1.0. It will error in
> >   that case.
> > - Dropped the patch to fix crash when PPI is enabled on TIS SysBus device since
> >   a separate patch submitted by Stefan Berger disables such an option.
> > - Fixed an issue where we default tpmEstablished=0 when it should be 1.
> > - In TPM CRB SysBus's ACPI entry, we accidently changed _UID from 0 to 1. This
> >   shouldn't be an issue but we changed it back just in case.
> > - Added a patch to migrate saved VMstate from an older version with the regs
> >   saved separately instead of as a RAM block.
> >
> > Joelle van Dyne (14):
> >   tpm_crb: refactor common code
> >   tpm_crb: CTRL_RSP_ADDR is 64-bits wide
> >   tpm_ppi: refactor memory space initialization
> >   tpm_crb: use a single read-as-mem/write-as-mmio mapping
> >   tpm_crb: move ACPI table building to device interface
> >   tpm-sysbus: add plug handler for TPM on SysBus
> >   hw/arm/virt: connect TPM to platform bus
> >   hw/loongarch/virt: connect TPM to platform bus
> >   tpm_tis_sysbus: move DSDT AML generation to device
> >   tests: acpi: prepare for TPM CRB tests
> >   tpm_crb_sysbus: introduce TPM CRB SysBus device
> >   tests: acpi: implement TPM CRB tests for ARM virt
> >   tests: acpi: updated expected blobs for TPM CRB
> >   tests: add TPM-CRB sysbus tests for aarch64
>
> The series looks good to me.
> Have you checked there are no regressions with Windows HLK?
I don't have any experience with Windows HLK. Is there any guide on
running it with QEMU? Preferably with a prebuilt image? Or maybe
someone in the mailing list can run it?

>
> thanks
>
> >
> >  docs/specs/tpm.rst                        |   2 +
> >  hw/tpm/tpm_crb.h                          |  79 ++++++
> >  hw/tpm/tpm_ppi.h                          |  10 +-
> >  include/hw/acpi/tpm.h                     |   3 +-
> >  include/sysemu/tpm.h                      |   7 +
> >  tests/qtest/tpm-tests.h                   |   2 +
> >  tests/qtest/tpm-util.h                    |   4 +-
> >  hw/acpi/aml-build.c                       |   7 +-
> >  hw/arm/virt-acpi-build.c                  |  38 +--
> >  hw/arm/virt.c                             |   8 +
> >  hw/core/sysbus-fdt.c                      |   1 +
> >  hw/i386/acpi-build.c                      |  16 +-
> >  hw/loongarch/acpi-build.c                 |  38 +--
> >  hw/loongarch/virt.c                       |   8 +
> >  hw/riscv/virt.c                           |   1 +
> >  hw/tpm/tpm-sysbus.c                       |  47 ++++
> >  hw/tpm/tpm_crb.c                          | 302 ++++++----------------
> >  hw/tpm/tpm_crb_common.c                   | 262 +++++++++++++++++++
> >  hw/tpm/tpm_crb_sysbus.c                   | 162 ++++++++++++
> >  hw/tpm/tpm_ppi.c                          |   5 +-
> >  hw/tpm/tpm_tis_isa.c                      |   5 +-
> >  hw/tpm/tpm_tis_sysbus.c                   |  37 +++
> >  tests/qtest/bios-tables-test.c            |  47 +++-
> >  tests/qtest/tpm-crb-device-swtpm-test.c   |  72 ++++++
> >  tests/qtest/tpm-crb-device-test.c         |  71 +++++
> >  tests/qtest/tpm-crb-swtpm-test.c          |   2 +
> >  tests/qtest/tpm-crb-test.c                | 121 +--------
> >  tests/qtest/tpm-tests.c                   | 121 +++++++++
> >  tests/qtest/tpm-tis-device-swtpm-test.c   |   2 +-
> >  tests/qtest/tpm-tis-device-test.c         |   2 +-
> >  tests/qtest/tpm-tis-i2c-test.c            |   3 +
> >  tests/qtest/tpm-tis-swtpm-test.c          |   2 +-
> >  tests/qtest/tpm-tis-test.c                |   2 +-
> >  tests/qtest/tpm-util.c                    |  16 +-
> >  hw/arm/Kconfig                            |   1 +
> >  hw/loongarch/Kconfig                      |   2 +
> >  hw/riscv/Kconfig                          |   1 +
> >  hw/tpm/Kconfig                            |   5 +
> >  hw/tpm/meson.build                        |   5 +
> >  hw/tpm/trace-events                       |   2 +-
> >  tests/data/acpi/q35/DSDT.crb.tpm2         | Bin 0 -> 8355 bytes
> >  tests/data/acpi/q35/TPM2.crb.tpm2         | Bin 0 -> 76 bytes
> >  tests/data/acpi/virt/DSDT.crb-device.tpm2 | Bin 0 -> 5276 bytes
> >  tests/data/acpi/virt/TPM2.crb-device.tpm2 | Bin 0 -> 76 bytes
> >  tests/qtest/meson.build                   |   4 +
> >  45 files changed, 1057 insertions(+), 468 deletions(-)
> >  create mode 100644 hw/tpm/tpm_crb.h
> >  create mode 100644 hw/tpm/tpm-sysbus.c
> >  create mode 100644 hw/tpm/tpm_crb_common.c
> >  create mode 100644 hw/tpm/tpm_crb_sysbus.c
> >  create mode 100644 tests/qtest/tpm-crb-device-swtpm-test.c
> >  create mode 100644 tests/qtest/tpm-crb-device-test.c
> >  create mode 100644 tests/data/acpi/q35/DSDT.crb.tpm2
> >  create mode 100644 tests/data/acpi/q35/TPM2.crb.tpm2
> >  create mode 100644 tests/data/acpi/virt/DSDT.crb-device.tpm2
> >  create mode 100644 tests/data/acpi/virt/TPM2.crb-device.tpm2
> >
> > --
> > 2.41.0
> >
> >
>
>
> --
> Marc-André Lureau

Hi

On Tue, Nov 14, 2023 at 11:25 PM Joelle van Dyne <j@getutm.app> wrote:
>
> On Tue, Nov 14, 2023 at 1:38 AM Marc-André Lureau
> <marcandre.lureau@gmail.com> wrote:
> >
> > Hi
> >
> > On Tue, Nov 14, 2023 at 6:10 AM Joelle van Dyne <j@getutm.app> wrote:
> > >
> > > The impetus for this patch set is to get TPM 2.0 working on Windows 11 ARM64.
> > > Windows' tpm.sys does not seem to work on a TPM TIS device (as verified with
> > > VMWare's implementation). However, the current TPM CRB device uses a fixed
> > > system bus address that is reserved for RAM in ARM64 Virt machines.
> > >
> > > In the process of adding the TPM CRB SysBus device, we also went ahead and
> > > cleaned up some of the existing TPM hardware code and fixed some bugs. We used
> > > the TPM TIS devices as a template for the TPM CRB devices and refactored out
> > > common code. We moved the ACPI DSDT generation to the device in order to handle
> > > dynamic base address requirements as well as reduce redundent code in different
> > > machine ACPI generation. We also changed the tpm_crb device to use the ISA bus
> > > instead of depending on the default system bus as the device only was built for
> > > the PC configuration.
> > >
> > > Another change is that the TPM CRB registers are now mapped in the same way that
> > > the pflash ROM devices are mapped. It is a memory region whose writes are
> > > trapped as MMIO accesses. This was needed because Apple Silicon does not decode
> > > LDP (AARCH64 load pair of registers) caused page faults. @agraf suggested that
> > > we do this to avoid having to do AARCH64 decoding in the HVF backend's fault
> > > handler.
> > >
> > > Unfortunately, it seems like the LDP fault still happens on HVF but the issue
> > > seems to be in the HVF backend which needs to be fixed in a separate patch.
> > >
> > > One last thing that's needed to get Windows 11 to recognize the TPM 2.0 device
> > > is for the OVMF firmware to setup the TPM device. Currently, OVMF for ARM64 Virt
> > > only recognizes the TPM TIS device through a FDT entry. A workaround is to
> > > falsely identify the TPM CRB device as a TPM TIS device in the FDT node but this
> > > causes issues for Linux. A proper fix would involve adding an ACPI device driver
> > > in OVMF.
> > >
> > > This has been tested on ARM64 with `tpm-crb-device` and on x86_64 with
> > > `tpm-crb`. Additional testing should be performed on other architectures (RISCV
> > > and Loongarch for example) as well as migration cases.
> > >
> > > v5:
> > > - Fixed a typo in "tpm_crb: use a single read-as-mem/write-as-mmio mapping"
> > > - Fixed ACPI tables not being created for pc CRB device
> > >
> > > v4:
> > > - Fixed broken test blobs
> > >
> > > v3:
> > > - Support backwards and forwards migration of existing tpm-crb device
> > > - Dropped patch which moved tpm-crb to ISA bus due to migration concerns
> > > - Unified `tpm_sysbus_plug` handler for ARM and Loongarch
> > > - Added ACPI table tests for tpm-crb-device
> > > - Refactored TPM CRB tests to run on tpm-crb-device for ARM Virt
> > >
> > > v2:
> > > - Fixed an issue where VMstate restore from an older version failed due to name
> > >   collision of the memory block.
> > > - In the ACPI table generation for CRB devices, the check for TPM 2.0 backend is
> > >   moved to the device realize as CRB does not support TPM 1.0. It will error in
> > >   that case.
> > > - Dropped the patch to fix crash when PPI is enabled on TIS SysBus device since
> > >   a separate patch submitted by Stefan Berger disables such an option.
> > > - Fixed an issue where we default tpmEstablished=0 when it should be 1.
> > > - In TPM CRB SysBus's ACPI entry, we accidently changed _UID from 0 to 1. This
> > >   shouldn't be an issue but we changed it back just in case.
> > > - Added a patch to migrate saved VMstate from an older version with the regs
> > >   saved separately instead of as a RAM block.
> > >
> > > Joelle van Dyne (14):
> > >   tpm_crb: refactor common code
> > >   tpm_crb: CTRL_RSP_ADDR is 64-bits wide
> > >   tpm_ppi: refactor memory space initialization
> > >   tpm_crb: use a single read-as-mem/write-as-mmio mapping
> > >   tpm_crb: move ACPI table building to device interface
> > >   tpm-sysbus: add plug handler for TPM on SysBus
> > >   hw/arm/virt: connect TPM to platform bus
> > >   hw/loongarch/virt: connect TPM to platform bus
> > >   tpm_tis_sysbus: move DSDT AML generation to device
> > >   tests: acpi: prepare for TPM CRB tests
> > >   tpm_crb_sysbus: introduce TPM CRB SysBus device
> > >   tests: acpi: implement TPM CRB tests for ARM virt
> > >   tests: acpi: updated expected blobs for TPM CRB
> > >   tests: add TPM-CRB sysbus tests for aarch64
> >
> > The series looks good to me.
> > Have you checked there are no regressions with Windows HLK?
> I don't have any experience with Windows HLK. Is there any guide on
> running it with QEMU? Preferably with a prebuilt image? Or maybe
> someone in the mailing list can run it?
>

I haven't done it for a while. (it wasn't that hard, just a bit
tedious: install a Controller on a Windows Server VM, and set up a
target VM)

There is also AutoHCK, but I never managed to use it.
(https://github.com/HCK-CI/AutoHCK)

For the current results, they are summarized here:
https://github.com/stefanberger/libtpms/wiki/Testing-of-libtpms-Functionality

Stefan, do you have a working setup? I'll try to make one if it can help.


> >
> > thanks
> >
> > >
> > >  docs/specs/tpm.rst                        |   2 +
> > >  hw/tpm/tpm_crb.h                          |  79 ++++++
> > >  hw/tpm/tpm_ppi.h                          |  10 +-
> > >  include/hw/acpi/tpm.h                     |   3 +-
> > >  include/sysemu/tpm.h                      |   7 +
> > >  tests/qtest/tpm-tests.h                   |   2 +
> > >  tests/qtest/tpm-util.h                    |   4 +-
> > >  hw/acpi/aml-build.c                       |   7 +-
> > >  hw/arm/virt-acpi-build.c                  |  38 +--
> > >  hw/arm/virt.c                             |   8 +
> > >  hw/core/sysbus-fdt.c                      |   1 +
> > >  hw/i386/acpi-build.c                      |  16 +-
> > >  hw/loongarch/acpi-build.c                 |  38 +--
> > >  hw/loongarch/virt.c                       |   8 +
> > >  hw/riscv/virt.c                           |   1 +
> > >  hw/tpm/tpm-sysbus.c                       |  47 ++++
> > >  hw/tpm/tpm_crb.c                          | 302 ++++++----------------
> > >  hw/tpm/tpm_crb_common.c                   | 262 +++++++++++++++++++
> > >  hw/tpm/tpm_crb_sysbus.c                   | 162 ++++++++++++
> > >  hw/tpm/tpm_ppi.c                          |   5 +-
> > >  hw/tpm/tpm_tis_isa.c                      |   5 +-
> > >  hw/tpm/tpm_tis_sysbus.c                   |  37 +++
> > >  tests/qtest/bios-tables-test.c            |  47 +++-
> > >  tests/qtest/tpm-crb-device-swtpm-test.c   |  72 ++++++
> > >  tests/qtest/tpm-crb-device-test.c         |  71 +++++
> > >  tests/qtest/tpm-crb-swtpm-test.c          |   2 +
> > >  tests/qtest/tpm-crb-test.c                | 121 +--------
> > >  tests/qtest/tpm-tests.c                   | 121 +++++++++
> > >  tests/qtest/tpm-tis-device-swtpm-test.c   |   2 +-
> > >  tests/qtest/tpm-tis-device-test.c         |   2 +-
> > >  tests/qtest/tpm-tis-i2c-test.c            |   3 +
> > >  tests/qtest/tpm-tis-swtpm-test.c          |   2 +-
> > >  tests/qtest/tpm-tis-test.c                |   2 +-
> > >  tests/qtest/tpm-util.c                    |  16 +-
> > >  hw/arm/Kconfig                            |   1 +
> > >  hw/loongarch/Kconfig                      |   2 +
> > >  hw/riscv/Kconfig                          |   1 +
> > >  hw/tpm/Kconfig                            |   5 +
> > >  hw/tpm/meson.build                        |   5 +
> > >  hw/tpm/trace-events                       |   2 +-
> > >  tests/data/acpi/q35/DSDT.crb.tpm2         | Bin 0 -> 8355 bytes
> > >  tests/data/acpi/q35/TPM2.crb.tpm2         | Bin 0 -> 76 bytes
> > >  tests/data/acpi/virt/DSDT.crb-device.tpm2 | Bin 0 -> 5276 bytes
> > >  tests/data/acpi/virt/TPM2.crb-device.tpm2 | Bin 0 -> 76 bytes
> > >  tests/qtest/meson.build                   |   4 +
> > >  45 files changed, 1057 insertions(+), 468 deletions(-)
> > >  create mode 100644 hw/tpm/tpm_crb.h
> > >  create mode 100644 hw/tpm/tpm-sysbus.c
> > >  create mode 100644 hw/tpm/tpm_crb_common.c
> > >  create mode 100644 hw/tpm/tpm_crb_sysbus.c
> > >  create mode 100644 tests/qtest/tpm-crb-device-swtpm-test.c
> > >  create mode 100644 tests/qtest/tpm-crb-device-test.c
> > >  create mode 100644 tests/data/acpi/q35/DSDT.crb.tpm2
> > >  create mode 100644 tests/data/acpi/q35/TPM2.crb.tpm2
> > >  create mode 100644 tests/data/acpi/virt/DSDT.crb-device.tpm2
> > >  create mode 100644 tests/data/acpi/virt/TPM2.crb-device.tpm2
> > >
> > > --
> > > 2.41.0
> > >
> > >
> >
> >
> > --
> > Marc-André Lureau



--
Marc-André Lureau

On 11/20/23 03:29, Marc-André Lureau wrote:
> Hi
> 
> On Tue, Nov 14, 2023 at 11:25 PM Joelle van Dyne <j@getutm.app> wrote:
>>
>> On Tue, Nov 14, 2023 at 1:38 AM Marc-André Lureau
>> <marcandre.lureau@gmail.com> wrote:
>>>
>>> Hi
>>>
>>> On Tue, Nov 14, 2023 at 6:10 AM Joelle van Dyne <j@getutm.app> wrote:
>>>>
>>>> The impetus for this patch set is to get TPM 2.0 working on Windows 11 ARM64.
>>>> Windows' tpm.sys does not seem to work on a TPM TIS device (as verified with
>>>> VMWare's implementation). However, the current TPM CRB device uses a fixed
>>>> system bus address that is reserved for RAM in ARM64 Virt machines.
>>>>
>>>> In the process of adding the TPM CRB SysBus device, we also went ahead and
>>>> cleaned up some of the existing TPM hardware code and fixed some bugs. We used
>>>> the TPM TIS devices as a template for the TPM CRB devices and refactored out
>>>> common code. We moved the ACPI DSDT generation to the device in order to handle
>>>> dynamic base address requirements as well as reduce redundent code in different
>>>> machine ACPI generation. We also changed the tpm_crb device to use the ISA bus
>>>> instead of depending on the default system bus as the device only was built for
>>>> the PC configuration.
>>>>
>>>> Another change is that the TPM CRB registers are now mapped in the same way that
>>>> the pflash ROM devices are mapped. It is a memory region whose writes are
>>>> trapped as MMIO accesses. This was needed because Apple Silicon does not decode
>>>> LDP (AARCH64 load pair of registers) caused page faults. @agraf suggested that
>>>> we do this to avoid having to do AARCH64 decoding in the HVF backend's fault
>>>> handler.
>>>>
>>>> Unfortunately, it seems like the LDP fault still happens on HVF but the issue
>>>> seems to be in the HVF backend which needs to be fixed in a separate patch.
>>>>
>>>> One last thing that's needed to get Windows 11 to recognize the TPM 2.0 device
>>>> is for the OVMF firmware to setup the TPM device. Currently, OVMF for ARM64 Virt
>>>> only recognizes the TPM TIS device through a FDT entry. A workaround is to
>>>> falsely identify the TPM CRB device as a TPM TIS device in the FDT node but this
>>>> causes issues for Linux. A proper fix would involve adding an ACPI device driver
>>>> in OVMF.
>>>>
>>>> This has been tested on ARM64 with `tpm-crb-device` and on x86_64 with
>>>> `tpm-crb`. Additional testing should be performed on other architectures (RISCV
>>>> and Loongarch for example) as well as migration cases.
>>>>
>>>> v5:
>>>> - Fixed a typo in "tpm_crb: use a single read-as-mem/write-as-mmio mapping"
>>>> - Fixed ACPI tables not being created for pc CRB device
>>>>
>>>> v4:
>>>> - Fixed broken test blobs
>>>>
>>>> v3:
>>>> - Support backwards and forwards migration of existing tpm-crb device
>>>> - Dropped patch which moved tpm-crb to ISA bus due to migration concerns
>>>> - Unified `tpm_sysbus_plug` handler for ARM and Loongarch
>>>> - Added ACPI table tests for tpm-crb-device
>>>> - Refactored TPM CRB tests to run on tpm-crb-device for ARM Virt
>>>>
>>>> v2:
>>>> - Fixed an issue where VMstate restore from an older version failed due to name
>>>>    collision of the memory block.
>>>> - In the ACPI table generation for CRB devices, the check for TPM 2.0 backend is
>>>>    moved to the device realize as CRB does not support TPM 1.0. It will error in
>>>>    that case.
>>>> - Dropped the patch to fix crash when PPI is enabled on TIS SysBus device since
>>>>    a separate patch submitted by Stefan Berger disables such an option.
>>>> - Fixed an issue where we default tpmEstablished=0 when it should be 1.
>>>> - In TPM CRB SysBus's ACPI entry, we accidently changed _UID from 0 to 1. This
>>>>    shouldn't be an issue but we changed it back just in case.
>>>> - Added a patch to migrate saved VMstate from an older version with the regs
>>>>    saved separately instead of as a RAM block.
>>>>
>>>> Joelle van Dyne (14):
>>>>    tpm_crb: refactor common code
>>>>    tpm_crb: CTRL_RSP_ADDR is 64-bits wide
>>>>    tpm_ppi: refactor memory space initialization
>>>>    tpm_crb: use a single read-as-mem/write-as-mmio mapping
>>>>    tpm_crb: move ACPI table building to device interface
>>>>    tpm-sysbus: add plug handler for TPM on SysBus
>>>>    hw/arm/virt: connect TPM to platform bus
>>>>    hw/loongarch/virt: connect TPM to platform bus
>>>>    tpm_tis_sysbus: move DSDT AML generation to device
>>>>    tests: acpi: prepare for TPM CRB tests
>>>>    tpm_crb_sysbus: introduce TPM CRB SysBus device
>>>>    tests: acpi: implement TPM CRB tests for ARM virt
>>>>    tests: acpi: updated expected blobs for TPM CRB
>>>>    tests: add TPM-CRB sysbus tests for aarch64
>>>
>>> The series looks good to me.
>>> Have you checked there are no regressions with Windows HLK?
>> I don't have any experience with Windows HLK. Is there any guide on
>> running it with QEMU? Preferably with a prebuilt image? Or maybe
>> someone in the mailing list can run it?
>>
> 
> I haven't done it for a while. (it wasn't that hard, just a bit
> tedious: install a Controller on a Windows Server VM, and set up a
> target VM)
> 
> There is also AutoHCK, but I never managed to use it.
> (https://github.com/HCK-CI/AutoHCK)
> 
> For the current results, they are summarized here:
> https://github.com/stefanberger/libtpms/wiki/Testing-of-libtpms-Functionality
> 
> Stefan, do you have a working setup? I'll try to make one if it can help.

Once we have the fixes for big endian support I can run some tests. It 
should be rather quick to verify that the CRB interface still works, 
even with Linux. What I would be more concerned are suspend/upgrade or 
downgrade/resume tests.

    Stefan

> 
> 
>>>
>>> thanks
>>>
>>>>
>>>>   docs/specs/tpm.rst                        |   2 +
>>>>   hw/tpm/tpm_crb.h                          |  79 ++++++
>>>>   hw/tpm/tpm_ppi.h                          |  10 +-
>>>>   include/hw/acpi/tpm.h                     |   3 +-
>>>>   include/sysemu/tpm.h                      |   7 +
>>>>   tests/qtest/tpm-tests.h                   |   2 +
>>>>   tests/qtest/tpm-util.h                    |   4 +-
>>>>   hw/acpi/aml-build.c                       |   7 +-
>>>>   hw/arm/virt-acpi-build.c                  |  38 +--
>>>>   hw/arm/virt.c                             |   8 +
>>>>   hw/core/sysbus-fdt.c                      |   1 +
>>>>   hw/i386/acpi-build.c                      |  16 +-
>>>>   hw/loongarch/acpi-build.c                 |  38 +--
>>>>   hw/loongarch/virt.c                       |   8 +
>>>>   hw/riscv/virt.c                           |   1 +
>>>>   hw/tpm/tpm-sysbus.c                       |  47 ++++
>>>>   hw/tpm/tpm_crb.c                          | 302 ++++++----------------
>>>>   hw/tpm/tpm_crb_common.c                   | 262 +++++++++++++++++++
>>>>   hw/tpm/tpm_crb_sysbus.c                   | 162 ++++++++++++
>>>>   hw/tpm/tpm_ppi.c                          |   5 +-
>>>>   hw/tpm/tpm_tis_isa.c                      |   5 +-
>>>>   hw/tpm/tpm_tis_sysbus.c                   |  37 +++
>>>>   tests/qtest/bios-tables-test.c            |  47 +++-
>>>>   tests/qtest/tpm-crb-device-swtpm-test.c   |  72 ++++++
>>>>   tests/qtest/tpm-crb-device-test.c         |  71 +++++
>>>>   tests/qtest/tpm-crb-swtpm-test.c          |   2 +
>>>>   tests/qtest/tpm-crb-test.c                | 121 +--------
>>>>   tests/qtest/tpm-tests.c                   | 121 +++++++++
>>>>   tests/qtest/tpm-tis-device-swtpm-test.c   |   2 +-
>>>>   tests/qtest/tpm-tis-device-test.c         |   2 +-
>>>>   tests/qtest/tpm-tis-i2c-test.c            |   3 +
>>>>   tests/qtest/tpm-tis-swtpm-test.c          |   2 +-
>>>>   tests/qtest/tpm-tis-test.c                |   2 +-
>>>>   tests/qtest/tpm-util.c                    |  16 +-
>>>>   hw/arm/Kconfig                            |   1 +
>>>>   hw/loongarch/Kconfig                      |   2 +
>>>>   hw/riscv/Kconfig                          |   1 +
>>>>   hw/tpm/Kconfig                            |   5 +
>>>>   hw/tpm/meson.build                        |   5 +
>>>>   hw/tpm/trace-events                       |   2 +-
>>>>   tests/data/acpi/q35/DSDT.crb.tpm2         | Bin 0 -> 8355 bytes
>>>>   tests/data/acpi/q35/TPM2.crb.tpm2         | Bin 0 -> 76 bytes
>>>>   tests/data/acpi/virt/DSDT.crb-device.tpm2 | Bin 0 -> 5276 bytes
>>>>   tests/data/acpi/virt/TPM2.crb-device.tpm2 | Bin 0 -> 76 bytes
>>>>   tests/qtest/meson.build                   |   4 +
>>>>   45 files changed, 1057 insertions(+), 468 deletions(-)
>>>>   create mode 100644 hw/tpm/tpm_crb.h
>>>>   create mode 100644 hw/tpm/tpm-sysbus.c
>>>>   create mode 100644 hw/tpm/tpm_crb_common.c
>>>>   create mode 100644 hw/tpm/tpm_crb_sysbus.c
>>>>   create mode 100644 tests/qtest/tpm-crb-device-swtpm-test.c
>>>>   create mode 100644 tests/qtest/tpm-crb-device-test.c
>>>>   create mode 100644 tests/data/acpi/q35/DSDT.crb.tpm2
>>>>   create mode 100644 tests/data/acpi/q35/TPM2.crb.tpm2
>>>>   create mode 100644 tests/data/acpi/virt/DSDT.crb-device.tpm2
>>>>   create mode 100644 tests/data/acpi/virt/TPM2.crb-device.tpm2
>>>>
>>>> --
>>>> 2.41.0
>>>>
>>>>
>>>
>>>
>>> --
>>> Marc-André Lureau
> 
> 
> 
> --
> Marc-André Lureau

On Tue, 14 Nov 2023 at 02:10, Joelle van Dyne <j@getutm.app> wrote:
> The impetus for this patch set is to get TPM 2.0 working on Windows 11 ARM64.
> Windows' tpm.sys does not seem to work on a TPM TIS device (as verified with
> VMWare's implementation). However, the current TPM CRB device uses a fixed
> system bus address that is reserved for RAM in ARM64 Virt machines.
>
> In the process of adding the TPM CRB SysBus device, we also went ahead and
> cleaned up some of the existing TPM hardware code and fixed some bugs. We used
> the TPM TIS devices as a template for the TPM CRB devices and refactored out
> common code. We moved the ACPI DSDT generation to the device in order to handle
> dynamic base address requirements as well as reduce redundent code in different
> machine ACPI generation. We also changed the tpm_crb device to use the ISA bus
> instead of depending on the default system bus as the device only was built for
> the PC configuration.
>
> Another change is that the TPM CRB registers are now mapped in the same way that
> the pflash ROM devices are mapped. It is a memory region whose writes are
> trapped as MMIO accesses. This was needed because Apple Silicon does not decode
> LDP (AARCH64 load pair of registers) caused page faults. @agraf suggested that
> we do this to avoid having to do AARCH64 decoding in the HVF backend's fault
> handler.

I had a conversation about this on IRC a week or so back (though I
forget who with, sorry) that made me realise there's a problem with this
approach, and I wanted to write that up for the mailing list.

The problem with turning this into a memory-backed device rather than
an MMIO backed device is that it breaks KVM on Arm CPUs which don't
have FEAT_S2FWB (i.e. anything older than Armv8.4). This is because
without FEAT_S2FWB the guest and host will disagree about the memory
attributes of the region:
 * the host knows this is RAM backed and it's normal-cacheable
   (certainly as far as the mapping that QEMU itself has)
 * the guest thinks it's real hardware device registers and maps it
   as Device
The resulting mismatch in cacheability attributes can cause unexpected
behaviour where the guest and QEMU views of the memory contents don't
necessarily match up. (This is the same underlying issue that means
that you can't use QEMU devices that emulate VGA framebuffers on
AArch64 KVM without FEAT_S2FWB.)

With FEAT_S2FWB the problem goes away because the hypervisor can
override the guest's specified memory attributes to get rid of
the attribute mismatch.

So given that this would cause a regression for KVM, my preference
is to stick with the current "the device is backed by MMIO read
and write functions in the normal way". If a particular guest is
trying to access it with LDP/STP that is best fixed in the guest.

Potentially we could emulate (interpret) some subset of complex
load/store insns in QEMU at the point where we get the "took a
data abort but no syndrome information". This ought to be doable
in a way that's shareable between hvf and KVM, and we can write
a decodetree file for the insns we want to interpret. (I would
not try to share with TCG decodetree, though the patterns can
probably be copied across.)

thanks
-- PMM