[RFC PATCH v2 00/23] NXP i.MX RT595, ARM SVD and device model unit tests

Octavian Purdila posted 23 patches 3 months, 1 week ago
meson.build                                   |    4 +
include/hw/arm/rt500.h                        |   44 +
include/hw/arm/svd/flexcomm.h                 |  109 +
include/hw/arm/svd/flexcomm_i2c.h             | 1134 ++++++
include/hw/arm/svd/flexcomm_spi.h             |  980 ++++++
include/hw/arm/svd/flexcomm_usart.h           |  959 +++++
include/hw/arm/svd/flexspi.h                  | 2100 +++++++++++
include/hw/arm/svd/rt500.h                    |   65 +
include/hw/arm/svd/rt500_clkctl0.h            | 2287 ++++++++++++
include/hw/arm/svd/rt500_clkctl1.h            | 3109 +++++++++++++++++
include/hw/arm/svd/rt500_rstctl0.h            |  832 +++++
include/hw/arm/svd/rt500_rstctl1.h            | 1344 +++++++
include/hw/char/flexcomm_usart.h              |   20 +
include/hw/i2c/flexcomm_i2c.h                 |   27 +
include/hw/misc/flexcomm.h                    |   85 +
include/hw/misc/rt500_clk_freqs.h             |   18 +
include/hw/misc/rt500_clkctl0.h               |   37 +
include/hw/misc/rt500_clkctl1.h               |   36 +
include/hw/misc/rt500_rstctl.h                |   32 +
include/hw/regs.h                             |   34 +
include/hw/ssi/flexcomm_spi.h                 |   20 +
include/hw/ssi/flexspi.h                      |   32 +
include/qemu/fifo32.h                         |   28 +
tests/unit/i2c_tester.h                       |   30 +
tests/unit/reg-utils.h                        |   97 +
tests/unit/spi_tester.h                       |   32 +
tests/unit/sysbus-mock.h                      |   82 +
hw/arm/rt500.c                                |  335 ++
hw/arm/rt595-evk.c                            |   64 +
hw/char/flexcomm_usart.c                      |  292 ++
hw/i2c/flexcomm_i2c.c                         |  207 ++
hw/misc/flexcomm.c                            |  304 ++
hw/misc/rt500_clkctl0.c                       |  226 ++
hw/misc/rt500_clkctl1.c                       |  207 ++
hw/misc/rt500_rstctl.c                        |  225 ++
hw/ssi/flexcomm_spi.c                         |  439 +++
hw/ssi/flexspi.c                              |  157 +
tests/unit/i2c_tester.c                       |  108 +
tests/unit/spi_tester.c                       |   57 +
tests/unit/sysbus-mock.c                      |  312 ++
tests/unit/test-fifo.c                        |   97 +
tests/unit/test-flexcomm-i2c.c                |  210 ++
tests/unit/test-flexcomm-spi.c                |  201 ++
tests/unit/test-flexcomm-usart.c              |  320 ++
tests/unit/test-flexcomm.c                    |  212 ++
tests/unit/test-rt500-clkctl.c                |  263 ++
hw/arm/Kconfig                                |   13 +
hw/arm/meson.build                            |    4 +
hw/arm/svd/meson.build                        |   35 +
hw/char/meson.build                           |    1 +
hw/char/trace-events                          |    9 +
hw/i2c/meson.build                            |    1 +
hw/i2c/trace-events                           |   10 +
hw/misc/Kconfig                               |   12 +
hw/misc/meson.build                           |    5 +
hw/misc/trace-events                          |   18 +
hw/ssi/Kconfig                                |    4 +
hw/ssi/meson.build                            |    2 +
hw/ssi/trace-events                           |   12 +
meson_options.txt                             |    3 +
scripts/meson-buildoptions.sh                 |    4 +
scripts/svd-gen-header.py                     |  343 ++
subprojects/.gitignore                        |    1 +
subprojects/mcux-soc-svd.wrap                 |    5 +
.../packagefiles/mcux-soc-svd/meson.build     |    5 +
tests/unit/meson.build                        |   56 +-
66 files changed, 18355 insertions(+), 1 deletion(-)
create mode 100644 include/hw/arm/rt500.h
create mode 100644 include/hw/arm/svd/flexcomm.h
create mode 100644 include/hw/arm/svd/flexcomm_i2c.h
create mode 100644 include/hw/arm/svd/flexcomm_spi.h
create mode 100644 include/hw/arm/svd/flexcomm_usart.h
create mode 100644 include/hw/arm/svd/flexspi.h
create mode 100644 include/hw/arm/svd/rt500.h
create mode 100644 include/hw/arm/svd/rt500_clkctl0.h
create mode 100644 include/hw/arm/svd/rt500_clkctl1.h
create mode 100644 include/hw/arm/svd/rt500_rstctl0.h
create mode 100644 include/hw/arm/svd/rt500_rstctl1.h
create mode 100644 include/hw/char/flexcomm_usart.h
create mode 100644 include/hw/i2c/flexcomm_i2c.h
create mode 100644 include/hw/misc/flexcomm.h
create mode 100644 include/hw/misc/rt500_clk_freqs.h
create mode 100644 include/hw/misc/rt500_clkctl0.h
create mode 100644 include/hw/misc/rt500_clkctl1.h
create mode 100644 include/hw/misc/rt500_rstctl.h
create mode 100644 include/hw/regs.h
create mode 100644 include/hw/ssi/flexcomm_spi.h
create mode 100644 include/hw/ssi/flexspi.h
create mode 100644 tests/unit/i2c_tester.h
create mode 100644 tests/unit/reg-utils.h
create mode 100644 tests/unit/spi_tester.h
create mode 100644 tests/unit/sysbus-mock.h
create mode 100644 hw/arm/rt500.c
create mode 100644 hw/arm/rt595-evk.c
create mode 100644 hw/char/flexcomm_usart.c
create mode 100644 hw/i2c/flexcomm_i2c.c
create mode 100644 hw/misc/flexcomm.c
create mode 100644 hw/misc/rt500_clkctl0.c
create mode 100644 hw/misc/rt500_clkctl1.c
create mode 100644 hw/misc/rt500_rstctl.c
create mode 100644 hw/ssi/flexcomm_spi.c
create mode 100644 hw/ssi/flexspi.c
create mode 100644 tests/unit/i2c_tester.c
create mode 100644 tests/unit/spi_tester.c
create mode 100644 tests/unit/sysbus-mock.c
create mode 100644 tests/unit/test-fifo.c
create mode 100644 tests/unit/test-flexcomm-i2c.c
create mode 100644 tests/unit/test-flexcomm-spi.c
create mode 100644 tests/unit/test-flexcomm-usart.c
create mode 100644 tests/unit/test-flexcomm.c
create mode 100644 tests/unit/test-rt500-clkctl.c
create mode 100644 hw/arm/svd/meson.build
create mode 100755 scripts/svd-gen-header.py
create mode 100644 subprojects/mcux-soc-svd.wrap
create mode 100644 subprojects/packagefiles/mcux-soc-svd/meson.build
[RFC PATCH v2 00/23] NXP i.MX RT595, ARM SVD and device model unit tests
Posted by Octavian Purdila 3 months, 1 week ago
This patch set adds support for NXP's RT500 MCU [1] and the RT595
EVK[2]. More RT500 device models will be submitted in future patch sets.

The goal of this first patch set is to provide a minimal set that
allows running the NXP MCU SDK hello world example[4].

The patch set introduces a (python) tool that generates C header files
from ARM SVD files[3]. This significantly reduces the effort to write
a new device model by automatically generating: register definitions
and layout (including bit fields), register names for easier debugging
and tracing, reset register values, register write masks, etc.

The generated files are commited and not generated at compile
time. Build targets are created so that they can be easily regenerated
if needed.

It also introduces unit tests for device models. To allow accessing
registers from unit tests a system bus mock is created.

This can potentially introduce maintainance issues, due to mocks or
unit tests getting outdated when code is refactored. However, I think
this is not an issue in this case because the APIs we mocked (system
bus MMIO access) or directly used (irq APIs, chardev APIs, clock tree
APIs) to interact with device models are stable at this
point. Anecdotally, our experience seems to confirm this: we only run
into one (trivially fixed) breaking upstream change (gpio getting
removed from hwcore) in the last three years.

Changes since v1:
 * setup scripts/git.orderfile
 * svd_gen_header: add support for multi line comments
 * svd_gen_header: use qemu register fields instead of bitfields
 * svd_gen_header: generate register write mask array
 * add mcux-soc-svd as a subproject instead of commiting the rt595 SVD
   file directly
 * svd generated headers: don't generate them at compile time, instead
   commit the generated headers and create compile targets so that it is
   easy to regenerate them
 * remove pysvd dependency; developers who need to (re)generate svd
   headers need to have it installed on the host machine
 * use DEFINE_TYPES where possible
 * use g_strdup_printf where possible
 * reduce iterator variables scopes
 * hard code rt500 to use the cortex-m33
 * use static for local arrays initializers
 * remove rt500 properties as they are not yet used
 * drop private/public comments
 * remove unused headers
 * free rt500 mem regions in rt500_unrealize() instead of
   rt500_finalize
 * use memops access constraints where possible instead of
   reg32_aligned_access
 * flexspi: use 64bit properties for mmap_size property
 * rstctl: use a generic abstract type and remove the  "num" property
 * flexspi: don't call get_system_memory() directly and remove
   mmap_base property

[1] https://www.nxp.com/docs/en/data-sheet/IMXRT500EC.pdf
[2] https://www.nxp.com/webapp/Download?colCode=MIMXRT595EVKHUG
[3] https://arm-software.github.io/CMSIS_5/SVD/html/index.html
[4] Building and running the NXP MCU SDK hello world example

    Clone the following git repos:

    https://github.com/nxp-mcuxpresso/cmsis.git,
    https://github.com/nxp-mcuxpresso/mcux-sdk.git
    https://github.com/nxp-mcuxpresso/mcux-sdk-examples.git

    in the following directories: CMSIS, core, examples.

    cd examples/evkmimxrt595/demo_apps/hello_world/armgcc

    ARMGCC_DIR=/usr CFLAGS=-I../../../../../CMSIS/CMSIS/Core/Include \
      sh build_flash_debug.sh

    qemu-system-arm --machine rt595-evk -kernel flash_debug/hello_world.elf \
      -global armv7m.init-nsvtor=0x08001000 -global armv7m.init-svtor=0x08001000 \
      -chardev stdio,id=flexcomm0

Octavian Purdila (19):
  fifo32: add peek function
  tests/unit: add fifo test
  Add mcux-soc-svd subproject
  hw: add register access utility functions
  hw/misc: add basic flexcomm device model
  test/unit: add register access macros and functions
  test/unit: add flexcomm unit test
  hw/char: add support for flexcomm usart
  test/unit: add flexcomm usart unit test
  hw/i2c: add support for flexcomm i2c
  test/unit: add i2c-tester
  test/unit: add unit tests for flexcomm i2c
  test/unit: add spi-tester
  hw/misc: add support for RT500's clock controller
  test/unit: add unit tests for RT500's clock controller
  hw/ssi: add support for flexspi
  hw/misc: add support for RT500's reset controller
  hw/arm: add basic support for the RT500 SoC
  hw/arm: add RT595-EVK board

Sebastian Ene (2):
  hw/ssi: add support for flexcomm spi
  test/unit: add unit tests for flexcomm spi

Stefan Stanacar (1):
  scripts: add script to generate C header files from SVD XML files

Valentin Ghita (1):
  tests/unit: add system bus mock

 meson.build                                   |    4 +
 include/hw/arm/rt500.h                        |   44 +
 include/hw/arm/svd/flexcomm.h                 |  109 +
 include/hw/arm/svd/flexcomm_i2c.h             | 1134 ++++++
 include/hw/arm/svd/flexcomm_spi.h             |  980 ++++++
 include/hw/arm/svd/flexcomm_usart.h           |  959 +++++
 include/hw/arm/svd/flexspi.h                  | 2100 +++++++++++
 include/hw/arm/svd/rt500.h                    |   65 +
 include/hw/arm/svd/rt500_clkctl0.h            | 2287 ++++++++++++
 include/hw/arm/svd/rt500_clkctl1.h            | 3109 +++++++++++++++++
 include/hw/arm/svd/rt500_rstctl0.h            |  832 +++++
 include/hw/arm/svd/rt500_rstctl1.h            | 1344 +++++++
 include/hw/char/flexcomm_usart.h              |   20 +
 include/hw/i2c/flexcomm_i2c.h                 |   27 +
 include/hw/misc/flexcomm.h                    |   85 +
 include/hw/misc/rt500_clk_freqs.h             |   18 +
 include/hw/misc/rt500_clkctl0.h               |   37 +
 include/hw/misc/rt500_clkctl1.h               |   36 +
 include/hw/misc/rt500_rstctl.h                |   32 +
 include/hw/regs.h                             |   34 +
 include/hw/ssi/flexcomm_spi.h                 |   20 +
 include/hw/ssi/flexspi.h                      |   32 +
 include/qemu/fifo32.h                         |   28 +
 tests/unit/i2c_tester.h                       |   30 +
 tests/unit/reg-utils.h                        |   97 +
 tests/unit/spi_tester.h                       |   32 +
 tests/unit/sysbus-mock.h                      |   82 +
 hw/arm/rt500.c                                |  335 ++
 hw/arm/rt595-evk.c                            |   64 +
 hw/char/flexcomm_usart.c                      |  292 ++
 hw/i2c/flexcomm_i2c.c                         |  207 ++
 hw/misc/flexcomm.c                            |  304 ++
 hw/misc/rt500_clkctl0.c                       |  226 ++
 hw/misc/rt500_clkctl1.c                       |  207 ++
 hw/misc/rt500_rstctl.c                        |  225 ++
 hw/ssi/flexcomm_spi.c                         |  439 +++
 hw/ssi/flexspi.c                              |  157 +
 tests/unit/i2c_tester.c                       |  108 +
 tests/unit/spi_tester.c                       |   57 +
 tests/unit/sysbus-mock.c                      |  312 ++
 tests/unit/test-fifo.c                        |   97 +
 tests/unit/test-flexcomm-i2c.c                |  210 ++
 tests/unit/test-flexcomm-spi.c                |  201 ++
 tests/unit/test-flexcomm-usart.c              |  320 ++
 tests/unit/test-flexcomm.c                    |  212 ++
 tests/unit/test-rt500-clkctl.c                |  263 ++
 hw/arm/Kconfig                                |   13 +
 hw/arm/meson.build                            |    4 +
 hw/arm/svd/meson.build                        |   35 +
 hw/char/meson.build                           |    1 +
 hw/char/trace-events                          |    9 +
 hw/i2c/meson.build                            |    1 +
 hw/i2c/trace-events                           |   10 +
 hw/misc/Kconfig                               |   12 +
 hw/misc/meson.build                           |    5 +
 hw/misc/trace-events                          |   18 +
 hw/ssi/Kconfig                                |    4 +
 hw/ssi/meson.build                            |    2 +
 hw/ssi/trace-events                           |   12 +
 meson_options.txt                             |    3 +
 scripts/meson-buildoptions.sh                 |    4 +
 scripts/svd-gen-header.py                     |  343 ++
 subprojects/.gitignore                        |    1 +
 subprojects/mcux-soc-svd.wrap                 |    5 +
 .../packagefiles/mcux-soc-svd/meson.build     |    5 +
 tests/unit/meson.build                        |   56 +-
 66 files changed, 18355 insertions(+), 1 deletion(-)
 create mode 100644 include/hw/arm/rt500.h
 create mode 100644 include/hw/arm/svd/flexcomm.h
 create mode 100644 include/hw/arm/svd/flexcomm_i2c.h
 create mode 100644 include/hw/arm/svd/flexcomm_spi.h
 create mode 100644 include/hw/arm/svd/flexcomm_usart.h
 create mode 100644 include/hw/arm/svd/flexspi.h
 create mode 100644 include/hw/arm/svd/rt500.h
 create mode 100644 include/hw/arm/svd/rt500_clkctl0.h
 create mode 100644 include/hw/arm/svd/rt500_clkctl1.h
 create mode 100644 include/hw/arm/svd/rt500_rstctl0.h
 create mode 100644 include/hw/arm/svd/rt500_rstctl1.h
 create mode 100644 include/hw/char/flexcomm_usart.h
 create mode 100644 include/hw/i2c/flexcomm_i2c.h
 create mode 100644 include/hw/misc/flexcomm.h
 create mode 100644 include/hw/misc/rt500_clk_freqs.h
 create mode 100644 include/hw/misc/rt500_clkctl0.h
 create mode 100644 include/hw/misc/rt500_clkctl1.h
 create mode 100644 include/hw/misc/rt500_rstctl.h
 create mode 100644 include/hw/regs.h
 create mode 100644 include/hw/ssi/flexcomm_spi.h
 create mode 100644 include/hw/ssi/flexspi.h
 create mode 100644 tests/unit/i2c_tester.h
 create mode 100644 tests/unit/reg-utils.h
 create mode 100644 tests/unit/spi_tester.h
 create mode 100644 tests/unit/sysbus-mock.h
 create mode 100644 hw/arm/rt500.c
 create mode 100644 hw/arm/rt595-evk.c
 create mode 100644 hw/char/flexcomm_usart.c
 create mode 100644 hw/i2c/flexcomm_i2c.c
 create mode 100644 hw/misc/flexcomm.c
 create mode 100644 hw/misc/rt500_clkctl0.c
 create mode 100644 hw/misc/rt500_clkctl1.c
 create mode 100644 hw/misc/rt500_rstctl.c
 create mode 100644 hw/ssi/flexcomm_spi.c
 create mode 100644 hw/ssi/flexspi.c
 create mode 100644 tests/unit/i2c_tester.c
 create mode 100644 tests/unit/spi_tester.c
 create mode 100644 tests/unit/sysbus-mock.c
 create mode 100644 tests/unit/test-fifo.c
 create mode 100644 tests/unit/test-flexcomm-i2c.c
 create mode 100644 tests/unit/test-flexcomm-spi.c
 create mode 100644 tests/unit/test-flexcomm-usart.c
 create mode 100644 tests/unit/test-flexcomm.c
 create mode 100644 tests/unit/test-rt500-clkctl.c
 create mode 100644 hw/arm/svd/meson.build
 create mode 100755 scripts/svd-gen-header.py
 create mode 100644 subprojects/mcux-soc-svd.wrap
 create mode 100644 subprojects/packagefiles/mcux-soc-svd/meson.build

-- 
2.46.0.184.g6999bdac58-goog
Re: [RFC PATCH v2 00/23] NXP i.MX RT595, ARM SVD and device model unit tests
Posted by Peter Maydell 3 months ago
On Sat, 17 Aug 2024 at 11:26, Octavian Purdila <tavip@google.com> wrote:
>
> This patch set adds support for NXP's RT500 MCU [1] and the RT595
> EVK[2]. More RT500 device models will be submitted in future patch sets.
>
> The goal of this first patch set is to provide a minimal set that
> allows running the NXP MCU SDK hello world example[4].
>
> The patch set introduces a (python) tool that generates C header files
> from ARM SVD files[3]. This significantly reduces the effort to write
> a new device model by automatically generating: register definitions
> and layout (including bit fields), register names for easier debugging
> and tracing, reset register values, register write masks, etc.
>
> The generated files are commited and not generated at compile
> time. Build targets are created so that they can be easily regenerated
> if needed.
>
> It also introduces unit tests for device models. To allow accessing
> registers from unit tests a system bus mock is created.
>
> This can potentially introduce maintainance issues, due to mocks or
> unit tests getting outdated when code is refactored. However, I think
> this is not an issue in this case because the APIs we mocked (system
> bus MMIO access) or directly used (irq APIs, chardev APIs, clock tree
> APIs) to interact with device models are stable at this
> point. Anecdotally, our experience seems to confirm this: we only run
> into one (trivially fixed) breaking upstream change (gpio getting
> removed from hwcore) in the last three years.

My main issue with the mocking is that it introduces a
completely different way of testing devices that is
not the same as what we use for any existing device.
QEMU already has too many places where there are multiple
different ways or styles of doing something, so adding a
new one should be a high bar (e.g. "this lets us test XYZ
that would be impossible in the old way") and preferably
also have a transition plan for how we would be
deprecating and dropping the old way of doing things.

So my inclination here is to say "you said that you could
do the testing of this device with qtest, so use qtest".
If we were designing a "test devices" framework from
scratch, using mocks would probably be a strong candidate
for that design, but we aren't starting from scratch.

thanks
-- PMM
Re: [RFC PATCH v2 00/23] NXP i.MX RT595, ARM SVD and device model unit tests
Posted by Octavian Purdila 3 months ago
On Thu, Aug 22, 2024 at 6:28 AM Peter Maydell <peter.maydell@linaro.org> wrote:
>
> On Sat, 17 Aug 2024 at 11:26, Octavian Purdila <tavip@google.com> wrote:
> >
> > This patch set adds support for NXP's RT500 MCU [1] and the RT595
> > EVK[2]. More RT500 device models will be submitted in future patch sets.
> >
> > The goal of this first patch set is to provide a minimal set that
> > allows running the NXP MCU SDK hello world example[4].
> >
> > The patch set introduces a (python) tool that generates C header files
> > from ARM SVD files[3]. This significantly reduces the effort to write
> > a new device model by automatically generating: register definitions
> > and layout (including bit fields), register names for easier debugging
> > and tracing, reset register values, register write masks, etc.
> >
> > The generated files are commited and not generated at compile
> > time. Build targets are created so that they can be easily regenerated
> > if needed.
> >
> > It also introduces unit tests for device models. To allow accessing
> > registers from unit tests a system bus mock is created.
> >
> > This can potentially introduce maintainance issues, due to mocks or
> > unit tests getting outdated when code is refactored. However, I think
> > this is not an issue in this case because the APIs we mocked (system
> > bus MMIO access) or directly used (irq APIs, chardev APIs, clock tree
> > APIs) to interact with device models are stable at this
> > point. Anecdotally, our experience seems to confirm this: we only run
> > into one (trivially fixed) breaking upstream change (gpio getting
> > removed from hwcore) in the last three years.
>
> My main issue with the mocking is that it introduces a
> completely different way of testing devices that is
> not the same as what we use for any existing device.
> QEMU already has too many places where there are multiple
> different ways or styles of doing something, so adding a
> new one should be a high bar (e.g. "this lets us test XYZ
> that would be impossible in the old way") and preferably
> also have a transition plan for how we would be
> deprecating and dropping the old way of doing things.
>
> So my inclination here is to say "you said that you could
> do the testing of this device with qtest, so use qtest".

Looks like I missed some things when I looked at it, probably minor or
a matter of preference in the great scheme of things. It might still
be worth mentioning them here.

In patch 19 we are testing exposed clocks and AFAICS there are no
qtest APIs for that. We can probably add a qest API for that though.

In patch 9 we are using internal APIs exposed by the generic flexcom
device to check that device selection works as expected. I don't see
how we can reimplement that with qtest, this is a classic example of
what unit tests can enable vs functional testing.

There are also a few things that are a bit cumbersome to do with
qtest. In patch 13 and 16 we introduce i2c and spi echo test devices
to test i2c/spi transactions. I've noticed that device qtests use an
existing i2c peripheral for tests. We could add the i2c and spi test
devices to qemu and have them enabled by default. Or continue using
existing spi/i2c peripherals for testing spi/i2c controllers for
qtests which I personally don't like because it requires knowledge of
specific peripherals.

That being said, I'll go ahead and switch to qtests, if only to better
understand advantages / disadvantages and separate this patch from the
bigger device unit tests discussion.

> If we were designing a "test devices" framework from
> scratch, using mocks would probably be a strong candidate
> for that design, but we aren't starting from scratch.

I still think that writing device tests as unit tests is better:
 - devices can be tested independently
 - we can test more, including internal behaviour and APIs, error paths, etc.
 - enable sanitizers by default for device tests*

I wonder what people think and what would be a path to enable device
unit tests. What would we need to prove, besides converting existing
device qtests to unit tests, which I can certainly help with.

*I was surprised to see that sanitizers are not enabled by default on
unit tests when host dependencies are available, what is preventing
that?