On Mon, Sep 23, 2024 at 2:19 PM Sami Tolvanen <samitolvanen@google.com> wrote:
>
> Hi,
>
> Here's v3 of the DWARF modversions series [1][2]. The main
> motivation remains modversions support for Rust, which is important
> for distributions like Android that are eager to ship Rust kernel
> modules. Per Luis' request [3], v2 dropped the Rust specific bits
> from the series and instead added the feature as an option for
> the entire kernel. Matt is still addressing Rust modversion_info
> compatibility issues in a separate series [4], and we'll follow up
> with a patch to actually allow CONFIG_MODVERSIONS with Rust once
> everything else has been sorted out.
>
> A short background recap: Unlike C, Rust source code doesn't
> have sufficient information about the final ABI, as the compiler
> has considerable freedom in adjusting structure layout, for
> example, which makes using a source code parser like genksyms a
> non-starter. Based on Matt's suggestion and previous feedback from
> maintainers, this series uses DWARF debugging information for
> computing versions. DWARF is an established and a relatively stable
> format, which includes all the necessary ABI details, and adding a
> CONFIG_DEBUG_INFO dependency for Rust symbol versioning seems like a
> reasonable trade-off.
>
> The first two patches add more list macros to scripts/include and
> move the genksyms CRC32 implementation to a shared header file. The
> next 15 patches add gendwarfksyms, a tool for computing symbol
> versions from DWARF. When passed a list of exported symbols and
> object files, the tool generates an expanded type string for each
> symbol and computes symbol CRCs similarly to genksyms. gendwarfksyms
> is written in C and uses libdw to process DWARF. Patch 18 ensures
> that debugging information is present where we need it, patch 19
> adds gendwarfksyms as an alternative to genksyms, and the last patch
> adds documentation.
>
> Note that v3 is based on next-20240923 as it depends on Masahiro's
> scripts/include changes. For x86, we also need a separate small
> patch to include asm/ptrace.h in asm/ftrace.h. [5] For your
> convenience, you can find this series with all the prerequisites
> here:
>
> https://github.com/samitolvanen/linux/commits/gendwarfksyms-v3
>
> If you also want to test the series with Rust modules, this branch
> adds Matt's modversion_info series and a small patch to enable Rust
> modversions:
>
> https://github.com/samitolvanen/linux/commits/rustmodversions-v3
>
> Looking forward to hearing your thoughts!
>
> Sami
>
>
> [1] v1: https://lore.kernel.org/lkml/20240617175818.58219-17-samitolvanen@google.com/
> [2] v2: https://lore.kernel.org/lkml/20240815173903.4172139-21-samitolvanen@google.com/
> [3] https://lore.kernel.org/lkml/ZnIZEtkkQWEIGf9n@bombadil.infradead.org/
> [4] https://lore.kernel.org/lkml/20240806212106.617164-1-mmaurer@google.com/
> [5] https://lore.kernel.org/lkml/20240916221557.846853-2-samitolvanen@google.com/
>
> ---
>
> Changes in v3:
> - Updated SPX license headers.
>
> - Squashed the first two patches in v2 and tried to reduce churn as
> much as reasonable.
>
> - Dropped patch 18 from v2 ("x86/asm-prototypes: Include
> <asm/ptrace.h>") as it's addressed by a separate patch. [5]
>
> - Changed the error handling code to immediately terminate instead
> of propagating the errors back to main, which cleaned up the code
> quite a bit.
>
> - Switched to the list and hashtable implementations in scripts and
> dropped the remaining tools/include dependencies. Added a couple
> missing list macros. (patch 1)
>
> - Moved the genksyms CRC32 implementation to scripts/include and
> dropped the duplicate code. (patches 2 and 14)
>
> - Switched from ad-hoc command line parsing to getopt_long (patch 3).
>
> - Added structure member and function parameter names to the DIE
> output to match genksyms behavior, and tweaked the symtypes format
> to be more parser-friendly in general based on Petr's suggestions.
>
> - Replaced the declaration-only struct annotations with more generic
> kABI stability rules that allow source code annotations to be used
> where #ifndef __GENKSYMS__ was previously used. Added support for
> rules that can be used to exclude enumerators from versioning.
> (patch 16)
>
> - Per Miroslav's suggestion, added an option to hide structure
> members from versioning when they're added to existing alignment
> holes, for example. (patch 16)
>
> - Per Greg's request, added documentation and example macros for the
> --stable features, and a couple of test cases. (patches 15, 16, and
> 20)
>
> - Fixed making symtypes files, which need to depend on .o files with
> gendwarfksyms. (patch 19)
>
> - Addressed several other smaller issues that Petr and Masahiro
> kindly pointed out during the v2 review.
>
> Changes in v2:
> - Per Luis' request, dropped Rust-specific patches and added
> gendwarfksyms as an alternative to genksyms for the entire
> kernel.
>
> - Added support for missing DWARF features needed to handle
> also non-Rust code.
>
> - Changed symbol address matching to use the symbol table
> information instead of relying on addresses in DWARF.
>
> - Added __gendwarfksyms_ptr patches to ensure the compiler emits
> the necessary type information in DWARF even for symbols that
> are defined in other TUs.
>
> - Refactored debugging output and moved the more verbose output
> behind --dump* flags.
>
> - Added a --symtypes flag for generating a genksyms-style
> symtypes output based on Petr's feedback, and refactored
> symbol version calculations to be based on symtypes instead
> of raw --dump-dies output.
>
> - Based on feedback from Greg and Petr, added --stable flag and
> support for reserved data structure fields and declaration-onl
> structures. Also added examples for using these features.
>
> - Added a GENDWARFKSYMS option and hooked up kbuild support
> for both C and assembly code. Note that with gendwarfksyms,
> we have to actually build a temporary .o file for calculating
> assembly modversions.
>
> ---
>
> Sami Tolvanen (20):
> scripts: import more list macros
> scripts: move genksyms crc32 implementation to a common include
> tools: Add gendwarfksyms
> gendwarfksyms: Add address matching
> gendwarfksyms: Expand base_type
> gendwarfksyms: Add a cache for processed DIEs
> gendwarfksyms: Expand type modifiers and typedefs
> gendwarfksyms: Expand subroutine_type
> gendwarfksyms: Expand array_type
> gendwarfksyms: Expand structure types
> gendwarfksyms: Limit structure expansion
> gendwarfksyms: Add die_map debugging
> gendwarfksyms: Add symtypes output
> gendwarfksyms: Add symbol versioning
> gendwarfksyms: Add support for kABI rules
> gendwarfksyms: Add support for reserved and ignored fields
> gendwarfksyms: Add support for symbol type pointers
> export: Add __gendwarfksyms_ptr_ references to exported symbols
> kbuild: Add gendwarfksyms as an alternative to genksyms
> Documentation/kbuild: Add DWARF module versioning
>
> Documentation/kbuild/gendwarfksyms.rst | 274 +++++
> Documentation/kbuild/index.rst | 1 +
> include/linux/export.h | 15 +
> kernel/module/Kconfig | 31 +
> scripts/Makefile | 3 +-
> scripts/Makefile.build | 39 +-
> scripts/gendwarfksyms/.gitignore | 2 +
> scripts/gendwarfksyms/Makefile | 12 +
> scripts/gendwarfksyms/cache.c | 44 +
> scripts/gendwarfksyms/die.c | 166 +++
> scripts/gendwarfksyms/dwarf.c | 1085 +++++++++++++++++++
> scripts/gendwarfksyms/examples/kabi.h | 141 +++
> scripts/gendwarfksyms/examples/kabi_ex0.c | 86 ++
> scripts/gendwarfksyms/examples/kabi_ex1.c | 89 ++
> scripts/gendwarfksyms/examples/kabi_ex2.c | 98 ++
> scripts/gendwarfksyms/examples/kabi_rules.c | 56 +
> scripts/gendwarfksyms/examples/symbolptr.c | 29 +
> scripts/gendwarfksyms/gendwarfksyms.c | 195 ++++
> scripts/gendwarfksyms/gendwarfksyms.h | 351 ++++++
> scripts/gendwarfksyms/kabi.c | 214 ++++
> scripts/gendwarfksyms/symbols.c | 317 ++++++
> scripts/gendwarfksyms/types.c | 477 ++++++++
> scripts/genksyms/genksyms.c | 77 +-
> scripts/include/crc32.h | 93 ++
> scripts/include/list.h | 50 +
> 25 files changed, 3860 insertions(+), 85 deletions(-)
> create mode 100644 Documentation/kbuild/gendwarfksyms.rst
> create mode 100644 scripts/gendwarfksyms/.gitignore
> create mode 100644 scripts/gendwarfksyms/Makefile
> create mode 100644 scripts/gendwarfksyms/cache.c
> create mode 100644 scripts/gendwarfksyms/die.c
> create mode 100644 scripts/gendwarfksyms/dwarf.c
> create mode 100644 scripts/gendwarfksyms/examples/kabi.h
> create mode 100644 scripts/gendwarfksyms/examples/kabi_ex0.c
> create mode 100644 scripts/gendwarfksyms/examples/kabi_ex1.c
> create mode 100644 scripts/gendwarfksyms/examples/kabi_ex2.c
> create mode 100644 scripts/gendwarfksyms/examples/kabi_rules.c
> create mode 100644 scripts/gendwarfksyms/examples/symbolptr.c
> create mode 100644 scripts/gendwarfksyms/gendwarfksyms.c
> create mode 100644 scripts/gendwarfksyms/gendwarfksyms.h
> create mode 100644 scripts/gendwarfksyms/kabi.c
> create mode 100644 scripts/gendwarfksyms/symbols.c
> create mode 100644 scripts/gendwarfksyms/types.c
> create mode 100644 scripts/include/crc32.h
>
>
> base-commit: ef545bc03a65438cabe87beb1b9a15b0ffcb6ace
> --
> 2.46.0.792.g87dc391469-goog
>
This patch set looks fairly reasonable.
Acked-by: Neal Gompa <neal@gompa.dev>
--
真実はいつも一つ！/ Always, there's only one truth!