Currently the only way for Rust code to call a static inline function is
to go through a helper in rust/helpers/. This introduces performance
costs due to additional function calls and also clutters backtraces and
flame graphs with helper symbols.
To get rid of these helper symbols, provide functionality to inline
helpers into Rust using llvm-link. This option complements full LTO, by
being much cheaper and avoiding incompatibility with BTF.
I ran a microbenchmark showing the benefit of this. All the benchmark
does is call refcount_inc() in a loop. This was chosen since refcounting
is quite hot in Binder. The results are that Rust spends 6.35 ns per
call vs 5.73 ns per call in C. When enabling this option, the two
languages become equally fast, and disassembly confirms the exact same
machine code is used (in particular there is no call to
rust_helper_refcount_inc). Benchmarking Binder also results in an
improvement from this change.
This patch is complementary to:
https://lore.kernel.org/all/20251202-define-rust-helper-v1-0-a2e13cbc17a6@google.com/
Signed-off-by: Alice Ryhl <aliceryhl@google.com>
---
Alice Ryhl (1):
vmalloc: export vrealloc_node_align_noprof
Gary Guo (3):
rust: helpers: #define __rust_helper
kbuild: rust: add `CONFIG_RUSTC_CLANG_LLVM_COMPATIBLE`
build: rust: provide an option to inline C helpers into Rust
Makefile | 4 +++-
init/Kconfig | 15 +++++++++++++++
lib/Kconfig.debug | 15 +++++++++++++++
mm/vmalloc.c | 1 +
rust/Makefile | 26 ++++++++++++++++++++++----
rust/exports.c | 5 ++++-
rust/helpers/atomic.c | 5 -----
rust/helpers/helpers.c | 31 +++++++++++++++++++++++++++++++
scripts/Makefile.build | 5 ++++-
scripts/atomic/gen-rust-atomic-helpers.sh | 5 -----
10 files changed, 95 insertions(+), 17 deletions(-)
---
base-commit: 54e3eae855629702c566bd2e130d9f40e7f35bde
change-id: 20251202-inline-helpers-996f4db65e18
Best regards,
--
Alice Ryhl <aliceryhl@google.com>