From nobody Sat Jul 25 01:54:02 2026 Received: from mail-wr1-f50.google.com (mail-wr1-f50.google.com [209.85.221.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EB7C3386C1C for ; Mon, 20 Jul 2026 19:12:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784574772; cv=none; b=OwLSi6C7fNY1TTzIRNV3sl+n2dYibJPxXSIhtBSgGdHSgX696G1pErzjoUEVHmpgeyAmZJeZJIyNroGsJn9CJtLYjunRsqWC6IltEiOW5qQN1toZFzRVvsgQnoUAyDYnGGXrEdQPhDwLt78A0muWgsY5Ie6JtCvS2semrzz5uSw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784574772; c=relaxed/simple; bh=gekiQCZESIIJdVg8sXF/vRmWXF30r7W7qnWzKeGpzi0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=cC0wDNsRr1+J7/jUs75Gb2K6GO0iuHQ7DrbM2giWW1GJ7wc/4liYRhMaLWGVrGvE9Ib4YXAL4B3ePCaz1Q0WljPWzPom1D3gsUYTVDyWWmUQXK+ES5z4aYhq5yotIfXH2Cc31/r3ntHI3E3ciXO1a8MF18o0vsDIEYreg4+0XEE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=yjn-systems.com; spf=none smtp.mailfrom=yjn-systems.com; dkim=pass (2048-bit key) header.d=yjn-systems.com header.i=@yjn-systems.com header.b=Iw0IrppS; arc=none smtp.client-ip=209.85.221.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=yjn-systems.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=yjn-systems.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=yjn-systems.com header.i=@yjn-systems.com header.b="Iw0IrppS" Received: by mail-wr1-f50.google.com with SMTP id ffacd0b85a97d-47f7854678cso607129f8f.1 for ; Mon, 20 Jul 2026 12:12:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yjn-systems.com; s=google; t=1784574765; x=1785179565; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=BYix+MVqcHojTJQiHy2+juJSxN4HngDPVlffP3SS58o=; b=Iw0IrppSArlpcyi6CJjMamWGC+O2jOXapgwFEJnD8TUM6lJ6B4gyoQPDSN84n9wTTw 2U1newGuKOQwkTk/c4cvNLcg5r7xpfmfnRL986BobywYHztgKaC0UPxvNiOdbi9Pdnp7 R70tg7jYwxQZEV3GnM7VtRVarH2S8dadVEgKeUkI3P9ZjHmWfYt67aYowAWPv+6j39/H hUqr5p4t/hKpAS7Fp9zgMfm7nH/yZ35+xSD194WoyawicbRNBl7BwcBBM6VLRhEdBBca BSgkqziuqDqT9KfdxFzB7t+rYjowjYd7H71uNuwbDH1fI1qnVCViIpl9fg96xCvhZkxf RwqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784574765; x=1785179565; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=BYix+MVqcHojTJQiHy2+juJSxN4HngDPVlffP3SS58o=; b=faQmrZsbL1Gtp+NL0x/sctdzXq7jWAEgz3gd/4iAq6314OQa3jr3sItq2tHphoNdF8 sD5HAIkqkZDbDPOiQ7GJc8eT1Vce/i4mJV+pElLJ0qZY2WsAzY/NReTtWoGg3M99ABow 8itaMElyl1Iqr1AbocL+daxBrD2D8OHWxoxmpbMDKIZTFsmHhDEe16KJmZScsh0JrQvX uo3nF+AoQfytM0JJRDpyO3GPH1QfvyvdvpT8Ritp4eTXkQ3INuJ6+QIf0ceDVWv8GqpQ P49uBuxOwX+dHtd3veBgHBIeuUsUUbaeZF9SqXcvAeaB92HRG8xoh3OLgMOW078xSuox nWlg== X-Gm-Message-State: AOJu0Yxz7f9zF4pM0LrgKKthNVTq63YDjqpIJRF0p+Bmks5X1Bgmbtub iaCGZZ1absNCC7FW9q3+9bmjx4ZQNuYwgxu/ts/5ko/6o12AQ6d9lnfTDWtjsD/TwZuw X-Gm-Gg: AR+sD11hT607+2drMHgta2ZyPYONNFOiXhDJ/5N5sLqFQbYaH5MTxuhBRvdLlSvLKD6 1iXIyclThyRBg3jxIh8ME2vj21hBvI/fsJfz9vgpiL8XiEOv896YQsOPn6Hk7Bv48sDc5du+Z4S cKeLxjFU1W5F0W0+Y7FnXsPB6U+1dyL4xGmXdLG8aSWgwfepoRGndtPZWv+0V1znbE/KPyXMp1k DEiaVxMkluR2tottZ8svXlu81cHbkXXcnMpovilpyNy7S3CX3/j7cglERz/n9lduj85Ov1MdB5e XuOk7Z4tzwTMf52N0nOvZjABbX2aq7PzE213xpE9R29fLVWbyIflC1pIkEVQa7Xnzr1/dtAPs7J YlfUGbY+BgCjvRYwiZGYCMkKwC2DF77ck2x9zbi8Wl/T8338uZrIw5h+MU8o7mRbGg4nCmiC0a0 Z9WSBRP9nqMG0UianmZanhRxOE10Glz6lt/O943iYUV8BXNmvFpY/ji+5cFA/EmF1DH5MeJUGNY qlZMs1sjYYZu97Y7oQ= X-Received: by 2002:a05:6000:1ace:b0:47f:6f98:498e with SMTP id ffacd0b85a97d-47f6f984b9dmr10760036f8f.24.1784574764503; Mon, 20 Jul 2026 12:12:44 -0700 (PDT) Received: from yjn-Zenbook-UX3404VA-UX3404VA.. (p200300dcbf448c00c9c364f8ed993120.dip0.t-ipconnect.de. [2003:dc:bf44:8c00:c9c3:64f8:ed99:3120]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-47f63eddd1csm32209266f8f.29.2026.07.20.12.12.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Jul 2026 12:12:44 -0700 (PDT) From: York Jasper Niebuhr To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kees@kernel.org, franzen@sec.in.tum.de, ardb@kernel.org Subject: [RFC v3 1/5] Pinpoint plugin Date: Mon, 20 Jul 2026 21:12:43 +0200 Message-ID: <20260720191243.21571-1-yjn@yjn-systems.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260720191146.21473-1-yjn@yjn-systems.com> References: <20260720191146.21473-1-yjn@yjn-systems.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Signed-off-by: York Jasper Niebuhr --- scripts/Makefile.gcc-plugins | 9 + scripts/gcc-plugins/Makefile | 18 + scripts/gcc-plugins/asm_offset_pass.c | 79 +++ scripts/gcc-plugins/dpin_registry.c | 199 +++++++ scripts/gcc-plugins/dpin_registry.h | 22 + scripts/gcc-plugins/ipin_registry.c | 367 +++++++++++++ scripts/gcc-plugins/ipin_registry.h | 57 ++ scripts/gcc-plugins/layout_hash.c | 61 +++ scripts/gcc-plugins/layout_hash.h | 8 + scripts/gcc-plugins/on_finish_decl.c | 8 + scripts/gcc-plugins/on_finish_type.c | 12 + scripts/gcc-plugins/on_finish_unit.c | 336 ++++++++++++ .../gcc-plugins/on_preserve_component_ref.c | 99 ++++ scripts/gcc-plugins/on_register_attributes.c | 52 ++ scripts/gcc-plugins/on_start_unit.c | 11 + scripts/gcc-plugins/passes.h | 31 ++ scripts/gcc-plugins/pinpoint.c | 103 ++++ scripts/gcc-plugins/pinpoint.h | 75 +++ .../gcc-plugins/rtl_ipin_survival_scan_pass.c | 37 ++ scripts/gcc-plugins/safe-attribs.h | 9 + scripts/gcc-plugins/safe-diagnostic.h | 10 + scripts/gcc-plugins/safe-gcc-plugin.h | 6 + scripts/gcc-plugins/safe-ggc.h | 8 + scripts/gcc-plugins/safe-gimple.h | 14 + scripts/gcc-plugins/safe-input.h | 9 + scripts/gcc-plugins/safe-langhooks.h | 8 + scripts/gcc-plugins/safe-md5.h | 8 + scripts/gcc-plugins/safe-output.h | 8 + scripts/gcc-plugins/safe-plugin-version.h | 8 + scripts/gcc-plugins/safe-rtl.h | 17 + scripts/gcc-plugins/safe-tree.h | 10 + scripts/gcc-plugins/separate_offset_pass.c | 327 ++++++++++++ scripts/gcc-plugins/serialize.c | 303 +++++++++++ scripts/gcc-plugins/serialize.h | 115 ++++ .../gcc-plugins/target_hash_builtin_pass.c | 167 ++++++ scripts/gcc-plugins/target_registry.c | 492 ++++++++++++++++++ scripts/gcc-plugins/target_registry.h | 59 +++ 37 files changed, 3162 insertions(+) create mode 100644 scripts/gcc-plugins/asm_offset_pass.c create mode 100644 scripts/gcc-plugins/dpin_registry.c create mode 100644 scripts/gcc-plugins/dpin_registry.h create mode 100644 scripts/gcc-plugins/ipin_registry.c create mode 100644 scripts/gcc-plugins/ipin_registry.h create mode 100644 scripts/gcc-plugins/layout_hash.c create mode 100644 scripts/gcc-plugins/layout_hash.h create mode 100644 scripts/gcc-plugins/on_finish_decl.c create mode 100644 scripts/gcc-plugins/on_finish_type.c create mode 100644 scripts/gcc-plugins/on_finish_unit.c create mode 100644 scripts/gcc-plugins/on_preserve_component_ref.c create mode 100644 scripts/gcc-plugins/on_register_attributes.c create mode 100644 scripts/gcc-plugins/on_start_unit.c create mode 100644 scripts/gcc-plugins/passes.h create mode 100644 scripts/gcc-plugins/pinpoint.c create mode 100644 scripts/gcc-plugins/pinpoint.h create mode 100644 scripts/gcc-plugins/rtl_ipin_survival_scan_pass.c create mode 100644 scripts/gcc-plugins/safe-attribs.h create mode 100644 scripts/gcc-plugins/safe-diagnostic.h create mode 100644 scripts/gcc-plugins/safe-gcc-plugin.h create mode 100644 scripts/gcc-plugins/safe-ggc.h create mode 100644 scripts/gcc-plugins/safe-gimple.h create mode 100644 scripts/gcc-plugins/safe-input.h create mode 100644 scripts/gcc-plugins/safe-langhooks.h create mode 100644 scripts/gcc-plugins/safe-md5.h create mode 100644 scripts/gcc-plugins/safe-output.h create mode 100644 scripts/gcc-plugins/safe-plugin-version.h create mode 100644 scripts/gcc-plugins/safe-rtl.h create mode 100644 scripts/gcc-plugins/safe-tree.h create mode 100644 scripts/gcc-plugins/separate_offset_pass.c create mode 100644 scripts/gcc-plugins/serialize.c create mode 100644 scripts/gcc-plugins/serialize.h create mode 100644 scripts/gcc-plugins/target_hash_builtin_pass.c create mode 100644 scripts/gcc-plugins/target_registry.c create mode 100644 scripts/gcc-plugins/target_registry.h diff --git a/scripts/Makefile.gcc-plugins b/scripts/Makefile.gcc-plugins index b0e1423b09c2..2ae9d571d812 100644 --- a/scripts/Makefile.gcc-plugins +++ b/scripts/Makefile.gcc-plugins @@ -8,6 +8,15 @@ ifdef CONFIG_GCC_PLUGIN_LATENT_ENTROPY endif export DISABLE_LATENT_ENTROPY_PLUGIN =20 +PINPOINT_PLUGIN_CFLAGS :=3D \ + -fplugin=3D$(objtree)/scripts/gcc-plugins/pinpoint_plugin.so \ + -D__SPSLR__ + +gcc-plugin-$(CONFIG_SPSLR) +=3D pinpoint_plugin.so +gcc-plugin-cflags-$(CONFIG_SPSLR) +=3D -D__SPSLR__ + +export PINPOINT_PLUGIN_CFLAGS + # All the plugin CFLAGS are collected here in case a build target needs to # filter them out of the KBUILD_CFLAGS. GCC_PLUGINS_CFLAGS :=3D $(strip $(addprefix -fplugin=3D$(objtree)/scripts/= gcc-plugins/, $(gcc-plugin-y)) $(gcc-plugin-cflags-y)) -DGCC_PLUGINS diff --git a/scripts/gcc-plugins/Makefile b/scripts/gcc-plugins/Makefile index 05b14aba41ef..1ecc9e923b85 100644 --- a/scripts/gcc-plugins/Makefile +++ b/scripts/gcc-plugins/Makefile @@ -22,6 +22,24 @@ targets +=3D randomize_layout_seed.h # # foo-objs :=3D foo.o foo2.o =20 +pinpoint_plugin-objs :=3D \ + pinpoint.o \ + asm_offset_pass.o \ + dpin_registry.o \ + ipin_registry.o \ + layout_hash.o \ + on_finish_decl.o \ + on_finish_type.o \ + on_finish_unit.o \ + on_preserve_component_ref.o \ + on_register_attributes.o \ + on_start_unit.o \ + rtl_ipin_survival_scan_pass.o \ + separate_offset_pass.o \ + serialize.o \ + target_registry.o \ + target_hash_builtin_pass.o + always-y +=3D $(GCC_PLUGIN) =20 GCC_PLUGINS_DIR =3D $(shell $(CC) -print-file-name=3Dplugin) diff --git a/scripts/gcc-plugins/asm_offset_pass.c b/scripts/gcc-plugins/as= m_offset_pass.c new file mode 100644 index 000000000000..5a1be25f1f2c --- /dev/null +++ b/scripts/gcc-plugins/asm_offset_pass.c @@ -0,0 +1,79 @@ +#include + +#include +#include +#include + +/* + * gsi_replace() changes the statement, but SSA names that were defined by= the + * separator call still point at the old def statement. Retarget those SSA= defs + * so later GCC passes see the asm marker as the producer of the offset va= lue. + */ + +static void pin_update_ssa_def(function *fn, gimple *old_def, gimple *new_= def) +{ + if (!fn || !old_def) + return; + + // Stage 0 separator call was definition statement of temporary variable + + unsigned i; + tree name; + FOR_EACH_SSA_NAME(i, name, fn) + { + if (!name) + continue; + + if (SSA_NAME_DEF_STMT(name) !=3D old_def) + continue; + + SSA_NAME_DEF_STMT(name) =3D new_def; + } +} + +static void pin_assemble_maybe(function *fn, gimple_stmt_iterator *gsi) +{ + if (!gsi) + return; + + gimple *stmt =3D gsi_stmt(*gsi); + if (!stmt) + return; + + ipin::handle pin =3D ipin::identify_gimple_separator(stmt); + if (pin =3D=3D ipin::invalid) + return; + + gimple *replacement =3D ipin::make_gimple_pin(gimple_call_lhs(stmt), pin); + if (!replacement) + pinpoint_fatal("failed to construct ipin"); + + gsi_replace(gsi, replacement, true); + pin_update_ssa_def(fn, stmt, replacement); +} + +static const pass_data asm_offset_pass_data =3D { + GIMPLE_PASS, "asm_offset", OPTGROUP_NONE, TV_NONE, 0, 0, 0, + 0, TODO_update_ssa +}; + +asm_offset_pass::asm_offset_pass(gcc::context *ctxt) + : gimple_opt_pass(asm_offset_pass_data, ctxt) +{ +} + +unsigned int asm_offset_pass::execute(function *fn) +{ + if (!fn) + return 0; + + basic_block bb; + FOR_EACH_BB_FN(bb, fn) + { + for (gimple_stmt_iterator gsi =3D gsi_start_bb(bb); + !gsi_end_p(gsi); gsi_next(&gsi)) + pin_assemble_maybe(fn, &gsi); + } + + return 0; +} diff --git a/scripts/gcc-plugins/dpin_registry.c b/scripts/gcc-plugins/dpin= _registry.c new file mode 100644 index 000000000000..415f7befe296 --- /dev/null +++ b/scripts/gcc-plugins/dpin_registry.c @@ -0,0 +1,199 @@ +#include +#include + +#include +#include +#include + +static std::list pins; +static std::unordered_set seen_dpin_symbols; + +PINPOINT_GC_PRESERVE_CALLBACK() +{ + for (const dpin &pin : pins) + for (const dpin::component &c : pin.components) + PINPOINT_GC_MARK_TREE(c.target); +} + +void dpin::reset() +{ + pins.clear(); + seen_dpin_symbols.clear(); +} + +const std::list &dpin::inspect() +{ + return pins; +} + +static std::list compile_datapin_components(tree type); +static std::list compile_datapin_record_components(tree t= ype); +static std::list compile_datapin_array_components(tree ty= pe); + +/* + * Build the list of randomized objects contained in a static object. + * + * A dpin may describe the object itself, nested target structs, arrays of + * targets, or targets embedded inside non-target containers. The level fi= eld + * records nesting depth so the runtime can patch inner objects before + * their containing objects. + */ + +static std::list compile_datapin_components(tree type) +{ + std::list components; + + tree relevant =3D target::main_variant(type); + if (target::is_target(relevant)) { + components.push_back(dpin::component{ + .offset =3D 0, + .level =3D 0, + .target =3D relevant, + }); + } + + std::list sub_components; + if (TREE_CODE(type) =3D=3D RECORD_TYPE) + sub_components =3D compile_datapin_record_components(type); + else if (TREE_CODE(type) =3D=3D ARRAY_TYPE) + sub_components =3D compile_datapin_array_components(type); + + for (const dpin::component &sc : sub_components) { + components.push_back(dpin::component{ + .offset =3D sc.offset, + .level =3D sc.level + 1, + .target =3D sc.target, + }); + } + + // Note -> should probably make sure that randomized structs are never us= ed in unions! + return components; +} + +static std::list compile_datapin_record_components(tree t= ype) +{ + std::list components; + + if (TREE_CODE(type) !=3D RECORD_TYPE || !COMPLETE_TYPE_P(type)) + return components; + + for (tree field =3D TYPE_FIELDS(type); field; field =3D TREE_CHAIN(field)= ) { + if (TREE_CODE(field) !=3D FIELD_DECL) + continue; + + if (!target::field_has_size(field)) + continue; // flexible array member / dynamic-size trailing array + + if (target::field_is_bitfield(field)) + continue; + + std::size_t field_offset =3D target::field_offset(field); + tree field_type =3D TREE_TYPE(field); + + std::list field_components =3D + compile_datapin_components(field_type); + + for (const dpin::component &fc : field_components) { + components.push_back(dpin::component{ + .offset =3D field_offset + fc.offset, + .level =3D fc.level, + .target =3D fc.target, + }); + } + } + + return components; +} + +static std::list compile_datapin_array_components(tree ty= pe) +{ + std::list components; + + if (TREE_CODE(type) !=3D ARRAY_TYPE) + return components; + + tree elem_type =3D TREE_TYPE(type); + if (!elem_type) + return components; + + std::list elem_components =3D + compile_datapin_components(elem_type); + if (elem_components.empty()) + return components; + + tree domain =3D TYPE_DOMAIN(type); + if (!domain) + pinpoint_fatal("dpin: failed to get array domain"); + + tree min_t =3D TYPE_MIN_VALUE(domain); + tree max_t =3D TYPE_MAX_VALUE(domain); + if (!min_t || !max_t || TREE_CODE(min_t) !=3D INTEGER_CST || + TREE_CODE(max_t) !=3D INTEGER_CST) + pinpoint_fatal("dpin: failed to get constant array bounds"); + + HOST_WIDE_INT min_i =3D tree_to_shwi(min_t); + HOST_WIDE_INT max_i =3D tree_to_shwi(max_t); + + tree elem_size_t =3D TYPE_SIZE_UNIT(elem_type); + if (!elem_size_t || TREE_CODE(elem_size_t) !=3D INTEGER_CST) + pinpoint_fatal("dpin: failed to get constant element size"); + + std::size_t elem_size =3D tree_to_uhwi(elem_size_t); + + for (HOST_WIDE_INT i =3D min_i; i <=3D max_i; ++i) { + std::size_t element_offset =3D + static_cast(i - min_i) * elem_size; + + for (const dpin::component &ec : elem_components) { + components.push_back(dpin::component{ + .offset =3D element_offset + ec.offset, + .level =3D ec.level, + .target =3D ec.target, + }); + } + } + + return components; +} + +static bool compile_datapin(tree type, dpin &pin) +{ + pin.components =3D compile_datapin_components(type); + return !pin.components.empty(); +} + +void dpin::consider_static_var(tree var) +{ + if (!var || TREE_CODE(var) !=3D VAR_DECL) + return; + + if (!TREE_STATIC(var) || DECL_EXTERNAL(var)) + return; + + tree type =3D TREE_TYPE(var); + if (!type) + return; + + tree symbol_tree =3D DECL_ASSEMBLER_NAME(var); + const char *symbol =3D symbol_tree ? IDENTIFIER_POINTER(symbol_tree) : + nullptr; + if (!symbol) + pinpoint_fatal("dpin: failed to get static variable symbol"); + + std::string sym{ symbol }; + + /* + * Multiple VAR_DECLs can name the same emitted object, for example throu= gh + * export or alias machinery. Emit at most one dpin per assembler symbol. + */ + if (!seen_dpin_symbols.insert(sym).second) + return; + + dpin pin; + if (!compile_datapin(type, pin)) + return; + + DECL_PRESERVE_P(var) =3D 1; + pin.symbol =3D sym; + pins.emplace_back(std::move(pin)); +} diff --git a/scripts/gcc-plugins/dpin_registry.h b/scripts/gcc-plugins/dpin= _registry.h new file mode 100644 index 000000000000..500535ce0bf2 --- /dev/null +++ b/scripts/gcc-plugins/dpin_registry.h @@ -0,0 +1,22 @@ +#pragma once + +#include +#include +#include + +#include + +struct dpin { + struct component { + std::size_t offset =3D 0; + std::size_t level =3D 0; + tree target =3D NULL_TREE; + }; + + std::string symbol; + std::list components; + + static void consider_static_var(tree var); + static void reset(); + static const std::list &inspect(); +}; diff --git a/scripts/gcc-plugins/ipin_registry.c b/scripts/gcc-plugins/ipin= _registry.c new file mode 100644 index 000000000000..41e2d525a08f --- /dev/null +++ b/scripts/gcc-plugins/ipin_registry.c @@ -0,0 +1,367 @@ +#include +#include + +#include +#include +#include + +#define PINPOINT_SEPARATOR "__spslr_offsetof" +#define PINPOINT_IPIN_MARKER "spslr_ipin_marker" +#define PINPOINT_IPIN_SYMBOL_PREFIX "spslr_ipin_" /* suffixed with ""= */ +#define PINPOINT_IPIN_WIDTH_SYMBOL_PREFIX \ + "spslr_ipin_width_" /* suffixed with "" */ + +static ipin::handle next_ipin_handle =3D 0; +static std::map ipins; + +static tree separator_decl =3D NULL_TREE; + +PINPOINT_GC_PRESERVE_CALLBACK() +{ + PINPOINT_GC_MARK_TREE(separator_decl); + + for (const auto &[h, pin] : ipins) + PINPOINT_GC_MARK_TREE(pin.field); +} + +/* + * Separators are compiler-internal marker calls, not runtime calls. + * + * They carry a unique ID with which metadata is associated. The call is + * declared pure/no-vops so GCC treats it as having no memory side effects= ; a + * later pinpoint pass must remove every separator before code generation. + */ + +static tree make_separator_decl() +{ + if (separator_decl) + return separator_decl; + + tree args =3D tree_cons(NULL_TREE, sizetype, NULL_TREE); + tree type =3D build_function_type(sizetype, args); + + tree tmp_decl =3D build_fn_decl(PINPOINT_SEPARATOR, type); + if (!tmp_decl) + return NULL_TREE; + + DECL_EXTERNAL(tmp_decl) =3D 1; + TREE_PUBLIC(tmp_decl) =3D 1; + DECL_ARTIFICIAL(tmp_decl) =3D 1; + + /* Prevent VOP problems later when removing calls (VOPs mark memory + side-effects, which these calls have none of anyways) */ + DECL_PURE_P(tmp_decl) =3D 1; + DECL_IS_NOVOPS(tmp_decl) =3D 1; + + return (separator_decl =3D tmp_decl); +} + +static ipin *get_pin(ipin::handle pin) +{ + auto it =3D ipins.find(pin); + return it =3D=3D ipins.end() ? nullptr : &it->second; +} + +ipin::handle ipin::make(tree field) +{ + handle h =3D next_ipin_handle++; + + ipin pin; + pin.status =3D state::pending; + pin.field =3D field; + + ipins.emplace(h, std::move(pin)); + return h; +} + +tree ipin::make_ast_separator(ipin::handle pin) +{ + ipin *p =3D get_pin(pin); + if (!p) + pinpoint_fatal("ipin: inknown pin in make_ast_separator"); + + tree decl =3D make_separator_decl(); + if (!decl) + return NULL_TREE; + + tree arg0 =3D size_int(pin); + if (!arg0) + return NULL_TREE; + + p->status =3D state::separator; + return build_call_expr(decl, 1, arg0); +} + +gimple *ipin::make_gimple_separator(tree lhs, ipin::handle pin) +{ + if (!lhs) + return nullptr; + + ipin *p =3D get_pin(pin); + if (!p) + pinpoint_fatal("ipin: unknown pin in make_gimple_separator"); + + tree decl =3D make_separator_decl(); + if (!decl) + return nullptr; + + tree arg0 =3D size_int(pin); + if (!arg0) + return nullptr; + + gimple *call =3D gimple_build_call(decl, 1, arg0); + if (!call) + return nullptr; + + gimple_call_set_lhs(call, lhs); + p->status =3D state::separator; + return call; +} + +static bool decl_is_separator(tree fndecl) +{ + if (!fndecl) + return false; + + tree name_tree =3D DECL_NAME(fndecl); + if (!name_tree) + return false; + + const char *name =3D IDENTIFIER_POINTER(name_tree); + if (!name) + return false; + + return strcmp(name, PINPOINT_SEPARATOR) =3D=3D 0; +} + +ipin::handle ipin::identify_gimple_separator(gimple *stmt) +{ + if (!stmt || !is_gimple_call(stmt)) + return invalid; + + tree fndecl =3D gimple_call_fndecl(stmt); + if (!decl_is_separator(fndecl)) + return invalid; + + tree arg0 =3D gimple_call_arg(stmt, 0); + if (!arg0 || TREE_CODE(arg0) !=3D INTEGER_CST) + pinpoint_fatal("ipin: separator has invalid ipin handle"); + + return static_cast(tree_to_uhwi(arg0)); +} + +static bool lhs_type_is_64bit(tree lhs) +{ + if (!lhs) + return false; + + tree type =3D TREE_TYPE(lhs); + if (!type) + return false; + + tree size =3D TYPE_SIZE(type); + if (!size || TREE_CODE(size) !=3D INTEGER_CST) + return false; + + return tree_to_uhwi(size) =3D=3D 64; +} + +static tree make_asm_operand(const char *constraint_text, tree operand_tre= e) +{ + tree constraint_str =3D + build_string(strlen(constraint_text) + 1, constraint_text); + tree inner_list =3D build_tree_list(NULL_TREE, constraint_str); + tree outer_list =3D build_tree_list(inner_list, operand_tree); + return outer_list; +} + +/* + * The symbol does not denote executable code as a callable function; it d= enotes + * the four-byte immediate field inside the instruction stream. + */ +static std::string make_final_x86_64_asm(const std::string &field_symbol, + const std::string &width_symbol, + std::size_t imm) +{ + char buf[512]; + + std::snprintf(buf, sizeof(buf), + "# %s\n" + " movq $fieldlabel(%zu, %s, %s), %%0", + PINPOINT_IPIN_MARKER, imm, field_symbol.c_str(), + width_symbol.c_str()); + + return std::string(buf); +} + +gimple *ipin::make_gimple_pin(tree lhs, ipin::handle pin) +{ + if (!lhs) + return nullptr; + + ipin *p =3D get_pin(pin); + if (!p) + pinpoint_fatal("ipin: unknown pin in make_gimple_pin"); + + if (!lhs_type_is_64bit(lhs)) + pinpoint_fatal("ipin: expected 64-bit destination type"); + + /* + * Do not emit the final label here. GCC may duplicate this asm later. + * The original pin id is carried only as an input operand. + */ + std::string asm_str =3D "# " PINPOINT_IPIN_MARKER; + + tree arg0 =3D build_int_cst(size_type_node, pin); + + vec *outputs =3D NULL; + vec *inputs =3D NULL; + + vec_safe_push(outputs, make_asm_operand("=3Dr", lhs)); + vec_safe_push(inputs, make_asm_operand("i", arg0)); + + gasm *new_gasm =3D gimple_build_asm_vec(ggc_strdup(asm_str.c_str()), + inputs, outputs, NULL, NULL); + if (!new_gasm) + return nullptr; + + /* + * Non-volatile is intentional: unused field-offset computations should d= ie + * normally. Only offsets that survive optimization become instruction pi= ns. + */ + gimple_asm_set_volatile(new_gasm, false); + + p->status =3D state::pin; + return new_gasm; +} + +static bool extract_asm_operands(rtx x, rtx &asm_out) +{ + if (!x) + return false; + + if (GET_CODE(x) =3D=3D ASM_OPERANDS) { + asm_out =3D x; + return true; + } + + if (GET_CODE(x) =3D=3D SET) + return extract_asm_operands(SET_SRC(x), asm_out); + + if (GET_CODE(x) =3D=3D PARALLEL) { + for (int i =3D 0; i < XVECLEN(x, 0); ++i) { + if (extract_asm_operands(XVECEXP(x, 0, i), asm_out)) + return true; + } + } + + return false; +} + +ipin::handle ipin::identify_rtl_pin(rtx x) +{ + rtx asm_rtx =3D nullptr; + if (!extract_asm_operands(x, asm_rtx)) + return invalid; + + if (!asm_rtx || GET_CODE(asm_rtx) !=3D ASM_OPERANDS) + return invalid; + + const char *templ =3D ASM_OPERANDS_TEMPLATE(asm_rtx); + if (!templ || !std::strstr(templ, PINPOINT_IPIN_MARKER)) + return invalid; + + if (ASM_OPERANDS_INPUT_LENGTH(asm_rtx) !=3D 1) + pinpoint_fatal( + "ipin: RTL pin asm has invalid number of inputs"); + + rtx in0 =3D ASM_OPERANDS_INPUT(asm_rtx, 0); + if (!CONST_INT_P(in0)) + pinpoint_fatal("ipin: RTL ipin id is not CONST_INT"); + + return static_cast(INTVAL(in0)); +} + +void ipin::mark_live(ipin::handle pin, rtx at) +{ + ipin::handle found =3D identify_rtl_pin(at); + + if (found !=3D pin) { + pinpoint_fatal( + "ipin: mark_live called with mismatching RTL pin"); + } + + rtx asm_rtx =3D nullptr; + + if (!extract_asm_operands(at, asm_rtx) || !asm_rtx || + GET_CODE(asm_rtx) !=3D ASM_OPERANDS) { + pinpoint_fatal( + "ipin: mark_live could not recover ASM_OPERANDS"); + } + + ipin *p =3D get_pin(pin); + if (!p) { + pinpoint_fatal("ipin: tried to mark unknown pin live"); + } + + ipin::handle live_handle =3D pin; + ipin *live_pin =3D p; + + if (p->status =3D=3D state::live) { + live_handle =3D next_ipin_handle++; + + ipin clone =3D *p; + clone.status =3D state::pin; + clone.symbol.clear(); + clone.width_symbol.clear(); + + auto inserted =3D ipins.emplace(live_handle, std::move(clone)); + live_pin =3D &inserted.first->second; + } + + live_pin->status =3D state::live; + + const std::string handle_suffix =3D std::to_string(live_handle); + + live_pin->symbol =3D + ".L" + std::string(PINPOINT_IPIN_SYMBOL_PREFIX) + handle_suffix; + + live_pin->width_symbol =3D + ".L" + std::string(PINPOINT_IPIN_WIDTH_SYMBOL_PREFIX) + + handle_suffix; + + std::size_t offset =3D target::field_offset(live_pin->field); + + std::string final_asm =3D make_final_x86_64_asm( + live_pin->symbol, live_pin->width_symbol, offset); + + XSTR(asm_rtx, 0) =3D ggc_strdup(final_asm.c_str()); +} + +std::size_t ipin::live_count() +{ + std::size_t n =3D 0; + + for (const auto &[h, pin] : ipins) { + if (pin.status =3D=3D state::live) + ++n; + } + + return n; +} + +const std::map &ipin::inspect() +{ + return ipins; +} + +const ipin *ipin::inspect(ipin::handle pin) +{ + return get_pin(pin); +} + +void ipin::reset() +{ + ipins.clear(); + next_ipin_handle =3D 0; +} diff --git a/scripts/gcc-plugins/ipin_registry.h b/scripts/gcc-plugins/ipin= _registry.h new file mode 100644 index 000000000000..0fff090ed6e7 --- /dev/null +++ b/scripts/gcc-plugins/ipin_registry.h @@ -0,0 +1,57 @@ +#pragma once +#include +#include +#include +#include + +#include +#include +#include + +struct ipin { + enum class state { pending, separator, pin, live }; + + state status =3D state::pending; + + /* Field that the ipin refers to */ + tree field =3D NULL_TREE; + + /* + * The field-address and field-width symbols are available once the pin is + * marked live and its final inline assembly has been constructed. + */ + std::string symbol; + std::string width_symbol; + + using handle =3D std::size_t; + static constexpr handle invalid =3D std::numeric_limits::max(); + + /* Register new ipin to track through compilation */ + static handle make(tree field); + + /* Construct an AST separator tree refering to an ipin */ + static tree make_ast_separator(handle pin); + + /* Construct a GIMPLE separator statement refering to an ipin */ + static gimple *make_gimple_separator(tree lhs, handle pin); + + /* Check if a GIMPLE statement is a separator refering to an ipin */ + static handle identify_gimple_separator(gimple *stmt); + + /* Construct a GIMPLE ipin - currently as ASM statement */ + static gimple *make_gimple_pin(tree lhs, handle pin); + + /* Check if an RTL instruction is an ipin */ + static handle identify_rtl_pin(rtx x); + + /* Mark an ipin as being present in the final asm */ + static void mark_live(handle pin, rtx at); + static std::size_t live_count(); + + /* Inspect the state of one or more ipins */ + static const std::map &inspect(); + static const ipin *inspect(handle pin); + + /* Reset ipin registry */ + static void reset(); +}; diff --git a/scripts/gcc-plugins/layout_hash.c b/scripts/gcc-plugins/layout= _hash.c new file mode 100644 index 000000000000..6200bd037f54 --- /dev/null +++ b/scripts/gcc-plugins/layout_hash.c @@ -0,0 +1,61 @@ +#include +#include +#include + +#include +#include + +#include + +namespace +{ + +void append_u64(std::string &buf, std::uint64_t v) +{ + for (unsigned i =3D 0; i < 8; i++) + buf.push_back(static_cast((v >> (i * 8)) & 0xff)); +} + +void append_size(std::string &buf, std::size_t v) +{ + append_u64(buf, static_cast(v)); +} + +void append_string(std::string &buf, const std::string &s) +{ + append_size(buf, s.size()); + buf.append(s); +} + +} + +std::array compute_layout_hash(tree target_type) +{ + std::string buf; + + append_string(buf, "spslr-layout-hash-v2"); + + for (const std::string &ctx : target::context_chain(target_type)) + append_string(buf, ctx); + + append_string(buf, target::name(target_type)); + append_size(buf, target::size(target_type)); + + for (const target::compressed_field &field : + target::compressed_fields(target_type)) { + append_string(buf, field.name); + append_size(buf, field.size); + append_size(buf, field.offset); + append_size(buf, field.alignment); + append_size(buf, field.fixed ? 1 : 0); + } + + unsigned char digest[16]; + md5_buffer(buf.data(), buf.size(), digest); + + std::array out; + for (std::size_t i =3D 0; i < out.size(); i++) + out[i] =3D static_cast(digest[i]); + + return out; +} diff --git a/scripts/gcc-plugins/layout_hash.h b/scripts/gcc-plugins/layout= _hash.h new file mode 100644 index 000000000000..b4b660bd0b96 --- /dev/null +++ b/scripts/gcc-plugins/layout_hash.h @@ -0,0 +1,8 @@ +#pragma once + +#include +#include + +#include + +std::array compute_layout_hash(tree target_type); diff --git a/scripts/gcc-plugins/on_finish_decl.c b/scripts/gcc-plugins/on_= finish_decl.c new file mode 100644 index 000000000000..6781d2516e75 --- /dev/null +++ b/scripts/gcc-plugins/on_finish_decl.c @@ -0,0 +1,8 @@ +#include +#include + +void on_finish_decl(void *plugin_data, void *user_data) +{ + tree decl =3D (tree)plugin_data; + dpin::consider_static_var(decl); +} diff --git a/scripts/gcc-plugins/on_finish_type.c b/scripts/gcc-plugins/on_= finish_type.c new file mode 100644 index 000000000000..023470f8cc8c --- /dev/null +++ b/scripts/gcc-plugins/on_finish_type.c @@ -0,0 +1,12 @@ +#include +#include + +void on_finish_type(void *plugin_data, void *user_data) +{ + tree t =3D target::main_variant((tree)plugin_data); + + if (!target::is_target(t)) + return; + + target::validate(t); +} diff --git a/scripts/gcc-plugins/on_finish_unit.c b/scripts/gcc-plugins/on_= finish_unit.c new file mode 100644 index 000000000000..5229dbb53c52 --- /dev/null +++ b/scripts/gcc-plugins/on_finish_unit.c @@ -0,0 +1,336 @@ +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include + +/* + * Finish-unit emits the per-compilation-unit metadata into the object asm. + * + * Only information that survived all earlier filtering should be dumped h= ere: + * target layouts, data pins for static objects, and live instruction pins + * whose immediates exist in the final object. + */ + +namespace +{ + +static std::string src_filename() +{ + namespace fs =3D std::filesystem; + + if (!main_input_filename) + return {}; + + return fs::weakly_canonical(main_input_filename).generic_string(); +} + +using selfpatch::hash16_t; + +struct emitted_target { + tree target; + hash16_t hash; + std::size_t unit_target_idx; +}; + +struct emitted_dpin { + std::string addr_expr; + std::size_t unit_target_idx; +}; + +static hash16_t to_hash16(const std::array &in) +{ + hash16_t out{}; + + for (std::size_t i =3D 0; i < out.size(); ++i) + out[i] =3D static_cast(in[i]); + + return out; +} + +static std::string add_offset_expr(const std::string &symbol, std::size_t = off) +{ + if (off =3D=3D 0) + return symbol; + + return symbol + " + " + std::to_string(off); +} + +static std::vector collect_all_targets() +{ + std::vector out; + out.reserve(target::target_count()); + + target::iterate_targets([&](tree t) { + out.push_back({ + .target =3D t, + .hash =3D to_hash16(target::layout_hash(t)), + .unit_target_idx =3D out.size(), + }); + }); + + return out; +} + +static std::unordered_map +make_unit_target_idx_map(const std::vector &targets) +{ + std::unordered_map out; + + for (const emitted_target &target : targets) + out[target.target] =3D target.unit_target_idx; + + return out; +} + +static std::vector +collect_dpins(const std::unordered_map &target_idx) +{ + std::vector out; + + for (const dpin &pin : dpin::inspect()) { + if (pin.symbol.empty()) + pinpoint_fatal( + "finish-unit: data pin has empty symbol"); + + std::vector components(pin.components.begin(), + pin.components.end()); + + /* + * Data pins with deeper nesting level must be patched first. + */ + std::sort(components.begin(), components.end(), + [](const dpin::component &a, + const dpin::component &b) { + return a.level > b.level; + }); + + for (const dpin::component &c : components) { + const auto target_it =3D target_idx.find(c.target); + + if (target_it =3D=3D target_idx.end()) + pinpoint_fatal( + "finish-unit: dpin target not in unit target map"); + + out.push_back({ + .addr_expr =3D + add_offset_expr(pin.symbol, c.offset), + .unit_target_idx =3D target_it->second, + }); + } + } + + return out; +} + +static void emit_target_metadata(FILE *out, + const std::vector &targets) +{ + for (const emitted_target &ut : targets) { + const std::string target_sym =3D + selfpatch::target_symbol(ut.hash); + const std::string hash_sym =3D + selfpatch::target_hash_symbol(ut.hash); + const std::string layout_sym =3D + selfpatch::target_layout_symbol(ut.hash); + const std::string fields_sym =3D + selfpatch::make_local_label("target_fields"); + + const std::string target_name =3D selfpatch::emit_strtab_entry( + out, target::qualified_name(ut.target)); + + selfpatch::emit_targets_section(out, ut.hash); + selfpatch::emit_hidden_global_label(out, target_sym); + + /* This can only be done here, because the hash is the first + * component of the target metadata. */ + selfpatch::emit_hidden_global_label(out, hash_sym); + + selfpatch::target_desc target_desc{ + .hash =3D ut.hash, + .name_label =3D target_name, + .layout_symbol =3D layout_sym, + }; + + selfpatch::emit_target(out, target_desc); + selfpatch::emit_pop_section(out); + + selfpatch::emit_target_layouts_section(out, ut.hash); + selfpatch::emit_hidden_global_label(out, layout_sym); + + const std::vector &fields =3D + target::compressed_fields(ut.target); + + selfpatch::target_layout_desc layout_desc{ + .size =3D target::size(ut.target), + .field_cnt =3D fields.size(), + .fields_symbol =3D fields_sym, + }; + + selfpatch::emit_target_layout(out, layout_desc); + + selfpatch::emit_label(out, fields_sym); + + for (const target::compressed_field &field : fields) { + const std::string field_name =3D + selfpatch::emit_strtab_entry(out, field.name); + + std::size_t field_flags =3D field.fixed ? 1 : 0; + + selfpatch::target_field_desc field_desc{ + .name_label =3D field_name, + .size =3D field.size, + .offset =3D field.offset, + .alignment =3D field.alignment, + .flags =3D field_flags, + }; + + selfpatch::emit_target_field(out, field_desc); + } + + selfpatch::emit_pop_section(out); + } +} + +static void emit_unit_target_refs(FILE *out, + const std::vector &targets, + const std::string &target_refs_sym) +{ + selfpatch::emit_cu_target_refs_section(out); + selfpatch::emit_label(out, target_refs_sym); + + for (const emitted_target &target : targets) { + selfpatch::target_ref_desc ref{ + .target_symbol =3D selfpatch::target_symbol(target.hash), + }; + + selfpatch::emit_target_ref(out, ref); + } + + selfpatch::emit_pop_section(out); +} + +static void emit_ipins(FILE *out, + const std::unordered_map &target_idx, + const std::string &ipins_sym) +{ + std::map expr_syms; + + selfpatch::emit_ipins_section(out); + selfpatch::emit_label(out, ipins_sym); + + for (const auto &[h, pin] : ipin::inspect()) { + if (pin.status !=3D ipin::state::live) + continue; + + if (pin.symbol.empty() || pin.width_symbol.empty()) + pinpoint_fatal( + "finish-unit: live ipin is missing field symbols"); + + std::string expr_sym =3D selfpatch::make_local_label("ipin_expr"); + expr_syms.emplace(h, expr_sym); + + selfpatch::ipin_desc desc{ + .addr_expr =3D pin.symbol, + .size_expr =3D pin.width_symbol, + .expr_symbol =3D expr_sym, + }; + + selfpatch::emit_ipin(out, desc); + } + + for (const auto &[h, pin] : ipin::inspect()) { + if (pin.status !=3D ipin::state::live) + continue; + + tree pin_target =3D target::from_field(pin.field); + const auto target_it =3D target_idx.find(pin_target); + + if (target_it =3D=3D target_idx.end()) + pinpoint_fatal( + "finish-unit: ipin target not in unit target map"); + + auto expr_sym_it =3D expr_syms.find(h); + if (expr_sym_it =3D=3D expr_syms.end()) + pinpoint_fatal("finish-unit: lost ipin expr symbol"); + + selfpatch::emit_label(out, expr_sym_it->second); + + selfpatch::ipin_expr_desc expr{ + .unit_target_idx =3D target_it->second, + .field =3D target::field_index(pin.field), + }; + + selfpatch::emit_ipin_expr(out, expr); + } + + selfpatch::emit_pop_section(out); +} + +static void emit_dpins(FILE *out, const std::vector &dpins, + const std::string &dpins_sym) +{ + selfpatch::emit_dpins_section(out); + selfpatch::emit_label(out, dpins_sym); + + for (const emitted_dpin &pin : dpins) { + selfpatch::dpin_desc desc{ + .addr_expr =3D pin.addr_expr, + .unit_target_idx =3D pin.unit_target_idx, + }; + + selfpatch::emit_dpin(out, desc); + } + + selfpatch::emit_pop_section(out); +} + +} // namespace + +void on_finish_unit(void *plugin_data, void *user_data) +{ + FILE *out =3D asm_out_file; + + const std::vector targets =3D collect_all_targets(); + const auto target_idx =3D make_unit_target_idx_map(targets); + const std::vector dpins =3D collect_dpins(target_idx); + + const std::string source_label =3D + selfpatch::emit_strtab_entry(out, src_filename()); + + const std::string target_refs_sym =3D + selfpatch::make_local_label("target_refs"); + const std::string ipins_sym =3D selfpatch::make_local_label("ipins"); + const std::string dpins_sym =3D selfpatch::make_local_label("dpins"); + + selfpatch::emit_comment(out, "SPSLR runtime metadata"); + + emit_target_metadata(out, targets); + emit_unit_target_refs(out, targets, target_refs_sym); + emit_ipins(out, target_idx, ipins_sym); + emit_dpins(out, dpins, dpins_sym); + + selfpatch::emit_units_section(out); + + selfpatch::unit_desc unit{ + .source_label =3D source_label, + .target_ref_cnt =3D targets.size(), + .target_refs_symbol =3D target_refs_sym, + .ipin_cnt =3D ipin::live_count(), + .ipins_symbol =3D ipins_sym, + .dpin_cnt =3D dpins.size(), + .dpins_symbol =3D dpins_sym, + }; + + selfpatch::emit_unit(out, unit); + selfpatch::emit_pop_section(out); +} diff --git a/scripts/gcc-plugins/on_preserve_component_ref.c b/scripts/gcc-= plugins/on_preserve_component_ref.c new file mode 100644 index 000000000000..133269438914 --- /dev/null +++ b/scripts/gcc-plugins/on_preserve_component_ref.c @@ -0,0 +1,99 @@ +#include +#include +#include +#include +#include + +static tree materialize_c_rvalue(location_t loc, tree expr) +{ + gcc_assert(!lvalue_p(expr)); + + tree type =3D TREE_TYPE(expr); + + tree tmp =3D build_decl(loc, VAR_DECL, create_tmp_var_name("spslr_rval"), + type); + + DECL_CONTEXT(tmp) =3D current_function_decl; + DECL_ARTIFICIAL(tmp) =3D 1; + DECL_IGNORED_P(tmp) =3D 1; + TREE_USED(tmp) =3D 1; + TREE_ADDRESSABLE(tmp) =3D 1; + DECL_CHAIN(tmp) =3D NULL_TREE; + + // layout_decl(tmp, 0); - not available to plugins + + tree init =3D build2_loc(loc, INIT_EXPR, type, tmp, expr); + + tree body =3D build2(COMPOUND_EXPR, type, init, tmp); + + SET_EXPR_LOCATION(body, loc); + + tree bind =3D build3(BIND_EXPR, type, tmp, body, NULL_TREE); + + SET_EXPR_LOCATION(bind, loc); + TREE_SIDE_EFFECTS(bind) =3D 1; + + return bind; +} + +/* + * Preserve an early COMPONENT_REF before GCC folds it into a plain consta= nt + * offset. + * + * The custom GCC hook calls this while the frontend still knows that an + * expression is "base.field". We rewrite it into pointer arithmetic whose + * offset comes from a synthetic separator call: + * + * base.field -> *(typeof(field) *)((char *)&base + separator(uid)) + * + * Later passes replace the separator with an instruction pin. + */ + +static tree ast_separate_offset(tree ref, ipin::handle pin) +{ + tree separator =3D ipin::make_ast_separator(pin); + if (!separator) + pinpoint_fatal( + "ast_separate_offset failed to generate AST separator"); + + tree base =3D TREE_OPERAND(ref, 0); + + // ADDR_EXPR can never be a valid base, but such trees can happen during = parsing before checks + if (TREE_CODE(base) =3D=3D ADDR_EXPR) + pinpoint_fatal( + "ast_separate_offset encountered ADDR_EXPR as COMPONENT_REF base"); + + // Turn rvalue objects into adressable lvalues + if (!lvalue_p(base)) + base =3D materialize_c_rvalue(EXPR_LOCATION(base), base); + + tree base_ptr =3D build_fold_addr_expr(base); + + tree field_type =3D TREE_TYPE( + ref); // Type of COMPONENT_REF is type of the accessed field + tree field_ptr_type =3D build_pointer_type(field_type); + tree field_ptr =3D + build2(POINTER_PLUS_EXPR, field_ptr_type, base_ptr, separator); + + tree field_ref =3D build1(INDIRECT_REF, field_type, field_ptr); + return field_ref; +} + +void on_preserve_component_ref(void *plugin_data, void *user_data) +{ + tree *ref =3D (tree *)plugin_data; + if (!ref) + return; + + tree field; + if (!target::component_ref(*ref, &field)) + return; + + if (target::field_is_fixed(field)) + return; + + ipin::handle pin =3D ipin::make(field); + tree separated =3D ast_separate_offset(*ref, pin); + if (separated) + *ref =3D separated; +} diff --git a/scripts/gcc-plugins/on_register_attributes.c b/scripts/gcc-plu= gins/on_register_attributes.c new file mode 100644 index 000000000000..221693a1c465 --- /dev/null +++ b/scripts/gcc-plugins/on_register_attributes.c @@ -0,0 +1,52 @@ +#include +#include + +static tree check_spslr_attribute(tree *node, tree name, tree args, int fl= ags, + bool *no_add_attrs) +{ + if (!node || !*node || TREE_CODE(*node) !=3D RECORD_TYPE) { + *no_add_attrs =3D true; + pinpoint_debug(SPSLR_ATTRIBUTE + " attribute only applies to record types"); + } + + return NULL_TREE; +} + +static tree check_spslr_field_fixed_attribute(tree *node, tree name, tree = args, + int flags, bool *no_add_attrs) +{ + if (!node || !*node || TREE_CODE(*node) !=3D FIELD_DECL) { + *no_add_attrs =3D true; + pinpoint_debug(SPSLR_FIELD_FIXED_ATTRIBUTE + " attribute only applies to struct fields"); + } + return NULL_TREE; +} + +/* + * __attribute__((spslr)) marks a record type as a randomization target. + */ + +static struct attribute_spec spslr_attribute =3D { + SPSLR_ATTRIBUTE, 0, 0, false, false, false, false, + check_spslr_attribute, NULL +}; + +/* + * __attribute__((spslr_field_fixed)) marks a field as layout-sensitive. + * Fixed fields remain part of the target, but are treated as dangerous so + * instruction/data pins are not generated for offsets that would become + * ambiguous after randomization. + */ + +static struct attribute_spec spslr_fixed_field_attribute =3D { + SPSLR_FIELD_FIXED_ATTRIBUTE, 0, 0, false, false, false, false, + check_spslr_field_fixed_attribute, NULL +}; + +void on_register_attributes(void *plugin_data, void *user_data) +{ + register_attribute(&spslr_attribute); + register_attribute(&spslr_fixed_field_attribute); +} diff --git a/scripts/gcc-plugins/on_start_unit.c b/scripts/gcc-plugins/on_s= tart_unit.c new file mode 100644 index 000000000000..dc8917209de7 --- /dev/null +++ b/scripts/gcc-plugins/on_start_unit.c @@ -0,0 +1,11 @@ +#include +#include +#include +#include + +void on_start_unit(void *plugin_data, void *user_data) +{ + target::reset(); + dpin::reset(); + ipin::reset(); +} diff --git a/scripts/gcc-plugins/passes.h b/scripts/gcc-plugins/passes.h new file mode 100644 index 000000000000..f22949330e80 --- /dev/null +++ b/scripts/gcc-plugins/passes.h @@ -0,0 +1,31 @@ +#pragma once + +#include +#include + +void on_register_attributes(void *plugin_data, void *user_data); +void on_finish_type(void *plugin_data, void *user_data); +void on_preserve_component_ref(void *plugin_data, void *user_data); +void on_finish_decl(void *plugin_data, void *user_data); +void on_start_unit(void *plugin_data, void *user_data); +void on_finish_unit(void *plugin_data, void *user_data); + +struct separate_offset_pass : gimple_opt_pass { + separate_offset_pass(gcc::context *ctxt); + unsigned int execute(function *fn) override; +}; + +struct asm_offset_pass : gimple_opt_pass { + asm_offset_pass(gcc::context *ctxt); + unsigned int execute(function *fn) override; +}; + +struct rtl_ipin_survival_scan_pass : rtl_opt_pass { + rtl_ipin_survival_scan_pass(gcc::context *ctxt); + unsigned int execute(function *fn) override; +}; + +struct target_hash_builtin_pass : gimple_opt_pass { + target_hash_builtin_pass(gcc::context *ctxt); + unsigned int execute(function *fn) override; +}; diff --git a/scripts/gcc-plugins/pinpoint.c b/scripts/gcc-plugins/pinpoint.c new file mode 100644 index 000000000000..619487fdb53f --- /dev/null +++ b/scripts/gcc-plugins/pinpoint.c @@ -0,0 +1,103 @@ +#include +#include + +#include +#include +#include +#include + +int plugin_is_GPL_compatible; + +bool pinpoint_verbose_enabled; + +void pinpoint_gc_preserve(void *, void *) +{ + for (pinpoint_gc_preserve_fn *p =3D PINPOINT_GC_SECTION_START; + p !=3D PINPOINT_GC_SECTION_END; ++p) { + if (*p) + (*p)(); + } +} + +int plugin_init(struct plugin_name_args *plugin_info, + struct plugin_gcc_version *version) +{ + if (!plugin_default_version_check(version, &gcc_version)) { + plugin_print_early_error( + "incompatible GCC/plugin versions: plugin built for GCC %s, loaded by G= CC %s", + gcc_version.basever, version->basever); + return 1; + } + + pinpoint_verbose_enabled =3D false; + + for (int i =3D 0; i < plugin_info->argc; ++i) { + if (!strcmp(plugin_info->argv[i].key, "verbose")) + pinpoint_verbose_enabled =3D true; + } + + /* + * Pinpoint runs as a staged GCC plugin because no single GCC IR level has + * all information SPSLR needs. + * + * Stage 0 runs while COMPONENT_REF trees are built or still available an= d records + * which structure field offsets are randomization-sensitive. + * + * Stage 1 replaces synthetic separator calls with asm instructions whose= immediate + * operands are labeled for runtime patching. + * + * The final callback emits the collected metadata for patchcompile. + */ + + register_callback(plugin_info->base_name, PLUGIN_START_UNIT, + on_start_unit, NULL); + register_callback(plugin_info->base_name, PLUGIN_ATTRIBUTES, + on_register_attributes, NULL); + register_callback(plugin_info->base_name, PLUGIN_FINISH_TYPE, + on_finish_type, NULL); + register_callback(plugin_info->base_name, PLUGIN_BUILD_COMPONENT_REF, + on_preserve_component_ref, NULL); + register_callback(plugin_info->base_name, PLUGIN_FINISH_DECL, + on_finish_decl, NULL); + register_callback(plugin_info->base_name, PLUGIN_GGC_MARKING, + pinpoint_gc_preserve, NULL); + + struct register_pass_info target_hash_builtin_pass_info; + target_hash_builtin_pass_info.pass =3D + new target_hash_builtin_pass(nullptr); + target_hash_builtin_pass_info.ref_pass_instance_number =3D 1; + target_hash_builtin_pass_info.reference_pass_name =3D "cfg"; + target_hash_builtin_pass_info.pos_op =3D PASS_POS_INSERT_AFTER; + register_callback(plugin_info->base_name, PLUGIN_PASS_MANAGER_SETUP, + nullptr, &target_hash_builtin_pass_info); + + struct register_pass_info separate_offset_pass_info; + separate_offset_pass_info.pass =3D new separate_offset_pass(nullptr); + separate_offset_pass_info.ref_pass_instance_number =3D 1; + separate_offset_pass_info.reference_pass_name =3D "spslr_target_hash"; + separate_offset_pass_info.pos_op =3D PASS_POS_INSERT_AFTER; + register_callback(plugin_info->base_name, PLUGIN_PASS_MANAGER_SETUP, + nullptr, &separate_offset_pass_info); + + struct register_pass_info asm_offset_pass_info; + asm_offset_pass_info.pass =3D new asm_offset_pass(nullptr); + asm_offset_pass_info.ref_pass_instance_number =3D 1; + asm_offset_pass_info.reference_pass_name =3D "separate_offset"; + asm_offset_pass_info.pos_op =3D PASS_POS_INSERT_AFTER; + register_callback(plugin_info->base_name, PLUGIN_PASS_MANAGER_SETUP, + nullptr, &asm_offset_pass_info); + + struct register_pass_info rtl_ipin_survival_scan_pass_info; + rtl_ipin_survival_scan_pass_info.pass =3D + new rtl_ipin_survival_scan_pass(nullptr); + rtl_ipin_survival_scan_pass_info.ref_pass_instance_number =3D 1; + rtl_ipin_survival_scan_pass_info.reference_pass_name =3D "final"; + rtl_ipin_survival_scan_pass_info.pos_op =3D PASS_POS_INSERT_BEFORE; + register_callback(plugin_info->base_name, PLUGIN_PASS_MANAGER_SETUP, + nullptr, &rtl_ipin_survival_scan_pass_info); + + register_callback(plugin_info->base_name, PLUGIN_FINISH_UNIT, + on_finish_unit, NULL); + + return 0; +} diff --git a/scripts/gcc-plugins/pinpoint.h b/scripts/gcc-plugins/pinpoint.h new file mode 100644 index 000000000000..7f88cda32f48 --- /dev/null +++ b/scripts/gcc-plugins/pinpoint.h @@ -0,0 +1,75 @@ +#pragma once +#include +#include +#include + +#define SPSLR_ATTRIBUTE "spslr" +#define SPSLR_FIELD_FIXED_ATTRIBUTE "spslr_field_fixed" +#define SPSLR_TARGET_HASH_BUILTIN "__spslr_target_hash" + +extern bool pinpoint_verbose_enabled; + +#define plugin_print_early_error(fmt, ...) = \ + do { \ + std::fprintf(stderr, "[spslr::pinpoint] error: " fmt "\n", \ + ##__VA_ARGS__); \ + } while (0) + +#define pinpoint_debug_loc(loc, fmt, ...) \ + do { \ + if (pinpoint_verbose_enabled) \ + inform((loc), "[spslr::pinpoint] " fmt, \ + ##__VA_ARGS__); \ + } while (0) + +#define pinpoint_debug(fmt, ...) \ + pinpoint_debug_loc(UNKNOWN_LOCATION, fmt, ##__VA_ARGS__) + +#define pinpoint_fatal_loc(loc, fmt, ...) \ + fatal_error((loc), "[spslr::pinpoint] " fmt, ##__VA_ARGS__) + +#define pinpoint_fatal(fmt, ...) \ + pinpoint_fatal_loc(UNKNOWN_LOCATION, fmt, ##__VA_ARGS__) + +using pinpoint_gc_preserve_fn =3D void (*)(); + +#define PINPOINT_GC_CONCAT2(a, b) a##b +#define PINPOINT_GC_CONCAT(a, b) PINPOINT_GC_CONCAT2(a, b) + +#define PINPOINT_GC_USED __attribute__((used)) + +#define PINPOINT_GC_SECTION "pinpoint_gc_mark" +#define PINPOINT_GC_SECTION_START __start_pinpoint_gc_mark +#define PINPOINT_GC_SECTION_END __stop_pinpoint_gc_mark + +#define PINPOINT_GC_SECTION_ATTRIB __attribute__((section(PINPOINT_GC_SECT= ION))) + +#define PINPOINT_GC_PRESERVE_CALLBACK() \ + PINPOINT_GC_PRESERVE_CALLBACK_IMPL(__COUNTER__) + +#define PINPOINT_GC_PRESERVE_CALLBACK_IMPL(id) \ + static void PINPOINT_GC_CONCAT(pinpoint_gc_preserve_cb_, id)(); \ + static pinpoint_gc_preserve_fn PINPOINT_GC_CONCAT( \ + pinpoint_gc_preserve_reg_, id) \ + PINPOINT_GC_USED PINPOINT_GC_SECTION_ATTRIB =3D \ + PINPOINT_GC_CONCAT(pinpoint_gc_preserve_cb_, id); \ + static void PINPOINT_GC_CONCAT(pinpoint_gc_preserve_cb_, id)() + +#define PINPOINT_GC_MARK_TREE(t) \ + do { \ + if ((t) !=3D NULL_TREE) \ + ggc_mark(t); \ + } while (0) + +#define PINPOINT_GC_MARK(p) \ + do { \ + if (p) \ + ggc_mark(p); \ + } while (0) + +extern "C" { +extern pinpoint_gc_preserve_fn PINPOINT_GC_SECTION_START[]; +extern pinpoint_gc_preserve_fn PINPOINT_GC_SECTION_END[]; +} + +void pinpoint_gc_preserve(void *, void *); diff --git a/scripts/gcc-plugins/rtl_ipin_survival_scan_pass.c b/scripts/gc= c-plugins/rtl_ipin_survival_scan_pass.c new file mode 100644 index 000000000000..a30f709f6fb6 --- /dev/null +++ b/scripts/gcc-plugins/rtl_ipin_survival_scan_pass.c @@ -0,0 +1,37 @@ +#include +#include +#include +#include + +static const pass_data rtl_ipin_survival_scan_pass_data =3D { + RTL_PASS, + "spslr_rtl_ipin_survival_scan", + OPTGROUP_NONE, + TV_NONE, + PROP_rtl, + 0, + 0, + 0, + 0, +}; + +rtl_ipin_survival_scan_pass::rtl_ipin_survival_scan_pass(gcc::context *ctx= t) + : rtl_opt_pass(rtl_ipin_survival_scan_pass_data, ctxt) +{ +} + +unsigned int rtl_ipin_survival_scan_pass::execute(function *fn) +{ + (void)fn; + + for (rtx_insn *insn =3D get_insns(); insn; insn =3D NEXT_INSN(insn)) { + if (!NONDEBUG_INSN_P(insn)) + continue; + + ipin::handle pin =3D ipin::identify_rtl_pin(PATTERN(insn)); + if (pin !=3D ipin::invalid) + ipin::mark_live(pin, PATTERN(insn)); + } + + return 0; +} diff --git a/scripts/gcc-plugins/safe-attribs.h b/scripts/gcc-plugins/safe-= attribs.h new file mode 100644 index 000000000000..2d62fe75c72b --- /dev/null +++ b/scripts/gcc-plugins/safe-attribs.h @@ -0,0 +1,9 @@ +#include + +#ifndef SAFEGCC_ATTRIBS_H +#define SAFEGCC_ATTRIBS_H + +#include +#include + +#endif diff --git a/scripts/gcc-plugins/safe-diagnostic.h b/scripts/gcc-plugins/sa= fe-diagnostic.h new file mode 100644 index 000000000000..c58b739d88ed --- /dev/null +++ b/scripts/gcc-plugins/safe-diagnostic.h @@ -0,0 +1,10 @@ +#include + +#ifndef SAFEGCC_DIAGNOSTIC_H +#define SAFEGCC_DIAGNOSTIC_H + +#include +#include +#include + +#endif diff --git a/scripts/gcc-plugins/safe-gcc-plugin.h b/scripts/gcc-plugins/sa= fe-gcc-plugin.h new file mode 100644 index 000000000000..fcd111636a1c --- /dev/null +++ b/scripts/gcc-plugins/safe-gcc-plugin.h @@ -0,0 +1,6 @@ +#ifndef SAFEGCC_GCC_PLUGIN_H +#define SAFEGCC_GCC_PLUGIN_H + +#include + +#endif diff --git a/scripts/gcc-plugins/safe-ggc.h b/scripts/gcc-plugins/safe-ggc.h new file mode 100644 index 000000000000..5df62ad492e8 --- /dev/null +++ b/scripts/gcc-plugins/safe-ggc.h @@ -0,0 +1,8 @@ +#include + +#ifndef SAFEGCC_GGC_H +#define SAFEGCC_GGC_H + +#include + +#endif diff --git a/scripts/gcc-plugins/safe-gimple.h b/scripts/gcc-plugins/safe-g= imple.h new file mode 100644 index 000000000000..2eb0bae2f0a4 --- /dev/null +++ b/scripts/gcc-plugins/safe-gimple.h @@ -0,0 +1,14 @@ +#include +#include + +#ifndef SAFEGCC_GIMPLE_H +#define SAFEGCC_GIMPLE_H + +#include +#include +#include +#include +#include +#include + +#endif diff --git a/scripts/gcc-plugins/safe-input.h b/scripts/gcc-plugins/safe-in= put.h new file mode 100644 index 000000000000..fe24435830fe --- /dev/null +++ b/scripts/gcc-plugins/safe-input.h @@ -0,0 +1,9 @@ +#include + +#ifndef SAFEGCC_INPUT_H +#define SAFEGCC_INPUT_H + +#include +#include + +#endif diff --git a/scripts/gcc-plugins/safe-langhooks.h b/scripts/gcc-plugins/saf= e-langhooks.h new file mode 100644 index 000000000000..3fbea6ccb579 --- /dev/null +++ b/scripts/gcc-plugins/safe-langhooks.h @@ -0,0 +1,8 @@ +#include + +#ifndef SAFEGCC_LANGHOOKS_H +#define SAFEGCC_LANGHOOKS_H + +#include + +#endif diff --git a/scripts/gcc-plugins/safe-md5.h b/scripts/gcc-plugins/safe-md5.h new file mode 100644 index 000000000000..8341cb193467 --- /dev/null +++ b/scripts/gcc-plugins/safe-md5.h @@ -0,0 +1,8 @@ +#include + +#ifndef SAFEGCC_MD5_H +#define SAFEGCC_MD5_H + +#include + +#endif diff --git a/scripts/gcc-plugins/safe-output.h b/scripts/gcc-plugins/safe-o= utput.h new file mode 100644 index 000000000000..5da7fec494be --- /dev/null +++ b/scripts/gcc-plugins/safe-output.h @@ -0,0 +1,8 @@ +#include + +#ifndef SAFEGCC_OUTPUT_H +#define SAFEGCC_OUTPUT_H + +#include + +#endif diff --git a/scripts/gcc-plugins/safe-plugin-version.h b/scripts/gcc-plugin= s/safe-plugin-version.h new file mode 100644 index 000000000000..e43a1689da8c --- /dev/null +++ b/scripts/gcc-plugins/safe-plugin-version.h @@ -0,0 +1,8 @@ +#include + +#ifndef SAFEGCC_PLUGIN_VERSION_H +#define SAFEGCC_PLUGIN_VERSION_H + +#include + +#endif diff --git a/scripts/gcc-plugins/safe-rtl.h b/scripts/gcc-plugins/safe-rtl.h new file mode 100644 index 000000000000..f89decdb1d18 --- /dev/null +++ b/scripts/gcc-plugins/safe-rtl.h @@ -0,0 +1,17 @@ +#include + +#ifndef SAFEGCC_RTL_H +#define SAFEGCC_RTL_H + +#include +#include +#include +#include +#include +#include +#include + +#undef toupper +#undef tolower + +#endif diff --git a/scripts/gcc-plugins/safe-tree.h b/scripts/gcc-plugins/safe-tre= e.h new file mode 100644 index 000000000000..d968f8b9675a --- /dev/null +++ b/scripts/gcc-plugins/safe-tree.h @@ -0,0 +1,10 @@ +#include + +#ifndef SAFEGCC_TREE_H +#define SAFEGCC_TREE_H + +#include +#include +#include + +#endif diff --git a/scripts/gcc-plugins/separate_offset_pass.c b/scripts/gcc-plugi= ns/separate_offset_pass.c new file mode 100644 index 000000000000..9598f1148896 --- /dev/null +++ b/scripts/gcc-plugins/separate_offset_pass.c @@ -0,0 +1,327 @@ +#include +#include + +#include +#include +#include +#include + +/* + * AccessChain flattens nested COMPONENT_REF / ARRAY_REF expressions from + * outer base to inner access. + * + * This lets the pass rebuild the expression one step at a time while repl= acing + * only SPSLR-relevant field offsets with separator calls. Non-target offs= ets + * stay as ordinary constant pointer arithmetic. + */ + +struct AccessChain { + struct Step { + enum Kind { STEP_COMPONENT, STEP_ARRAY } kind =3D STEP_COMPONENT; + + tree t =3D NULL_TREE; + + /* COMPONENT_REF data */ + bool relevant =3D false; + tree field =3D NULL_TREE; + ipin::handle pin =3D ipin::invalid; + }; + + bool relevant =3D false; + std::list steps; + tree base =3D NULL_TREE; +}; + +static tree walk_tree_contains_component_ref(tree *tp, int *walk_subtrees, + void *data) +{ + int *found_flag =3D (int *)data; + + if (!tp || !*tp) + return NULL_TREE; + + if (TREE_CODE(*tp) =3D=3D COMPONENT_REF) + *found_flag =3D 1; + + return NULL_TREE; +} + +static bool tree_contains_component_ref(tree ref) +{ + int found_flag =3D 0; + walk_tree(&ref, walk_tree_contains_component_ref, &found_flag, NULL); + return found_flag !=3D 0; +} + +static bool access_chain(tree ref, AccessChain &chain) +{ + if (!ref) + return false; + + switch (TREE_CODE(ref)) { + case COMPONENT_REF: { + AccessChain::Step step; + step.kind =3D AccessChain::Step::STEP_COMPONENT; + step.t =3D ref; + tree field; + step.relevant =3D target::component_ref(ref, &field) && + !target::field_is_fixed(field); + if (step.relevant) { + step.field =3D field; + step.pin =3D ipin::make(field); + chain.relevant =3D true; + } + chain.steps.push_front(step); + return access_chain(TREE_OPERAND(ref, 0), chain); + } + + case ARRAY_REF: + case ARRAY_RANGE_REF: { + AccessChain::Step step; + step.kind =3D AccessChain::Step::STEP_ARRAY; + step.t =3D ref; + chain.steps.push_front(step); + return access_chain(TREE_OPERAND(ref, 0), chain); + } + + default: + /* + * Access chain construction needs to reach all COMPONENT_REFs. Further + * implementation may be required to cover all possible AST scenarios. + */ + if (tree_contains_component_ref(ref)) + return false; + + chain.base =3D ref; + return true; + } +} + +/* + * Rewrite a relevant field-access chain into explicit pointer arithmetic. + * + * The resulting MEM_REF has offset zero; all interesting byte offsets have + * either become separator calls or fixed constants. This makes the later = asm + * marker pass independent of GCC's original COMPONENT_REF tree shape. + */ + +static tree separate_offset_chain_maybe(tree ref, gimple_stmt_iterator *gs= i) +{ + AccessChain chain; + if (!access_chain(ref, chain)) { + pinpoint_fatal( + "separate_offset_chain_maybe encountered invalid access chain: top=3D%s= base=3D%s", + get_tree_code_name(TREE_CODE(ref)), + TREE_OPERAND(ref, 0) ? get_tree_code_name(TREE_CODE( + TREE_OPERAND(ref, 0))) : + ""); + } + + if (!chain.relevant) + return NULL_TREE; + + tree cur_expr =3D chain.base; + + // NOTE -> Could fold into single call here (needs to track what offsets = contribute, +irrelevant combined) + + for (const AccessChain::Step &step : chain.steps) { + if (step.kind =3D=3D AccessChain::Step::STEP_COMPONENT) { + if (TREE_CODE(cur_expr) =3D=3D ADDR_EXPR) + pinpoint_fatal( + "separate_offset_chain_maybe encountered ADDR_EXPR as base of COMPONE= NT_REF"); + + tree cur_ptr =3D build_fold_addr_expr(cur_expr); + + tree field_ptr_type =3D + build_pointer_type(TREE_TYPE(step.t)); + tree field_ptr; + + if (step.relevant) { + tree return_tmp =3D + create_tmp_var(size_type_node, NULL); + gimple *call_stmt =3D ipin::make_gimple_separator( + return_tmp, step.pin); + if (!call_stmt) + pinpoint_fatal( + "separate_offset_chain_maybe failed to make gimple separator"); + + gsi_insert_before(gsi, call_stmt, + GSI_SAME_STMT); + field_ptr =3D build2(POINTER_PLUS_EXPR, + field_ptr_type, cur_ptr, + return_tmp); + } else { + tree field_decl =3D TREE_OPERAND(step.t, 1); + + std::size_t field_offset =3D + target::field_offset(field_decl); + bool field_bitfield =3D + target::field_is_bitfield(field_decl); + if (field_bitfield) + pinpoint_fatal( + "separate_offset_chain_maybe encountered bitfield access in relevant= COMPONENT_REF chain"); + + field_ptr =3D build2(POINTER_PLUS_EXPR, + field_ptr_type, cur_ptr, + build_int_cst(sizetype, + field_offset)); + } + + tree ptr_tmp =3D create_tmp_var(field_ptr_type, NULL); + + tree ptr_val =3D force_gimple_operand_gsi( + gsi, field_ptr, + true, // require simple result + ptr_tmp, // target temp + true, // insert before current stmt + GSI_SAME_STMT); + + tree offset0 =3D fold_convert(TREE_TYPE(ptr_val), + build_int_cst(sizetype, 0)); + + cur_expr =3D build2(MEM_REF, TREE_TYPE(step.t), ptr_val, + offset0); + + continue; + } + + if (step.kind =3D=3D AccessChain::Step::STEP_ARRAY) { + tree idx =3D TREE_OPERAND(step.t, 1); + tree low =3D TREE_OPERAND(step.t, 2); + tree elts =3D TREE_OPERAND(step.t, 3); + + cur_expr =3D build4(TREE_CODE(step.t), TREE_TYPE(step.t), + cur_expr, idx, low, elts); + continue; + } + } + + return cur_expr; +} + +static void dispatch_separation_maybe(const std::list &path, + gimple_stmt_iterator *gsi, + unsigned &cancel_levels) +{ + if (path.empty() || !gsi) + return; + + tree ref =3D *path.back(); + if (!ref || TREE_CODE(ref) !=3D COMPONENT_REF) + return; + + cancel_levels =3D 1; + + tree instrumented_ref =3D separate_offset_chain_maybe(ref, gsi); + if (!instrumented_ref) + return; + + gimple_set_modified(gsi_stmt(*gsi), true); + *path.back() =3D instrumented_ref; + + // At this point, instrumented_ref is a MEM_REF node (off=3D0). A wrappin= g ADDR_EXPR cancels it out. + + if (path.size() < 2) + return; + + tree *parent =3D *(++path.rbegin()); + + if (TREE_CODE(*parent) =3D=3D ADDR_EXPR) { + // Note -> the base of the MEM_REF is expected to have the same type as = the ADDR_EXPR + *parent =3D TREE_OPERAND(instrumented_ref, 0); + cancel_levels++; + } +} + +static const pass_data separate_offset_pass_data =3D { + GIMPLE_PASS, "separate_offset", OPTGROUP_NONE, TV_NONE, 0, 0, 0, + 0, TODO_update_ssa +}; + +separate_offset_pass::separate_offset_pass(gcc::context *ctxt) + : gimple_opt_pass(separate_offset_pass_data, ctxt) +{ +} + +struct TreeWalkData { + std::list path; + gimple_stmt_iterator *gsi; + unsigned cancel_levels; + std::function &, gimple_stmt_iterator *, + unsigned &)> + callback; +}; + +static tree walk_tree_level(tree *tp, int *walk_subtrees, void *data) +{ + TreeWalkData *twd =3D (TreeWalkData *)data; + if (!twd) + return NULL_TREE; + + if (!twd->path.empty() && twd->path.back() =3D=3D tp) + return NULL_TREE; // root of this level + + if (walk_subtrees) + *walk_subtrees =3D 0; + + twd->cancel_levels =3D 0; + twd->path.push_back(tp); + + twd->callback(twd->path, twd->gsi, twd->cancel_levels); + + if (twd->cancel_levels =3D=3D 0) + walk_tree(tp, walk_tree_level, data, NULL); + + twd->path.pop_back(); + + if (twd->cancel_levels > 0) + twd->cancel_levels--; + + // Cancel current level if there are still cancel_levels due + return twd->cancel_levels =3D=3D 0 ? NULL_TREE : *tp; +} + +static bool +walk_gimple_stmt(gimple_stmt_iterator *gsi, + std::function &, + gimple_stmt_iterator *, unsigned &)> + callback) +{ + if (!gsi || gsi_end_p(*gsi) || !callback) + return false; + + gimple *stmt =3D gsi_stmt(*gsi); + + for (std::size_t i =3D 0; i < gimple_num_ops(stmt); i++) { + tree *op =3D gimple_op_ptr(stmt, i); + if (!op || !*op) + continue; + + TreeWalkData twd; + twd.gsi =3D gsi; + twd.callback =3D callback; + + walk_tree_level(op, NULL, &twd); + } + + return true; +} + +unsigned int separate_offset_pass::execute(function *fn) +{ + if (!fn) + return 0; + + basic_block bb; + FOR_EACH_BB_FN(bb, fn) + { + for (gimple_stmt_iterator gsi =3D gsi_start_bb(bb); + !gsi_end_p(gsi); gsi_next(&gsi)) { + if (!walk_gimple_stmt(&gsi, dispatch_separation_maybe)) + pinpoint_fatal( + "separate_offset pass failed to walk gimple statement"); + } + } + + return 0; +} diff --git a/scripts/gcc-plugins/serialize.c b/scripts/gcc-plugins/serializ= e.c new file mode 100644 index 000000000000..d30e49464aea --- /dev/null +++ b/scripts/gcc-plugins/serialize.c @@ -0,0 +1,303 @@ +#include "serialize.h" + +#include + +namespace selfpatch +{ + +namespace +{ + +constexpr std::string_view section_entry =3D "spslr_entry"; +constexpr std::string_view section_units =3D "spslr_units"; +constexpr std::string_view section_targets =3D "spslr_targets"; +constexpr std::string_view section_target_layouts =3D "spslr_target_layout= s"; +constexpr std::string_view section_ipins =3D "spslr_ipins"; +constexpr std::string_view section_dpins =3D "spslr_dpins"; +constexpr std::string_view section_strtab =3D "spslr_strtab"; +constexpr std::string_view section_cu_target_refs =3D "spslr_cu_target_ref= s"; + +std::size_t next_local_label_id =3D 0; + +std::string quote_asm_string(std::string_view s) +{ + std::string out; + out.reserve(s.size() + 8); + out.push_back('"'); + + for (unsigned char c : s) { + switch (c) { + case '\\': + out +=3D "\\\\"; + break; + case '"': + out +=3D "\\\""; + break; + case '\n': + out +=3D "\\n"; + break; + case '\r': + out +=3D "\\r"; + break; + case '\t': + out +=3D "\\t"; + break; + case '\0': + out +=3D "\\000"; + break; + default: + if (std::isprint(c)) { + out.push_back(static_cast(c)); + } else { + char buf[5]; + std::snprintf(buf, sizeof(buf), "\\%03o", c); + out +=3D buf; + } + break; + } + } + + out.push_back('"'); + return out; +} + +} // namespace + +void emit_comment(FILE *out, std::string_view text) +{ + std::fprintf(out, "\n/* %.*s */\n", static_cast(text.size()), + text.data()); +} + +void emit_push_section(FILE *out, std::string_view name) +{ + /* Future implementation should differentiate between module and host + * section permissions. Module relocations may cause DT_TEXTREL issues + * if the metadata sections are read-only. */ + std::fprintf(out, ".pushsection %.*s,\"aw\",@progbits\n", + static_cast(name.size()), name.data()); +} + +void emit_push_comdat_section(FILE *out, std::string_view name, + std::string_view group_symbol) +{ + /* Future implementation should differentiate between module and host + * section permissions. Module relocations may cause DT_TEXTREL issues + * if the metadata sections are read-only. */ + std::fprintf(out, ".pushsection %.*s,\"awG\",@progbits,%.*s,comdat\n", + static_cast(name.size()), name.data(), + static_cast(group_symbol.size()), + group_symbol.data()); +} + +void emit_pop_section(FILE *out) +{ + std::fprintf(out, ".popsection\n"); +} + +void emit_units_section(FILE *out) +{ + emit_push_section(out, section_units); +} + +void emit_targets_section(FILE *out, const hash16_t &hash) +{ + emit_push_comdat_section(out, section_targets, + comdat_target_symbol(hash)); +} + +void emit_target_layouts_section(FILE *out, const hash16_t &hash) +{ + emit_push_comdat_section(out, section_target_layouts, + comdat_target_symbol(hash)); +} + +void emit_ipins_section(FILE *out) +{ + emit_push_section(out, section_ipins); +} + +void emit_dpins_section(FILE *out) +{ + emit_push_section(out, section_dpins); +} + +void emit_strtab_section(FILE *out) +{ + emit_push_section(out, section_strtab); +} + +void emit_cu_target_refs_section(FILE *out) +{ + emit_push_section(out, section_cu_target_refs); +} + +void emit_label(FILE *out, std::string_view label) +{ + std::fprintf(out, "%.*s:\n", static_cast(label.size()), + label.data()); +} + +void emit_hidden_global_label(FILE *out, std::string_view label) +{ + std::fprintf(out, ".globl %.*s\n", static_cast(label.size()), + label.data()); + std::fprintf(out, ".hidden %.*s\n", static_cast(label.size()), + label.data()); + emit_label(out, label); +} + +std::string make_local_label(std::string_view stem) +{ + std::string out =3D ".Lspslr_"; + out.append(stem); + out.push_back('_'); + out.append(std::to_string(next_local_label_id++)); + return out; +} + +std::string emit_strtab_entry(FILE *out, std::string_view value) +{ + const std::string label =3D make_local_label("str"); + + emit_strtab_section(out); + emit_label(out, label); + emit_c_string(out, value); + emit_pop_section(out); + + return label; +} + +void emit_unit(FILE *out, const unit_desc &unit) +{ + emit_quad_symbol(out, unit.source_label); + emit_quad(out, unit.target_ref_cnt); + emit_quad_symbol(out, unit.target_refs_symbol); + emit_quad(out, unit.ipin_cnt); + emit_quad_symbol(out, unit.ipins_symbol); + emit_quad(out, unit.dpin_cnt); + emit_quad_symbol(out, unit.dpins_symbol); +} + +void emit_target(FILE *out, const target_desc &target) +{ + emit_bytes(out, target.hash.data(), target.hash.size()); + emit_quad_symbol(out, target.name_label); + emit_quad_symbol(out, target.layout_symbol); +} + +void emit_target_layout(FILE *out, const target_layout_desc &layout) +{ + emit_quad(out, layout.size); + emit_quad(out, layout.field_cnt); + emit_quad_symbol(out, layout.fields_symbol); +} + +void emit_target_field(FILE *out, const target_field_desc &field) +{ + emit_quad_symbol(out, field.name_label); + emit_quad(out, field.size); + emit_quad(out, field.offset); + emit_quad(out, field.alignment); + emit_quad(out, field.flags); +} + +void emit_target_ref(FILE *out, const target_ref_desc &target) +{ + emit_quad_symbol(out, target.target_symbol); +} + +void emit_ipin(FILE *out, const ipin_desc &ipin) +{ + emit_quad_expr(out, ipin.addr_expr); + emit_quad_expr(out, ipin.size_expr); + emit_quad_symbol(out, ipin.expr_symbol); +} + +void emit_dpin(FILE *out, const dpin_desc &dpin) +{ + emit_quad_expr(out, dpin.addr_expr); + emit_quad(out, dpin.unit_target_idx); +} + +void emit_ipin_expr(FILE *out, const ipin_expr_desc &expr) +{ + emit_quad(out, expr.unit_target_idx); + emit_quad(out, expr.field); +} + +void emit_quad(FILE *out, std::size_t value) +{ + std::fprintf(out, ".quad %zu\n", value); +} + +void emit_quad_symbol(FILE *out, std::string_view symbol) +{ + emit_quad_expr(out, symbol); +} + +void emit_quad_expr(FILE *out, std::string_view expr) +{ + std::fprintf(out, ".quad %.*s\n", static_cast(expr.size()), + expr.data()); +} + +void emit_bytes(FILE *out, const void *data, std::size_t size) +{ + const auto *bytes =3D static_cast(data); + + for (std::size_t i =3D 0; i < size; ++i) { + if (i % 16 =3D=3D 0) + std::fprintf(out, ".byte "); + else + std::fprintf(out, ","); + + std::fprintf(out, "0x%02x", bytes[i]); + + if (i % 16 =3D=3D 15 || i + 1 =3D=3D size) + std::fprintf(out, "\n"); + } +} + +void emit_c_string(FILE *out, std::string_view value) +{ + const std::string quoted =3D quote_asm_string(value); + std::fprintf(out, ".asciz %s\n", quoted.c_str()); +} + +std::string hash_hex(const hash16_t &hash) +{ + static constexpr char digits[] =3D "0123456789abcdef"; + + std::string out; + out.resize(hash.size() * 2); + + for (std::size_t i =3D 0; i < hash.size(); ++i) { + out[i * 2] =3D digits[(hash[i] >> 4) & 0x0f]; + out[i * 2 + 1] =3D digits[hash[i] & 0x0f]; + } + + return out; +} + +std::string comdat_target_symbol(const hash16_t &hash) +{ + return "__comdat_spslr_target_" + hash_hex(hash); +} + +std::string target_symbol(const hash16_t &hash) +{ + return "__spslr_target_" + hash_hex(hash); +} + +std::string target_hash_symbol(const hash16_t &hash) +{ + return "__spslr_target_hash_" + hash_hex(hash); +} + +std::string target_layout_symbol(const hash16_t &hash) +{ + return "__spslr_target_layout_" + hash_hex(hash); +} + +} // namespace selfpatch diff --git a/scripts/gcc-plugins/serialize.h b/scripts/gcc-plugins/serializ= e.h new file mode 100644 index 000000000000..7c358dd17645 --- /dev/null +++ b/scripts/gcc-plugins/serialize.h @@ -0,0 +1,115 @@ +#pragma once + +#include +#include +#include +#include +#include +#include + +namespace selfpatch +{ + +using hash16_t =3D std::array; + +/* + * Entry point is: + * __start_spslr_units + * __stop_spslr_units + * __start_spslr_targets + * __stop_spslr_targets + */ + +struct unit_desc { + std::string source_label; + std::size_t target_ref_cnt =3D 0; + std::string target_refs_symbol; + std::size_t ipin_cnt =3D 0; + std::string ipins_symbol; + std::size_t dpin_cnt =3D 0; + std::string dpins_symbol; +}; + +struct target_desc { + hash16_t hash{}; + std::string name_label; + std::string layout_symbol; +}; + +struct target_layout_desc { + std::size_t size =3D 0; + std::size_t field_cnt =3D 0; + std::string fields_symbol; +}; + +struct target_field_desc { + std::string name_label; + std::size_t size =3D 0; + std::size_t offset =3D 0; + std::size_t alignment =3D 0; + std::uint64_t flags =3D 0; +}; + +struct target_ref_desc { + std::string target_symbol; +}; + +struct ipin_desc { + std::string addr_expr; + std::string size_expr; + std::string expr_symbol; +}; + +struct dpin_desc { + std::string addr_expr; + std::size_t unit_target_idx =3D 0; +}; + +struct ipin_expr_desc { + std::size_t unit_target_idx =3D 0; + std::size_t field =3D 0; +}; + +void emit_comment(FILE *out, std::string_view text); + +void emit_push_section(FILE *out, std::string_view name); +void emit_push_comdat_section(FILE *out, std::string_view name, + std::string_view group_symbol); +void emit_pop_section(FILE *out); + +void emit_units_section(FILE *out); +void emit_targets_section(FILE *out, const hash16_t &hash); +void emit_target_layouts_section(FILE *out, const hash16_t &hash); +void emit_ipins_section(FILE *out); +void emit_dpins_section(FILE *out); +void emit_strtab_section(FILE *out); +void emit_cu_target_refs_section(FILE *out); + +void emit_label(FILE *out, std::string_view label); +void emit_hidden_global_label(FILE *out, std::string_view label); + +std::string make_local_label(std::string_view stem); +std::string emit_strtab_entry(FILE *out, std::string_view value); + +void emit_unit(FILE *out, const unit_desc &unit); +void emit_target(FILE *out, const target_desc &target); +void emit_target_layout(FILE *out, const target_layout_desc &layout); +void emit_target_field(FILE *out, const target_field_desc &field); +void emit_target_ref(FILE *out, const target_ref_desc &target); +void emit_ipin(FILE *out, const ipin_desc &ipin); +void emit_dpin(FILE *out, const dpin_desc &dpin); +void emit_ipin_expr(FILE *out, const ipin_expr_desc &expr); + +void emit_quad(FILE *out, std::size_t value); +void emit_quad_symbol(FILE *out, std::string_view symbol); +void emit_quad_expr(FILE *out, std::string_view expr); +void emit_bytes(FILE *out, const void *data, std::size_t size); +void emit_c_string(FILE *out, std::string_view value); + +std::string hash_hex(const hash16_t &hash); +std::string comdat_target_symbol(const hash16_t &hash); +std::string target_symbol(const hash16_t &hash); +std::string target_hash_symbol(const hash16_t &hash); +std::string target_layout_symbol(const hash16_t &hash); + +} // namespace selfpatch diff --git a/scripts/gcc-plugins/target_hash_builtin_pass.c b/scripts/gcc-p= lugins/target_hash_builtin_pass.c new file mode 100644 index 000000000000..d35b8df9abbc --- /dev/null +++ b/scripts/gcc-plugins/target_hash_builtin_pass.c @@ -0,0 +1,167 @@ +#include +#include +#include + +#include +#include +#include +#include + +#include +#include + +namespace +{ + +static std::map hash_symbol_decls; + +static selfpatch::hash16_t to_hash16(const target::layout_hash_t &in) +{ + selfpatch::hash16_t out{}; + + for (std::size_t i =3D 0; i < out.size(); ++i) + out[i] =3D static_cast(in[i]); + + return out; +} + +static bool called_decl_name_is(tree fndecl, const char *wanted) +{ + if (!fndecl || !DECL_NAME(fndecl)) + return false; + + const char *name =3D IDENTIFIER_POINTER(DECL_NAME(fndecl)); + return name && std::strcmp(name, wanted) =3D=3D 0; +} + +static tree call_argument_pointee_type(gimple *stmt) +{ + if (!stmt || !is_gimple_call(stmt) || gimple_call_num_args(stmt) !=3D 1) + return NULL_TREE; + + tree arg =3D gimple_call_arg(stmt, 0); + if (!arg) + return NULL_TREE; + + /* Case: &__spslr_target_hash_type_anchor_N */ + if (TREE_CODE(arg) =3D=3D ADDR_EXPR) { + tree obj =3D TREE_OPERAND(arg, 0); + if (obj) { + tree obj_type =3D TREE_TYPE(obj); + if (obj_type) + return target::main_variant(obj_type); + } + } + + /* Fallback: argument still has pointer type T *. */ + tree arg_type =3D TREE_TYPE(arg); + if (arg_type && POINTER_TYPE_P(arg_type)) + return target::main_variant(TREE_TYPE(arg_type)); + + return NULL_TREE; +} + +static tree make_hash_symbol_decl(const std::string &symbol) +{ + auto it =3D hash_symbol_decls.find(symbol); + if (it !=3D hash_symbol_decls.end()) + return it->second; + + tree byte_type =3D + build_qualified_type(unsigned_char_type_node, TYPE_QUAL_CONST); + tree array_type =3D build_array_type_nelts(byte_type, 16); + + tree decl =3D build_decl(UNKNOWN_LOCATION, VAR_DECL, + get_identifier(symbol.c_str()), array_type); + + DECL_EXTERNAL(decl) =3D 1; + TREE_PUBLIC(decl) =3D 1; + TREE_READONLY(decl) =3D 1; + DECL_ARTIFICIAL(decl) =3D 1; + DECL_IGNORED_P(decl) =3D 1; + + hash_symbol_decls.emplace(symbol, decl); + return decl; +} + +static tree make_hash_pointer_expr(tree target_type, tree result_type) +{ + const selfpatch::hash16_t hash =3D + to_hash16(target::layout_hash(target_type)); + + const std::string symbol =3D selfpatch::target_hash_symbol(hash); + + tree decl =3D make_hash_symbol_decl(symbol); + tree addr =3D build_fold_addr_expr(decl); + + return fold_convert(result_type, addr); +} + +static tree make_null_pointer_expr(tree result_type) +{ + return fold_convert(result_type, null_pointer_node); +} + +} // namespace + +PINPOINT_GC_PRESERVE_CALLBACK() +{ + for (const auto &[symbol, decl] : hash_symbol_decls) + PINPOINT_GC_MARK_TREE(decl); +} + +static const pass_data target_hash_builtin_pass_data =3D { + GIMPLE_PASS, "spslr_target_hash", OPTGROUP_NONE, TV_NONE, 0, 0, 0, + 0, TODO_update_ssa +}; + +target_hash_builtin_pass::target_hash_builtin_pass(gcc::context *ctxt) + : gimple_opt_pass(target_hash_builtin_pass_data, ctxt) +{ +} + +unsigned int target_hash_builtin_pass::execute(function *fn) +{ + if (!fn) + return 0; + + basic_block bb; + FOR_EACH_BB_FN(bb, fn) + { + for (gimple_stmt_iterator gsi =3D gsi_start_bb(bb); + !gsi_end_p(gsi);) { + gimple *stmt =3D gsi_stmt(gsi); + + if (!is_gimple_call(stmt) || + !called_decl_name_is(gimple_call_fndecl(stmt), + SPSLR_TARGET_HASH_BUILTIN)) { + gsi_next(&gsi); + continue; + } + + tree lhs =3D gimple_call_lhs(stmt); + if (!lhs) { + gsi_remove(&gsi, true); + continue; + } + + tree result_type =3D TREE_TYPE(lhs); + tree target_type =3D call_argument_pointee_type(stmt); + + tree rhs =3D NULL_TREE; + + if (target_type && + target::is_validated_target(target_type)) + rhs =3D make_hash_pointer_expr(target_type, + result_type); + else + rhs =3D make_null_pointer_expr(result_type); + + gimple *replacement =3D gimple_build_assign(lhs, rhs); + gsi_replace(&gsi, replacement, true); + gsi_next(&gsi); + } + } + + return 0; +} diff --git a/scripts/gcc-plugins/target_registry.c b/scripts/gcc-plugins/ta= rget_registry.c new file mode 100644 index 000000000000..a692520467bd --- /dev/null +++ b/scripts/gcc-plugins/target_registry.c @@ -0,0 +1,492 @@ +#include +#include +#include +#include + +#include +#include +#include + +#include +#include + +struct validated_target { + std::vector fields{}; + std::map field_indices{}; + target::layout_hash_t hash{}; +}; + +static std::map validated_targets; + +PINPOINT_GC_PRESERVE_CALLBACK() +{ + for (const auto &[t, info] : validated_targets) + PINPOINT_GC_MARK_TREE(t); +} + +using field_callback =3D std::function; + +static void iterate_fields(tree type, const field_callback &cb) +{ + type =3D target::main_variant(type); + + if (!type || !COMPLETE_TYPE_P(type)) + pinpoint_fatal("target::iterate_fields: incomplete target"); + + for (tree f =3D TYPE_FIELDS(type); f; f =3D DECL_CHAIN(f)) { + if (TREE_CODE(f) =3D=3D FIELD_DECL) + cb(f); + } +} + +static void build_compressed_fields(tree type, validated_target &vt) +{ + vt.fields.clear(); + + std::size_t compressed_idx =3D 0; + + iterate_fields(type, [&](tree field) { + std::string name =3D target::field_name(field); + std::size_t off =3D target::field_offset(field); + std::size_t sz =3D target::field_size(field); + std::size_t end =3D off + sz; + bool fixed =3D target::field_is_fixed(field); + + if (vt.fields.empty()) { + vt.fields.push_back({ + .name =3D name, + .offset =3D off, + .size =3D sz, + .alignment =3D target::field_alignment(field), + .fixed =3D fixed, + }); + vt.field_indices[field] =3D compressed_idx; + return; + } + + target::compressed_field &prev =3D vt.fields.back(); + std::size_t prev_end =3D prev.offset + prev.size; + + if (off < prev.offset) + pinpoint_fatal( + "target::build_compressed_fields: invalid field order in target \"%s\"= ", + target::qualified_name(type).c_str()); + + if (off >=3D prev_end) { + vt.fields.push_back({ + .name =3D name, + .offset =3D off, + .size =3D sz, + .alignment =3D target::field_alignment(field), + .fixed =3D fixed, + }); + vt.field_indices[field] =3D ++compressed_idx; + return; + } + + if (!prev.fixed || !fixed) { + pinpoint_fatal( + "target::build_compressed_fields: overlapping non-fixed field in targe= t \"%s\": \"%s\"", + target::qualified_name(type).c_str(), + target::field_name(field).c_str()); + } + + /* + * Fixed overlapping fields are represented as one immovable byte + * range in the runtime metadata. Alignment is irrelevant because + * the randomizer will never move this synthetic field. + */ + if (end > prev_end) + prev.size =3D end - prev.offset; + + prev.alignment =3D 1; + prev.fixed =3D true; + prev.name =3D prev.name + "+" + name; + + vt.field_indices[field] =3D compressed_idx; + }); +} + +static void remember_target(tree type) +{ + type =3D target::main_variant(type); + if (!type) + return; + + if (validated_targets.find(type) !=3D validated_targets.end()) + return; + + auto new_vt =3D validated_targets.emplace(type, validated_target{}); + if (!new_vt.second) + pinpoint_fatal( + "remember_target failed to log new validated target"); + + validated_target &vt =3D new_vt.first->second; + + /* Must build compressed fields before hash, because hash queries them */ + build_compressed_fields(type, vt); + vt.hash =3D compute_layout_hash(type); +} + +bool target::is_validated_target(tree type) +{ + type =3D main_variant(type); + return type && validated_targets.find(type) !=3D validated_targets.end(); +} + +const target::layout_hash_t &target::layout_hash(tree type) +{ + type =3D main_variant(type); + + auto it =3D validated_targets.find(type); + if (it =3D=3D validated_targets.end()) + pinpoint_fatal( + "target::layout_hash: type is not a validated SPSLR target"); + + return it->second.hash; +} + +tree target::main_variant(tree type) +{ + if (!type || TREE_CODE(type) !=3D RECORD_TYPE) + return NULL_TREE; + + return TYPE_MAIN_VARIANT(type); +} + +tree target::from_field(tree field_decl) +{ + if (!field_decl || TREE_CODE(field_decl) !=3D FIELD_DECL) + return NULL_TREE; + + return main_variant(DECL_CONTEXT(field_decl)); +} + +bool target::is_target(tree type) +{ + type =3D main_variant(type); + + if (!type || TREE_CODE(type) !=3D RECORD_TYPE) + return false; + + return lookup_attribute(SPSLR_ATTRIBUTE, TYPE_ATTRIBUTES(type)); +} + +std::string target::field_name(tree field_decl) +{ + if (!field_decl || TREE_CODE(field_decl) !=3D FIELD_DECL) + return ""; + + tree name =3D DECL_NAME(field_decl); + if (!name) + return ""; + + return IDENTIFIER_POINTER(name); +} + +std::string target::name(tree type) +{ + type =3D main_variant(type); + if (!type) + return ""; + + tree name_tree =3D TYPE_NAME(type); + if (!name_tree) + return ""; + + if (TREE_CODE(name_tree) =3D=3D TYPE_DECL && DECL_NAME(name_tree)) + return IDENTIFIER_POINTER(DECL_NAME(name_tree)); + + if (TREE_CODE(name_tree) =3D=3D IDENTIFIER_NODE) + return IDENTIFIER_POINTER(name_tree); + + return ""; +} + +static std::string decl_context_name(tree decl) +{ + if (!decl) + return ""; + + tree name =3D DECL_NAME(decl); + if (!name) + return ""; + + return IDENTIFIER_POINTER(name); +} + +std::vector target::context_chain(tree type) +{ + std::vector out; + + type =3D main_variant(type); + if (!type) + return out; + + tree type_name =3D TYPE_NAME(type); + tree ctx =3D NULL_TREE; + + if (type_name && TREE_CODE(type_name) =3D=3D TYPE_DECL) + ctx =3D DECL_CONTEXT(type_name); + + if (!ctx) + ctx =3D TYPE_CONTEXT(type); + + for (; ctx;) { + if (TREE_CODE(ctx) =3D=3D TRANSLATION_UNIT_DECL) + break; + + if (TREE_CODE(ctx) =3D=3D RECORD_TYPE) { + out.push_back(name(ctx)); + ctx =3D TYPE_CONTEXT(ctx); + continue; + } + + if (DECL_P(ctx)) { + out.push_back(decl_context_name(ctx)); + ctx =3D DECL_CONTEXT(ctx); + continue; + } + + if (TYPE_P(ctx)) { + out.push_back(name(ctx)); + ctx =3D TYPE_CONTEXT(ctx); + continue; + } + + break; + } + + std::reverse(out.begin(), out.end()); + return out; +} + +std::string target::qualified_name(tree type) +{ + std::string out; + + for (const std::string &ctx : context_chain(type)) { + if (!out.empty()) + out +=3D "::"; + out +=3D ctx; + } + + if (!out.empty()) + out +=3D "::"; + + out +=3D name(type); + return out; +} + +std::size_t target::size(tree type) +{ + type =3D main_variant(type); + + tree size_tree =3D type ? TYPE_SIZE(type) : NULL_TREE; + if (!size_tree || TREE_CODE(size_tree) !=3D INTEGER_CST) + pinpoint_fatal("target::size: non-constant target size"); + + HOST_WIDE_INT bits =3D tree_to_uhwi(size_tree); + if (bits < 0 || bits % BITS_PER_UNIT) + pinpoint_fatal("target::size: target size is not byte-aligned"); + + return static_cast(bits / BITS_PER_UNIT); +} + +std::size_t target::field_offset(tree field_decl) +{ + if (!field_decl || TREE_CODE(field_decl) !=3D FIELD_DECL) + pinpoint_fatal( + "target::field_offset can only be applied to FIELD_DECL trees"); + + tree field_byte_offset_tree =3D DECL_FIELD_OFFSET(field_decl); + tree field_bit_offset_tree =3D DECL_FIELD_BIT_OFFSET(field_decl); + + if (!field_byte_offset_tree || + TREE_CODE(field_byte_offset_tree) !=3D INTEGER_CST) + pinpoint_fatal( + "target::field_offset was unable to fetch byte offset"); + + if (!field_bit_offset_tree || + TREE_CODE(field_bit_offset_tree) !=3D INTEGER_CST) + pinpoint_fatal( + "target::field_offset was unable to fetch bit offset"); + + HOST_WIDE_INT byte_offset =3D tree_to_uhwi(field_byte_offset_tree); + HOST_WIDE_INT bit_offset =3D tree_to_uhwi(field_bit_offset_tree); + HOST_WIDE_INT bit_offset_bytes =3D bit_offset / BITS_PER_UNIT; + + return byte_offset + bit_offset_bytes; +} + +bool target::field_has_size(tree field_decl) +{ + if (!field_decl || TREE_CODE(field_decl) !=3D FIELD_DECL) + return false; + + tree bit_offset =3D DECL_FIELD_BIT_OFFSET(field_decl); + tree bit_size =3D DECL_SIZE(field_decl); + + return bit_offset && TREE_CODE(bit_offset) =3D=3D INTEGER_CST && bit_size= && + TREE_CODE(bit_size) =3D=3D INTEGER_CST; +} + +std::size_t target::field_size(tree field_decl) +{ + if (!field_decl || TREE_CODE(field_decl) !=3D FIELD_DECL) + pinpoint_fatal( + "target::field_size can only be applied to FIELD_DECL trees"); + + tree field_bit_offset_tree =3D DECL_FIELD_BIT_OFFSET(field_decl); + tree field_bit_size_tree =3D DECL_SIZE(field_decl); + + if (!field_bit_offset_tree || + TREE_CODE(field_bit_offset_tree) !=3D INTEGER_CST) + pinpoint_fatal( + "target::field_size was unable to fetch bit offset"); + + if (!field_bit_size_tree || + TREE_CODE(field_bit_size_tree) !=3D INTEGER_CST) + pinpoint_fatal( + "target::field_size was unable to fetch bit size"); + + HOST_WIDE_INT bit_offset =3D + tree_to_uhwi(field_bit_offset_tree) % BITS_PER_UNIT; + HOST_WIDE_INT bit_size =3D tree_to_uhwi(field_bit_size_tree) + bit_offset; + + HOST_WIDE_INT bit_overhang =3D bit_size % BITS_PER_UNIT; + if (bit_overhang !=3D 0) + bit_size +=3D (8 - bit_overhang); + + return static_cast(bit_size / BITS_PER_UNIT); +} + +std::size_t target::field_alignment(tree field_decl) +{ + if (!field_decl || TREE_CODE(field_decl) !=3D FIELD_DECL) + pinpoint_fatal( + "target::field_alignment can only be applied to FIELD_DECL trees"); + + HOST_WIDE_INT alignment_bits =3D DECL_ALIGN(field_decl); + if (alignment_bits <=3D 0 && TREE_TYPE(field_decl)) + alignment_bits =3D TYPE_ALIGN(TREE_TYPE(field_decl)); + if (alignment_bits <=3D 0) + alignment_bits =3D BITS_PER_UNIT; + + std::size_t alignment =3D static_cast( + (alignment_bits + BITS_PER_UNIT - 1) / BITS_PER_UNIT); + if (alignment =3D=3D 0) + alignment =3D 1; + + return alignment; +} + +bool target::field_is_bitfield(tree field_decl) +{ + if (!field_decl || TREE_CODE(field_decl) !=3D FIELD_DECL) + pinpoint_fatal( + "target::field_is_bitfield can only be applied to FIELD_DECL trees"); + + tree field_bit_offset_tree =3D DECL_FIELD_BIT_OFFSET(field_decl); + tree field_bit_size_tree =3D DECL_SIZE(field_decl); + + if (!field_bit_offset_tree || + TREE_CODE(field_bit_offset_tree) !=3D INTEGER_CST) + pinpoint_fatal( + "target::field_is_bitfield was unable to fetch bit offset"); + + if (!field_bit_size_tree || + TREE_CODE(field_bit_size_tree) !=3D INTEGER_CST) + pinpoint_fatal( + "target::field_is_bitfield was unable to fetch bit size"); + + HOST_WIDE_INT bit_offset =3D + tree_to_uhwi(field_bit_offset_tree) % BITS_PER_UNIT; + HOST_WIDE_INT bit_size =3D tree_to_uhwi(field_bit_size_tree); + + bool decl_bitfield =3D DECL_BIT_FIELD_TYPE(field_decl) !=3D NULL_TREE; + bool extra_bitfield =3D bit_size % 8 !=3D 0 || bit_offset !=3D 0; + + return decl_bitfield || extra_bitfield; +} + +bool target::field_is_fixed(tree field_decl) +{ + return field_is_bitfield(field_decl) || + lookup_attribute(SPSLR_FIELD_FIXED_ATTRIBUTE, + DECL_ATTRIBUTES(field_decl)); +} + +const std::vector & +target::compressed_fields(tree type) +{ + type =3D main_variant(type); + + auto it =3D validated_targets.find(type); + if (it =3D=3D validated_targets.end()) + pinpoint_fatal( + "target::compressed_fields: type is not a validated SPSLR target"); + + return it->second.fields; +} + +void target::iterate_targets(const target_callback &cb) +{ + for (const auto &[t, info] : validated_targets) + cb(t); +} + +std::size_t target::target_count() +{ + return validated_targets.size(); +} + +void target::validate(tree type) +{ + type =3D main_variant(type); + remember_target(type); +} + +bool target::component_ref(tree ref, tree *field_decl) +{ + if (!ref || TREE_CODE(ref) !=3D COMPONENT_REF) + return false; + + tree field =3D TREE_OPERAND(ref, 1); + if (!field || TREE_CODE(field) !=3D FIELD_DECL) + return false; + + tree type =3D from_field(field); + if (!is_target(type)) + return false; + + if (field_decl) + *field_decl =3D field; + + return true; +} + +std::size_t target::field_index(tree field_decl) +{ + tree type =3D from_field(field_decl); + if (!type) + pinpoint_fatal( + "target::field_index: field does not belong to a target"); + + auto vt =3D validated_targets.find(type); + if (vt =3D=3D validated_targets.end()) + pinpoint_fatal( + "target::field_index: field does not belong to a validated target"); + + auto it =3D vt->second.field_indices.find(field_decl); + if (it =3D=3D vt->second.field_indices.end()) + pinpoint_fatal( + "target::field_index: field does not belong to a validated target"); + + return it->second; +} + +void target::reset() +{ + validated_targets.clear(); +} diff --git a/scripts/gcc-plugins/target_registry.h b/scripts/gcc-plugins/ta= rget_registry.h new file mode 100644 index 000000000000..5531cefffb38 --- /dev/null +++ b/scripts/gcc-plugins/target_registry.h @@ -0,0 +1,59 @@ +#pragma once + +#include +#include +#include +#include +#include + +#include + +struct target { + struct compressed_field { + std::string name{}; + std::size_t offset{}; + std::size_t size{}; + std::size_t alignment{}; + bool fixed{}; + }; + + static tree main_variant(tree type); + static tree from_field(tree field_decl); + + static bool is_target(tree type); + + static std::string name(tree type); + static std::vector context_chain(tree type); + static std::string qualified_name(tree type); + + static std::size_t size(tree type); + + static std::string field_name(tree field_decl); + static std::size_t field_offset(tree field_decl); + static bool field_has_size(tree field_decl); + static std::size_t field_size(tree field_decl); + static std::size_t field_alignment(tree field_decl); + static bool field_is_bitfield(tree field_decl); + static bool field_is_fixed(tree field_decl); + + static bool component_ref(tree ref, tree *field_decl); + + static const std::vector & + compressed_fields(tree type); + + using target_callback =3D std::function; + static void iterate_targets(const target_callback &cb); + static std::size_t target_count(); + + static void validate(tree type); + + /* THe field index is into compressed_fields */ + static std::size_t field_index(tree field_decl); + + using layout_hash_t =3D std::array; + + static bool is_validated_target(tree type); + static const layout_hash_t &layout_hash(tree type); + + static void reset(); +}; --=20 2.43.0 From nobody Sat Jul 25 01:54:02 2026 Received: from mail-wm1-f41.google.com (mail-wm1-f41.google.com [209.85.128.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 83C7B3939C1 for ; Mon, 20 Jul 2026 19:13:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784574785; cv=none; b=LG6dP+z42WT5V5BxJjMQxr3/NEoNYj864fZFe/AlimoERtQLkKNiYpaeDMBT7CrTPT1Nk9OU5gXf5F4wLO5mTd/BEY7hJ4U92fGwrd9FJVkV/7KjyLN0A17D1wZGVx2UNNOId6joMYUQTf2roIO0azMUNgDXA8cBFq3jxOlp1kg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784574785; c=relaxed/simple; bh=ddg1TiBzfmxYokYD9lx4TN1R+ypHVJGwLdK+JynGg9k=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IwhtadGP8hvIDlkl3b8U1JPX1cpPZHGEhr7Vgxsc6//MAJ9EnhdHLSYCHdxNDU4hqWARQeKmmzYFfkCpeExSLktx49Tt8xSf+2MliqdpLIOkrmeyhmHYyzXaGLh/nX5scfgTaRKzUUN65E4NMZyOuEtaKHq+wJjVpMBBxBVHCPk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=yjn-systems.com; spf=none smtp.mailfrom=yjn-systems.com; dkim=pass (2048-bit key) header.d=yjn-systems.com header.i=@yjn-systems.com header.b=ub93uWb4; arc=none smtp.client-ip=209.85.128.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=yjn-systems.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=yjn-systems.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=yjn-systems.com header.i=@yjn-systems.com header.b="ub93uWb4" Received: by mail-wm1-f41.google.com with SMTP id 5b1f17b1804b1-4954a2e73a9so21087935e9.3 for ; Mon, 20 Jul 2026 12:13:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yjn-systems.com; s=google; t=1784574780; x=1785179580; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=J7L2KY4SdHJXgXkR49tG1QlCrSHG7Ro8bzv6kAARvpU=; b=ub93uWb4kcyJJ9qow+Ik4pZGYkn1Wq+oRjjXudl6gUXM2azkDoACUuXvgV6E+pE+4F OE8mWvU0xjVB2xAoxv6J9IEc9CeTbAtqW5Pm0gzj+AXpM1Oe/7XU2OJIqZkttTWdqaoC KHWh7a/JH8SnOo6eBlLvwZg5CjQT2f5itomis/WjztJOvK1w2qYjGJf5tESr/X2hLFoA BhJxa/5QYSWF7KaeX3gGs+cIhknm79buWIlkrsFvX2PHluDv6gGDy9l3qI1C/krLfW1i vGvmYQ+DFkKSQMbYk2i7GYexDNubBILtAhaQlv2zLbPA4L/UK4MxTt03X95BX/qSTXo8 PMCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784574780; x=1785179580; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=J7L2KY4SdHJXgXkR49tG1QlCrSHG7Ro8bzv6kAARvpU=; b=TynjKFZ9tzcuDCsu6g58pP5r7eu8/wI3uVKgN4s4BmyuqpViZqeaGDgkRUm9s9oyPH W6E0WZ8anLxs7LyYIAGoGQ+eNMCdcfLv2A3DXfLPYZW9VWglxwhdVqY2w/mgr5Q3oxpS aRimdYcJMgMytgw8kPks+x5yod+p02c9WGxoSHyxtnZw4kYcUUYtOYsz5oVxxgv0Fr8I F9F8clLwP1zv6IZHiQPt13U1JBW8eaSFa1i7KA3d+nyaikNv8ebB91K+Nq5E8N94aBYI Fsf1WeBxYINFn7BCLpVzls6ud1H+nrwgVUy+hj5EagRCEM3MnhhQrflQU5efx+YApA4t qzVw== X-Gm-Message-State: AOJu0Yy5xIjwlFlYefhwrJUx+jpiRCumWfA4mVM50X5FFgFE92EgBECD +FGQT+NBK/Yw4UylgCxdPfB40QQ3aeqKBC+Y2Elcue4bKMrSprASIVk2YolVljDVdtK1uccdnkt OapLYOkJD X-Gm-Gg: AfdE7ckKyhIlb4RgpnK9SkMOzcpZZUps5eWTaHsqxxpdibVgJ8syFXUf0uOQhQCof7g fUWELHeIeFfWyQW3YeHtejUwZwTv8m7iaVedlrvDOKcZ+MadSV9Gza+dVM3f4mGiHoLlJ+idw0K KmfEHM9vV6uzM7OhNR4LX3T9kLPvi/89gWACUI4c2rDjkip1HC+2kFaUbvMCk0msd5xTVW9kXhX hlNWme85y02S7aSzVJ8vZCCEUZisuClF3GqLPJ7pjhGicjJHWX0tBVslZ3VzeAVwA9W/Dvl4LXE E41/xItmwPaqXNuXrpa+3TTgrLBOyKlgTrcfg2DmH7VhvV/g5uJ/r9pm81iaVjiZBFdglQmAh5A untJhmX1Iq0hs1SKr7YhGHt4ogvCZ1a6GhJirLFnks7ZBAdCqucL0b91wIHsZGelCL4J/YE6ZaJ eJbKOeIS0Xf7TmvXCEJeLw+XcwnRUDTt0+KMxPYSDeZulBJgcliH6DQHHx6/JlI36Ol4gmt0bL9 +r+ofZ5QYP7da093CY= X-Received: by 2002:a05:600c:1993:b0:495:64c6:84e9 with SMTP id 5b1f17b1804b1-4956533caf4mr8178815e9.0.1784574779489; Mon, 20 Jul 2026 12:12:59 -0700 (PDT) Received: from yjn-Zenbook-UX3404VA-UX3404VA.. (p200300dcbf448c00c9c364f8ed993120.dip0.t-ipconnect.de. [2003:dc:bf44:8c00:c9c3:64f8:ed99:3120]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-495653d38dbsm9231695e9.15.2026.07.20.12.12.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Jul 2026 12:12:59 -0700 (PDT) From: York Jasper Niebuhr To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kees@kernel.org, franzen@sec.in.tum.de, ardb@kernel.org Subject: [RFC v3 2/5] Selfpatch runtime Date: Mon, 20 Jul 2026 21:12:58 +0200 Message-ID: <20260720191258.21602-1-yjn@yjn-systems.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260720191146.21473-1-yjn@yjn-systems.com> References: <20260720191146.21473-1-yjn@yjn-systems.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Signed-off-by: York Jasper Niebuhr --- include/linux/spslr.h | 82 ++++ include/sanemaker/traps.h | 135 +++++++ kernel/Makefile | 2 + kernel/spslr/Makefile | 7 + kernel/spslr/pinpoint.h | 76 ++++ kernel/spslr/sanemaker_traps.c | 117 ++++++ kernel/spslr/spslr.c | 555 +++++++++++++++++++++++++++ kernel/spslr/spslr_env.c | 74 ++++ kernel/spslr/spslr_env.h | 45 +++ kernel/spslr/spslr_randomizer.c | 646 ++++++++++++++++++++++++++++++++ kernel/spslr/spslr_randomizer.h | 29 ++ 11 files changed, 1768 insertions(+) create mode 100644 include/linux/spslr.h create mode 100644 include/sanemaker/traps.h create mode 100644 kernel/spslr/Makefile create mode 100644 kernel/spslr/pinpoint.h create mode 100644 kernel/spslr/sanemaker_traps.c create mode 100644 kernel/spslr/spslr.c create mode 100644 kernel/spslr/spslr_env.c create mode 100644 kernel/spslr/spslr_env.h create mode 100644 kernel/spslr/spslr_randomizer.c create mode 100644 kernel/spslr/spslr_randomizer.h diff --git a/include/linux/spslr.h b/include/linux/spslr.h new file mode 100644 index 000000000000..2606d66c39f0 --- /dev/null +++ b/include/linux/spslr.h @@ -0,0 +1,82 @@ +#ifndef SPSLR_SELFPATCH_H +#define SPSLR_SELFPATCH_H + +#ifdef CONFIG_SPSLR + +#include + +#define SPSLR_START_UNITS_SYM __start_spslr_units +#define SPSLR_STOP_UNITS_SYM __stop_spslr_units +#define SPSLR_START_TARGETS_SYM __start_spslr_targets +#define SPSLR_STOP_TARGETS_SYM __stop_spslr_targets + +extern bool spslr_enabled; + +enum spslr_viability { SPSLR_VIABLE, SPSLR_NONVIABLE }; + +enum spslr_error { + SPSLR_OK, + SPSLR_ERROR_INCOMPLETE_CTX, + SPSLR_ERROR_INCOMPATIBLE_CTX, + SPSLR_ERROR_RANDOMIZER_INIT, + SPSLR_ERROR_INITIAL_TARGET_LAYOUT, + SPSLR_ERROR_RANDOMIZED_TARGET_LAYOUT, + SPSLR_ERROR_RANDOMIZE, + SPSLR_ERROR_MEMORY, + SPSLR_ERROR_UNINITIALIZED, + SPSLR_ERROR_ALREADY_PATCHED, + SPSLR_ERROR_PATCH_DPINS, + SPSLR_ERROR_PATCH_IPINS, + SPSLR_ERROR_MAP_TARGETS +}; + +struct spslr_status { + enum spslr_viability viability; + enum spslr_error error; +}; + +struct spslr_entry { + const void *start_units; // Address of SPSLR_START_UNITS_SYM + const void *stop_units; // Address of SPSLR_STOP_UNITS_SYM + const void *start_targets; // Address of SPSLR_START_TARGETS_SYM + const void *stop_targets; // Address of SPSLR_STOP_TARGETS_SYM +}; + +struct spslr_ctx { + struct spslr_entry entry; + void *workspace; // Temporary buffer of spslr_workspace_size(&entry) bytes +}; + +/* + * Runtime entry points are intentionally split: + * + * spslr_init() creates randomized layouts. + * spslr_selfpatch() patches the main executable. + * spslr_patch_module() patches with module-local metadata using host layo= uts. + */ + +struct spslr_status spslr_init(void); +struct spslr_status spslr_selfpatch(void); +unsigned long spslr_workspace_size(const struct spslr_entry *entry); +struct spslr_status spslr_patch_module(const struct spslr_ctx *m); + +/* Use spslr_target_hash(type) to get a pointer to the 16 byte md5 hash + of the target type. If type is not an SPSLR target, NULL is returned. */ + +extern const unsigned char *__spslr_target_hash(const void *); + +#define __SPSLR_CAT2(a, b) a##b +#define __SPSLR_CAT(a, b) __SPSLR_CAT2(a, b) + +#define __spslr_target_hash_impl(T, n) = \ + ({ \ + extern T __SPSLR_CAT(__spslr_target_hash_type_anchor_, n); \ + __spslr_target_hash( \ + &__SPSLR_CAT(__spslr_target_hash_type_anchor_, n)); \ + }) + +#define spslr_target_hash(T) __spslr_target_hash_impl(T, __COUNTER__) + +#endif /* CONFIG_SPSLR */ + +#endif diff --git a/include/sanemaker/traps.h b/include/sanemaker/traps.h new file mode 100644 index 000000000000..bcb75198b18b --- /dev/null +++ b/include/sanemaker/traps.h @@ -0,0 +1,135 @@ +#ifndef SANEMAKER_TRAPS_H +#define SANEMAKER_TRAPS_H + +/* Define trap API attributes */ + +#define SANEMAKER_TRAP_API extern __attribute__((__visibility__("default")= )) + +/* Use sanemaker_target_tag(&obj) to make sanemaker watch memops to that o= bject */ + +#ifdef CONFIG_SANEMAKER + +SANEMAKER_TRAP_API +void __sanemaker_target_tag_trap(const void *ptr, const unsigned char *tar= get); + +#define sanemaker_target_tag(ptr, type) \ + __sanemaker_target_tag_trap(ptr, spslr_target_hash(type)) + +#else + +#define sanemaker_target_tag(ptr, type) + +#endif + +/* Use sanemaker_target_untag(&obj) to make sanemaker stop watching memops= to that object */ + +#ifdef CONFIG_SANEMAKER + +SANEMAKER_TRAP_API +void __sanemaker_target_untag_trap(const void *ptr); + +#define sanemaker_target_untag(ptr) __sanemaker_target_untag_trap(ptr) + +#else + +#define sanemaker_target_untag(ptr) + +#endif + +/* The sanemaker_finish_layout(&fieldarr, &target_hash) should be called + by spslr selfpatch when a target layout has been randomized */ + +#ifdef CONFIG_SANEMAKER + +SANEMAKER_TRAP_API +void __sanemaker_finish_layout_trap(const void *fields, + const unsigned char *target); + +#define sanemaker_finish_layout(fields, target) \ + __sanemaker_finish_layout_trap(fields, target) + +#else + +#define sanemaker_finish_layout(fields, target) + +#endif + +/* Use sanemaker_fetch(what, default) to let sanemaker make decisions at r= untime */ + +typedef enum { + SANEMAKER_FETCH_SPSLR_ENABLED =3D 1, +} sanemaker_fetch_t; + +#ifdef CONFIG_SANEMAKER + +SANEMAKER_TRAP_API +int __sanemaker_fetch_trap(sanemaker_fetch_t what, int def); + +#define sanemaker_fetch(what, def) __sanemaker_fetch_trap(what, def) + +#else + +#define sanemaker_fetch(what, def) (def) + +#endif + +/* Use sanemaker_signal(signal) to control sanemaker behavior */ + +typedef enum { + SANEMAKER_SIGNAL_PATCH_BOUNDARY =3D 1, /* the image has been patched */ + SANEMAKER_SIGNAL_PAUSE =3D 2, + SANEMAKER_SIGNAL_RESUME =3D 3, +} sanemaker_signal_t; + +#ifdef CONFIG_SANEMAKER + +SANEMAKER_TRAP_API +void __sanemaker_signal_trap(sanemaker_signal_t signal); + +#define sanemaker_signal(signal) __sanemaker_signal_trap(signal) + +#else + +#define sanemaker_signal(signal) + +#endif + +/* Use sanemaker_new_image(name, ptr) and sanemaker_new_image_text(image, = begin, end) + to allow normalization of program counters inside dynamically loaded te= xt segments */ + +#ifdef CONFIG_SANEMAKER + +SANEMAKER_TRAP_API +void __sanemaker_new_image_trap(const char *name, const void *base); + +SANEMAKER_TRAP_API +void __sanemaker_new_image_text_trap(const char *image, const void *begin, + const void *end); + +SANEMAKER_TRAP_API +void __sanemaker_drop_image_trap(const char *image); + +SANEMAKER_TRAP_API +void __sanemaker_drop_image_text_trap(const char *image, const void *begin, + const void *end); + +#define sanemaker_new_image(name, base) __sanemaker_new_image_trap(name, b= ase) + +#define sanemaker_new_image_text(image, begin, end) \ + __sanemaker_new_image_text_trap(image, begin, end) + +#define sanemaker_drop_image(image) __sanemaker_drop_image_trap(image) + +#define sanemaker_drop_image_text(image, begin, end) \ + __sanemaker_drop_image_text_trap(image, begin, end) + +#else + +#define sanemaker_new_image(name, base) +#define sanemaker_new_image_text(image, begin, end) +#define sanemaker_drop_image(image) +#define sanemaker_drop_image_text(image, begin, end) + +#endif + +#endif diff --git a/kernel/Makefile b/kernel/Makefile index 1e1a31673577..cddfbff9f9f0 100644 --- a/kernel/Makefile +++ b/kernel/Makefile @@ -142,6 +142,8 @@ obj-$(CONFIG_WATCH_QUEUE) +=3D watch_queue.o obj-$(CONFIG_RESOURCE_KUNIT_TEST) +=3D resource_kunit.o obj-$(CONFIG_SYSCTL_KUNIT_TEST) +=3D sysctl-test.o =20 +obj-$(CONFIG_SPSLR) +=3D spslr/ + CFLAGS_kstack_erase.o +=3D $(DISABLE_KSTACK_ERASE) CFLAGS_kstack_erase.o +=3D $(call cc-option,-mgeneral-regs-only) obj-$(CONFIG_KSTACK_ERASE) +=3D kstack_erase.o diff --git a/kernel/spslr/Makefile b/kernel/spslr/Makefile new file mode 100644 index 000000000000..a736d0aab05c --- /dev/null +++ b/kernel/spslr/Makefile @@ -0,0 +1,7 @@ +obj-$(CONFIG_SPSLR) +=3D spslr.o spslr_env.o spslr_randomizer.o +obj-$(CONFIG_SANEMAKER) +=3D sanemaker_traps.o + +CFLAGS_REMOVE_spslr.o +=3D $(PINPOINT_PLUGIN_CFLAGS) +CFLAGS_REMOVE_spslr_env.o +=3D $(PINPOINT_PLUGIN_CFLAGS) +CFLAGS_REMOVE_spslr_randomizer.o +=3D $(PINPOINT_PLUGIN_CFLAGS) +CFLAGS_REMOVE_sanemaker_traps.o +=3D $(PINPOINT_PLUGIN_CFLAGS) diff --git a/kernel/spslr/pinpoint.h b/kernel/spslr/pinpoint.h new file mode 100644 index 000000000000..39ca81e6491d --- /dev/null +++ b/kernel/spslr/pinpoint.h @@ -0,0 +1,76 @@ +#ifndef SPSLR_PINPOINT_H +#define SPSLR_PINPOINT_H + +#include "spslr_env.h" + +/* Field must remain at its original offset during layout randomization. */ +#define SPSLR_FLAG_FIELD_FIXED 1 + +struct spslr_unit; +struct spslr_ipin; +struct spslr_ipin_expr; +struct spslr_dpin; +struct spslr_target; +struct spslr_target_layout; +struct spslr_target_field; + +/* CU-local target reference; points into the global deduplicated target t= able. */ +typedef const struct spslr_target *spslr_target_ref; + +/* + * Metadata for one compilation unit. The target array is CU-local and maps + * unit_target_idx values used by pins to deduplicated global target heade= rs. + */ +struct spslr_unit { + const char *source; // Source file name + spslr_u64 target_cnt; + const spslr_target_ref *target_refs; // CU-local target ref array + spslr_u64 ipin_cnt; + const struct spslr_ipin *ipins; + spslr_u64 dpin_cnt; + const struct spslr_dpin *dpins; +} __packed; + +/* Instruction patch site: address of patchable immediate/displacement byt= es. */ +struct spslr_ipin { + void *addr; + spslr_u64 size; + const struct spslr_ipin_expr *expr; +} __packed; + +/* Data patch site: address of an object/subobject whose layout must be ad= justed. */ +struct spslr_dpin { + void *addr; + spslr_u64 unit_target_idx; +} __packed; + +/* Current simple expression: randomized offset of one field in one CU-loc= al target. */ +struct spslr_ipin_expr { + spslr_u64 unit_target_idx; + spslr_u64 field_idx; +} __packed; + +/* Deduplicated target type descriptor, keyed by deterministic layout hash= . */ +struct spslr_target { + unsigned char hash[16]; + const char *name; + const struct spslr_target_layout *layout; +} __packed; + +/* Physical layout of a target type before runtime randomization. */ +struct spslr_target_layout { + spslr_u64 size; + spslr_u64 field_cnt; + const struct spslr_target_field *fields; +} __packed; + +/* One randomizable or fixed field/range within a target layout. */ +struct spslr_target_field { + const char *name; + spslr_u64 size; + spslr_u64 offset; + spslr_u64 alignment; + spslr_u64 flags; +} __packed; + +#endif diff --git a/kernel/spslr/sanemaker_traps.c b/kernel/spslr/sanemaker_traps.c new file mode 100644 index 000000000000..63aa1a4b9182 --- /dev/null +++ b/kernel/spslr/sanemaker_traps.c @@ -0,0 +1,117 @@ +#include + +#define SANEMAKER_TRAP_FN \ + __attribute__((__noinline__, __noclone__, __used__, \ + __externally_visible__, \ + __visibility__("default"), __naked__)) + +/* Sanemaker reads object pointer from rdi and target hash pointer from rs= i */ +SANEMAKER_TRAP_FN +void __sanemaker_target_tag_trap(const void *ptr, const unsigned char *tar= get) +{ + __asm__ volatile( + ".globl __sanemaker_target_tag_trap_incision\n" + ".type __sanemaker_target_tag_trap_incision, @notype\n" + "__sanemaker_target_tag_trap_incision:\n" + "ret\n" + ); +} + +/* Sanemaker reads object pointer from rdi */ +SANEMAKER_TRAP_FN +void __sanemaker_target_untag_trap(const void *ptr) +{ + __asm__ volatile( + ".globl __sanemaker_target_untag_trap_incision\n" + ".type __sanemaker_target_untag_trap_incision, @notype\n" + "__sanemaker_target_untag_trap_incision:\n" + "ret\n" + ); +} + +/* Sanemaker reads field pointer from rdi and target hash pointer from rsi= */ +SANEMAKER_TRAP_FN +void __sanemaker_finish_layout_trap(const void *fields, const unsigned cha= r *target) +{ + __asm__ volatile( + ".globl __sanemaker_finish_layout_trap_incision\n" + ".type __sanemaker_finish_layout_trap_incision, @notype\n" + "__sanemaker_finish_layout_trap_incision:\n" + "ret\n" + ); +} + +/* Sanemaker reads name from rdi and overwrites rsi to set a value */ +SANEMAKER_TRAP_FN +int __sanemaker_fetch_trap(sanemaker_fetch_t what, int def) +{ + __asm__ volatile( + ".globl __sanemaker_fetch_trap_incision\n" + ".type __sanemaker_fetch_trap_incision, @notype\n" + "__sanemaker_fetch_trap_incision:\n" + "nop\n" + "movl %esi, %eax\n" + "ret\n" + ); +} + +/* Sanemaker reads the signal event from rdi */ +SANEMAKER_TRAP_FN +void __sanemaker_signal_trap(sanemaker_signal_t signal) +{ + __asm__ volatile( + ".globl __sanemaker_signal_trap_incision\n" + ".type __sanemaker_signal_trap_incision, @notype\n" + "__sanemaker_signal_trap_incision:\n" + "ret\n" + ); +} + +/* Sanemaker reads name from rdi and base from rsi */ +SANEMAKER_TRAP_FN +void __sanemaker_new_image_trap(const char *name, const void *base) +{ + __asm__ volatile( + ".globl __sanemaker_new_image_trap_incision\n" + ".type __sanemaker_new_image_trap_incision, @notype\n" + "__sanemaker_new_image_trap_incision:\n" + "ret\n" + ); +} + +/* Sanemaker reads image name from rdi, begin from rsi and end from rdx */ +SANEMAKER_TRAP_FN +void __sanemaker_new_image_text_trap(const char *image, const void *begin,= const void *end) +{ + __asm__ volatile( + ".globl __sanemaker_new_image_text_trap_incision\n" + ".type __sanemaker_new_image_text_trap_incision, @notype\n" + "__sanemaker_new_image_text_trap_incision:\n" + "ret\n" + ); +} + +/* Sanemaker reads image name from rdi */ +SANEMAKER_TRAP_FN +void __sanemaker_drop_image_trap(const char *image) +{ + __asm__ volatile( + ".globl __sanemaker_drop_image_trap_incision\n" + ".type __sanemaker_drop_image_trap_incision, @notype\n" + "__sanemaker_drop_image_trap_incision:\n" + "ret\n" + ); +} + +/* Sanemaker reads image name from rdi, begin from rsi and end from rdx */ +SANEMAKER_TRAP_FN +void __sanemaker_drop_image_text_trap(const char *image, const void *begin= , const void *end) +{ + __asm__ volatile( + ".globl __sanemaker_drop_image_text_trap_incision\n" + ".type __sanemaker_drop_image_text_trap_incision, @notype\n" + "__sanemaker_drop_image_text_trap_incision:\n" + "ret\n" + ); +} + diff --git a/kernel/spslr/spslr.c b/kernel/spslr/spslr.c new file mode 100644 index 000000000000..5ef36731f75e --- /dev/null +++ b/kernel/spslr/spslr.c @@ -0,0 +1,555 @@ +#include +#include + +#include "spslr_randomizer.h" +#include "spslr_env.h" +#include "pinpoint.h" + +/* + * Runtime portion of SPSLR. + * + * This code consumes the metadata emitted by pinpoint, randomizes target + * layouts, rewrites static data objects, and patches instruction immediat= es + * that encode structure field offsets. + */ + +#define SPSLR_SANITY_CHECK + +struct target_map { + spslr_u64 *map; + spslr_u64 size; +}; + +static void init_spslr_meta(void); +static int spslr_targets_compatible(const struct spslr_target *begin, + const struct spslr_target *end); +static enum spslr_error spslr_patch_unit(const struct spslr_unit *unit, + spslr_u64 *tmap_buffer, + void *reorder_buffer); +static struct spslr_status spslr_patch(const struct spslr_ctx *ctx); +static spslr_u64 spslr_target_mapping_size(void); +static spslr_u64 *workspace_target_mapping(void *workspace); +static void *workspace_reorder_buffer(void *workspace); + +static int spslr_patch_dpins(const struct spslr_dpin *dpins, spslr_u64 cnt, + const struct target_map *tmap, + void *reorder_buffer); +static int spslr_patch_dpin(void *addr, spslr_u64 target, void *reorder_bu= ffer); +static int spslr_patch_ipins(const struct spslr_ipin *ipins, spslr_u64 cnt, + const struct target_map *tmap); + +static int reorder_object(void *dst, const void *src, spslr_u64 target); +static int spslr_calculate_ipin_value(const struct spslr_ipin_expr *expr, + spslr_s64 *res, + const struct target_map *tmap); + +static int spslr_map_target(const struct spslr_target *target, spslr_u64 *= idx); +static int spslr_map_targets(const struct spslr_unit *unit, + struct target_map *tmap); + +static int initialized =3D 0, patched =3D 0; +static enum spslr_viability viable =3D SPSLR_VIABLE; + +spslr_u64 spslr_target_cnt =3D 0; +const struct spslr_target *spslr_targets =3D NULL; + +/* Host image spslr metadata entry point */ +extern const struct spslr_unit SPSLR_START_UNITS_SYM[]; +extern const struct spslr_unit SPSLR_STOP_UNITS_SYM[]; +extern const struct spslr_target SPSLR_START_TARGETS_SYM[]; +extern const struct spslr_target SPSLR_STOP_TARGETS_SYM[]; + +/* + * Initialize runtime randomization state. + * + * After this point target layouts are randomized, but code/data does still + * contain original-layout offsets until the patching entry points run. + */ + +static void __init init_spslr_meta(void) +{ + spslr_target_cnt =3D + (spslr_u64)(SPSLR_STOP_TARGETS_SYM - SPSLR_START_TARGETS_SYM); + spslr_targets =3D SPSLR_START_TARGETS_SYM; +} + +struct spslr_status __init spslr_init(void) +{ + if (initialized) + return (struct spslr_status){ .viability =3D viable, + .error =3D SPSLR_OK }; + + init_spslr_meta(); + + if (spslr_randomizer_init() < 0) + return (struct spslr_status){ + .viability =3D viable, + .error =3D SPSLR_ERROR_RANDOMIZER_INIT + }; + +#ifdef SPSLR_SANITY_CHECK + for (spslr_u64 tidx =3D 0; tidx < spslr_target_cnt; tidx++) { + if (spslr_randomizer_validate_target(tidx) < 0) + return (struct spslr_status){ + .viability =3D viable, + .error =3D SPSLR_ERROR_INITIAL_TARGET_LAYOUT + }; + } +#endif + + if (sanemaker_fetch(SANEMAKER_FETCH_SPSLR_ENABLED, 1)) { + if (spslr_randomize() < 0) + return (struct spslr_status){ + .viability =3D viable, + .error =3D SPSLR_ERROR_RANDOMIZE + }; + } + +#ifdef SPSLR_SANITY_CHECK + for (spslr_u64 tidx =3D 0; tidx < spslr_target_cnt; tidx++) { + if (spslr_randomizer_validate_target(tidx) < 0) + return (struct spslr_status){ + .viability =3D viable, + .error =3D SPSLR_ERROR_RANDOMIZED_TARGET_LAYOUT + }; + } +#endif + + initialized =3D 1; + return (struct spslr_status){ .viability =3D viable, .error =3D SPSLR_OK = }; +} + +/* + * Calculate required workspace buffer size. This includes the reorder + * buffer for data pins and the storage for the mapping of local to + * global target indices. + */ + +static spslr_u64 spslr_target_mapping_size(void) +{ + return spslr_target_cnt * sizeof(spslr_u64); +} + +unsigned long spslr_workspace_size(const struct spslr_entry *entry) +{ + if (!entry || !entry->start_units || !entry->stop_units) + return 0; + + const struct spslr_unit *start_units =3D + (const struct spslr_unit *)entry->start_units; + const struct spslr_unit *stop_units =3D + (const struct spslr_unit *)entry->stop_units; + + spslr_u64 max_dpin_size =3D 0; + for (const struct spslr_unit *unit =3D start_units; unit !=3D stop_units; + unit++) { + for (spslr_u64 dpin =3D 0; dpin < unit->dpin_cnt; dpin++) { + const struct spslr_target *target =3D + unit->target_refs[unit->dpins[dpin] + .unit_target_idx]; + + if (target->layout->size > max_dpin_size) + max_dpin_size =3D target->layout->size; + } + } + + return spslr_target_mapping_size() + max_dpin_size; +} + +/* + * Check if the given target space is compatible with that of the + * host. + */ + +static int spslr_meta_known_target(const struct spslr_target *t) +{ + for (spslr_u64 i =3D 0; i < spslr_target_cnt; i++) { + if (spslr_env_memcmp(t->hash, spslr_targets[i].hash, + sizeof(t->hash)) =3D=3D 0) + return 1; + } + + return 0; +} + +static int spslr_targets_compatible(const struct spslr_target *begin, + const struct spslr_target *end) +{ + spslr_u64 cnt =3D (spslr_u64)(end - begin); + if (cnt > spslr_target_cnt) + return 0; + + for (spslr_u64 i =3D 0; i < cnt; i++) { + if (!spslr_meta_known_target(begin + i)) + return 0; + } + + return 1; +} + +/* + * For each CU, map local target indices to global target indices and then + * to host indices. Afterwards, patch ipins and dpins. + */ + +static spslr_u64 *workspace_target_mapping(void *workspace) +{ + return (spslr_u64 *)workspace; +} + +static void *workspace_reorder_buffer(void *workspace) +{ + return (spslr_u8 *)workspace + spslr_target_mapping_size(); +} + +static int spslr_map_target(const struct spslr_target *target, spslr_u64 *= idx) +{ + for (spslr_u64 i =3D 0; i < spslr_target_cnt; i++) { + if (spslr_env_memcmp(target->hash, spslr_targets[i].hash, + sizeof(target->hash)) =3D=3D 0) { + *idx =3D i; + return 0; + } + } + + return -1; +} + +static int spslr_map_targets(const struct spslr_unit *unit, + struct target_map *tmap) +{ + tmap->size =3D 0; + + for (spslr_u64 i =3D 0; i < unit->target_cnt; i++) { + if (spslr_map_target(unit->target_refs[i], tmap->map + i) < 0) + return -1; + } + + tmap->size =3D unit->target_cnt; + return 0; +} + +static enum spslr_error spslr_patch_unit(const struct spslr_unit *unit, + spslr_u64 *tmap_buffer, + void *reorder_buffer) +{ + struct target_map tmap =3D { .map =3D tmap_buffer, .size =3D 0 }; + + if (spslr_map_targets(unit, &tmap) < 0) + return SPSLR_ERROR_MAP_TARGETS; + + if (spslr_patch_dpins(unit->dpins, unit->dpin_cnt, &tmap, + reorder_buffer) < 0) + return SPSLR_ERROR_PATCH_DPINS; + + if (spslr_patch_ipins(unit->ipins, unit->ipin_cnt, &tmap) < 0) + return SPSLR_ERROR_PATCH_IPINS; + + return SPSLR_OK; +} + +static struct spslr_status spslr_patch(const struct spslr_ctx *ctx) +{ + enum spslr_error err =3D SPSLR_OK; + enum spslr_viability via =3D SPSLR_VIABLE; + + spslr_u64 *target_map_buffer =3D NULL; + void *reorder_buffer =3D NULL; + + const struct spslr_unit *start_units =3D NULL; + const struct spslr_unit *stop_units =3D NULL; + const struct spslr_target *start_targets =3D NULL; + const struct spslr_target *stop_targets =3D NULL; + + if (!ctx || !ctx->entry.start_units || !ctx->entry.stop_units || + !ctx->entry.start_targets || !ctx->entry.stop_targets || + !ctx->workspace) { + err =3D SPSLR_ERROR_INCOMPLETE_CTX; + goto finish; + } + + start_units =3D (const struct spslr_unit *)ctx->entry.start_units; + stop_units =3D (const struct spslr_unit *)ctx->entry.stop_units; + start_targets =3D (const struct spslr_target *)ctx->entry.start_targets; + stop_targets =3D (const struct spslr_target *)ctx->entry.stop_targets; + + target_map_buffer =3D workspace_target_mapping(ctx->workspace); + reorder_buffer =3D workspace_reorder_buffer(ctx->workspace); + + if (!spslr_targets_compatible(start_targets, stop_targets)) { + err =3D SPSLR_ERROR_INCOMPATIBLE_CTX; + goto finish; + } + + via =3D SPSLR_NONVIABLE; + + if (sanemaker_fetch(SANEMAKER_FETCH_SPSLR_ENABLED, 1)) { + for (const struct spslr_unit *unit =3D start_units; + unit !=3D stop_units; unit++) { + err =3D spslr_patch_unit(unit, target_map_buffer, + reorder_buffer); + if (err !=3D SPSLR_OK) + goto finish; + } + } + + via =3D SPSLR_VIABLE; + +finish: + return (struct spslr_status){ .viability =3D via, .error =3D err }; +} + +/* + * Patch the main executable. + * + * Instruction pins rewrite immediate operands in text, while data pins re= write + * existing static objects from original layout into randomized layout. + */ + +struct spslr_status __init spslr_selfpatch(void) +{ + enum spslr_error err =3D SPSLR_OK; + + spslr_u64 host_workspace_size; + + struct spslr_ctx host_ctx; + host_ctx.entry.start_units =3D SPSLR_START_UNITS_SYM; + host_ctx.entry.stop_units =3D SPSLR_STOP_UNITS_SYM; + host_ctx.entry.start_targets =3D SPSLR_START_TARGETS_SYM; + host_ctx.entry.stop_targets =3D SPSLR_STOP_TARGETS_SYM; + host_ctx.workspace =3D NULL; + + struct spslr_status internal_patch_status; + + if (patched) { + err =3D SPSLR_ERROR_ALREADY_PATCHED; + goto finish; + } + + if (!initialized) { + err =3D SPSLR_ERROR_UNINITIALIZED; + goto finish; + } + + host_workspace_size =3D spslr_workspace_size(&host_ctx.entry); + host_ctx.workspace =3D spslr_env_malloc(host_workspace_size); + + if (!host_ctx.workspace) { + err =3D SPSLR_ERROR_MEMORY; + goto finish; + } + + internal_patch_status =3D spslr_patch(&host_ctx); + if (internal_patch_status.error =3D=3D SPSLR_OK) + patched =3D 1; + + viable =3D internal_patch_status.viability; + err =3D internal_patch_status.error; + +finish: + if (host_ctx.workspace) + spslr_env_free(host_ctx.workspace, host_workspace_size); + + sanemaker_signal(SANEMAKER_SIGNAL_PATCH_BOUNDARY); + return (struct spslr_status){ .viability =3D viable, .error =3D err }; +} + +/* + * Patch metadata belonging to a separately loaded module. + * + * Modules reuse the target randomization state created by the main execut= able; + * they contribute only their own instruction and data patch sites. + */ + +struct spslr_status spslr_patch_module(const struct spslr_ctx *m) +{ + if (!initialized) + return (struct spslr_status){ + .viability =3D SPSLR_VIABLE, + .error =3D SPSLR_ERROR_UNINITIALIZED + }; + + if (!m || !m->entry.start_units || !m->entry.stop_units || + !m->entry.start_targets || !m->entry.stop_targets || !m->workspace) + return (struct spslr_status){ + .viability =3D SPSLR_VIABLE, + .error =3D SPSLR_ERROR_INCOMPLETE_CTX + }; + + struct spslr_status s =3D spslr_patch(m); + return (struct spslr_status){ .viability =3D (s.error =3D=3D SPSLR_OK ? + SPSLR_VIABLE : + SPSLR_NONVIABLE), + .error =3D s.error }; +} + +/* + * Rewrite one object instance from original layout into randomized layout. + * + * A temporary buffer is used so overlapping source/destination field rang= es do + * not corrupt data while fields are moved. + */ + +static int reorder_object(void *dst, const void *src, spslr_u64 target) +{ + spslr_u64 field_count; + if (spslr_randomizer_get_target(target, NULL, &field_count)) + return -1; + + const spslr_u8 *src_countable =3D (const spslr_u8 *)src; + spslr_u8 *dst_countable =3D (spslr_u8 *)dst; + + for (spslr_u64 i =3D 0; i < field_count; i++) { + struct spslr_randomizer_field_info finfo; + if (spslr_randomizer_get_field( + target, i, SPSLR_RANDOMIZER_FIELD_IDX_MODE_FINAL, + &finfo)) + return -1; + + spslr_env_memcpy(dst_countable + finfo.offset, + src_countable + finfo.initial_offset, + finfo.size); + } + + return 0; +} + +/* + * Apply data pin patches. + * + * Each pin's address already points at an existing object in original lay= out. Patching + * converts that storage in-place to the target's randomized layout. + */ + +static int spslr_patch_dpins(const struct spslr_dpin *dpins, spslr_u64 cnt, + const struct target_map *tmap, + void *reorder_buffer) +{ + for (spslr_u64 dpidx =3D 0; dpidx < cnt; dpidx++) { + const struct spslr_dpin *dp =3D &dpins[dpidx]; + + if (dp->unit_target_idx >=3D tmap->size) + return -1; + + if (spslr_patch_dpin((void *)dp->addr, + tmap->map[dp->unit_target_idx], + reorder_buffer) < 0) + return -1; + } + + return 0; +} + +static int spslr_patch_dpin(void *addr, spslr_u64 target, void *reorder_bu= ffer) +{ + if (target >=3D spslr_target_cnt) + return -1; + + int res =3D -1; + const struct spslr_target *t =3D &spslr_targets[target]; + + sanemaker_signal(SANEMAKER_SIGNAL_PAUSE); + + spslr_env_memset(reorder_buffer, 0, t->layout->size); + + if (reorder_object(reorder_buffer, addr, target) < 0) + goto finish; + + if (spslr_env_poke_data(addr, reorder_buffer, t->layout->size) < 0) + goto finish; + + res =3D 0; +finish: + sanemaker_signal(SANEMAKER_SIGNAL_RESUME); + return res; +} + +static int spslr_ipin_value_fits(spslr_u64 value, spslr_u64 size) +{ + if (size =3D=3D 0) + return 0; + + spslr_u64 bound =3D (spslr_u64)1 << (8 * size); + return value < bound; +} + +static int spslr_patch_ipins(const struct spslr_ipin *ipins, spslr_u64 cnt, + const struct target_map *tmap) +{ + for (spslr_u64 ipidx =3D 0; ipidx < cnt; ipidx++) { + const struct spslr_ipin *ip =3D &ipins[ipidx]; + + spslr_s64 value; + if (spslr_calculate_ipin_value(ip->expr, &value, tmap) < 0) + return -1; + + if (value < 0 || + !spslr_ipin_value_fits((spslr_u64)value, ip->size)) + return -1; + + /* + * Text patching is deliberately scoped to the immediate field only. + * The surrounding instruction bytes were fixed by pinpoint/patchcompile= and + * must not change at runtime. + */ + + switch (ip->size) { + case 1: + if (spslr_env_poke_text_8((void *)ip->addr, + (spslr_u8)value) < 0) + return -1; + break; + case 2: + if (spslr_env_poke_text_16((void *)ip->addr, + (spslr_u16)value) < 0) + return -1; + break; + case 4: + if (spslr_env_poke_text_32((void *)ip->addr, + (spslr_u32)value) < 0) + return -1; + break; + case 8: + if (spslr_env_poke_text_64((void *)ip->addr, + (spslr_u64)value) < 0) + return -1; + break; + default: + return -1; + } + } + + return 0; +} + +/* + * Interpret one ipin program and compute the replacement immediate value. + * + * The program describes original target/field references; this function m= aps + * them through the randomized runtime layout and returns the value writte= n into + * the instruction stream. + */ + +static int spslr_calculate_ipin_value(const struct spslr_ipin_expr *expr, + spslr_s64 *res, + const struct target_map *tmap) +{ + if (!res) + return -1; + + *res =3D 0; + + if (expr->unit_target_idx >=3D tmap->size) + return -1; + + spslr_u64 global_target_idx =3D tmap->map[expr->unit_target_idx]; + + struct spslr_randomizer_field_info finfo; + if (spslr_randomizer_get_field(global_target_idx, expr->field_idx, + SPSLR_RANDOMIZER_FIELD_IDX_MODE_ORIGINAL, + &finfo) !=3D 0) + return -1; + + *res =3D finfo.offset; + return 0; +} diff --git a/kernel/spslr/spslr_env.c b/kernel/spslr/spslr_env.c new file mode 100644 index 000000000000..a4a2d2389dcd --- /dev/null +++ b/kernel/spslr/spslr_env.c @@ -0,0 +1,74 @@ +#include "spslr_env.h" + +#include +#include +#include +#include + +#ifdef CONFIG_X86 + +#include + +static __always_inline int spslr_env_poke_text(void *dst, const void *src,= size_t n) +{ + text_poke_early(dst, src, n); + return 0; +} + +#endif + +int spslr_env_poke_text_8(void *dst, u8 value) +{ + return spslr_env_poke_text(dst, &value, sizeof(value)); +} + +int spslr_env_poke_text_16(void *dst, u16 value) +{ + return spslr_env_poke_text(dst, &value, sizeof(value)); +} + +int spslr_env_poke_text_32(void *dst, u32 value) +{ + return spslr_env_poke_text(dst, &value, sizeof(value)); +} + +int spslr_env_poke_text_64(void *dst, u64 value) +{ + return spslr_env_poke_text(dst, &value, sizeof(value)); +} + +/* + * Hook runs before slab allocators are available. + * memblock_alloc() is the correct early-boot allocator. + */ +void* __init spslr_env_malloc(spslr_u64 n) { + size_t size =3D PAGE_ALIGN(n ? n : 1); + return memblock_alloc(size, SMP_CACHE_BYTES); +} + +void __init spslr_env_free(void *ptr, spslr_u64 n) { + if (ptr) + memblock_free(ptr, PAGE_ALIGN(n ? n : 1)); +} + +int spslr_env_poke_data(void* dst, const void* src, spslr_u64 n) { + memcpy(dst, src, n); + return 0; +} + +void spslr_env_memset(void* dst, int v, spslr_u64 n) { + memset(dst, v, n); +} + +void spslr_env_memcpy(void* dst, const void* src, spslr_u64 n) { + memcpy(dst, src, n); +} + +int spslr_env_memcmp(const void *x, const void *y, spslr_u64 n) { + return memcmp(x, y, n); +} + +spslr_u64 __init spslr_env_random_u64(void) { + return get_random_u64(); // Hook runs after random_init_early() +} + diff --git a/kernel/spslr/spslr_env.h b/kernel/spslr/spslr_env.h new file mode 100644 index 000000000000..f48d7e02c57e --- /dev/null +++ b/kernel/spslr/spslr_env.h @@ -0,0 +1,45 @@ +#ifndef SPSLR_ENV_H +#define SPSLR_ENV_H + +#include +#include +#include + +#ifndef __packed +#define __packed __attribute__((packed)) +#endif + +#ifndef __init +#define __init /* only required in kernel */ +#endif + +#ifndef __printf +#define __printf(fmt_pos, arg_pos) \ + __attribute__((format(printf, fmt_pos, arg_pos))) +#endif + +#ifndef NULL +#define NULL ((void *)0) +#endif + +typedef uint8_t spslr_u8; +typedef uint16_t spslr_u16; +typedef uint32_t spslr_u32; +typedef uint64_t spslr_u64; +typedef int32_t spslr_s32; +typedef int64_t spslr_s64; +typedef uintptr_t spslr_uintptr; + +int spslr_env_poke_text_8(void *dst, spslr_u8 value); +int spslr_env_poke_text_16(void *dst, spslr_u16 value); +int spslr_env_poke_text_32(void *dst, spslr_u32 value); +int spslr_env_poke_text_64(void *dst, spslr_u64 value); +int spslr_env_poke_data(void *dst, const void *src, spslr_u64 n); +void *spslr_env_malloc(spslr_u64 n); +void spslr_env_free(void *ptr, spslr_u64 n); +void spslr_env_memset(void *dst, int v, spslr_u64 n); +void spslr_env_memcpy(void *dst, const void *src, spslr_u64 n); +int spslr_env_memcmp(const void *x, const void *y, spslr_u64 n); +spslr_u64 spslr_env_random_u64(void); + +#endif diff --git a/kernel/spslr/spslr_randomizer.c b/kernel/spslr/spslr_randomize= r.c new file mode 100644 index 000000000000..879ead0a2384 --- /dev/null +++ b/kernel/spslr/spslr_randomizer.c @@ -0,0 +1,646 @@ +#include "spslr_randomizer.h" + +#include "spslr_env.h" +#include "pinpoint.h" + +#include + +/* + * Target layout randomizer. + * + * The randomizer builds a permutation from original field order to random= ized + * field order while preserving field size, alignment, and fixed-field + * constraints. + */ + +/* + * Field tracks both directions of the permutation: + * + * original index -> randomized position + * randomized position -> original index + * + * The runtime needs both: data patching copies from original offsets to n= ew + * offsets, while ipin patching maps an original field offset to its rando= mized + * offset. + */ +struct Field { + spslr_u64 offset; /* Final field offset -> fields[i].offset =3D offset of= field i in final layout */ + spslr_u64 oidx; /* Original field idx -> fields[i].oidx =3D original posi= tion of field i in final layout */ + spslr_u64 fidx; /* Final field idx -> fields[i].fidx =3D randomized/final= position of original field i */ +}; + +extern spslr_u64 spslr_target_cnt; +extern const struct spslr_target *spslr_targets; + +static spslr_u64 *field_base_indices =3D NULL; +static struct Field *fields =3D NULL; + +static int init_field_base_indices(void); +static int init_fields_buffer(void); + +static const struct spslr_target_field *meta_original_field(spslr_u64 targ= et, + spslr_u64 field); +static struct Field *state_current_field_base(spslr_u64 target); +static struct Field *state_current_field(spslr_u64 target, spslr_u64 field= ); + +static int __init init_field_base_indices(void) +{ + field_base_indices =3D (spslr_u64 *)spslr_env_malloc(sizeof(spslr_u64) * + spslr_target_cnt); + if (!field_base_indices) + return -1; + + spslr_u64 current_field_base_idx =3D 0; + for (spslr_u64 i =3D 0; i < spslr_target_cnt; i++) { + field_base_indices[i] =3D current_field_base_idx; + current_field_base_idx +=3D spslr_targets[i].layout->field_cnt; + } + + return 0; +} + +static const struct spslr_target_field *meta_original_field(spslr_u64 targ= et, + spslr_u64 field) +{ + if (target >=3D spslr_target_cnt) + return NULL; + + const struct spslr_target_layout *layout =3D spslr_targets[target].layout; + + if (field >=3D layout->field_cnt) + return NULL; + + return layout->fields + field; +} + +static struct Field *state_current_field_base(spslr_u64 target) +{ + if (target >=3D spslr_target_cnt) + return NULL; + + spslr_u64 field_base_idx =3D field_base_indices[target]; + return fields + field_base_idx; +} + +static struct Field *state_current_field(spslr_u64 target, spslr_u64 field) +{ + if (target >=3D spslr_target_cnt) + return NULL; + + struct Field *field_base =3D state_current_field_base(target); + const struct spslr_target_layout *layout =3D spslr_targets[target].layout; + + if (!field_base || field >=3D layout->field_cnt) + return NULL; + + return field_base + field; +} + +static int __init init_fields_buffer(void) +{ + spslr_u64 total_field_count =3D 0; + for (spslr_u64 i =3D 0; i < spslr_target_cnt; i++) + total_field_count +=3D spslr_targets[i].layout->field_cnt; + + fields =3D (struct Field *)spslr_env_malloc(sizeof(struct Field) * + total_field_count); + if (!fields) + return -1; + + for (spslr_u64 i =3D 0; i < spslr_target_cnt; i++) { + spslr_u64 field_base_idx =3D field_base_indices[i]; + + for (spslr_u64 field_idx =3D 0; + field_idx < spslr_targets[i].layout->field_cnt; + field_idx++) { + const struct spslr_target_field *src_field =3D + spslr_targets[i].layout->fields + field_idx; + struct Field *dst_field =3D + &fields[field_base_idx + field_idx]; + + dst_field->offset =3D src_field->offset; + dst_field->oidx =3D field_idx; + dst_field->fidx =3D field_idx; + } + } + + return 0; +} + +int __init spslr_randomizer_init(void) +{ + if (init_field_base_indices() !=3D 0) + return -1; + + if (init_fields_buffer() !=3D 0) + return -1; + + return 0; +} + +int spslr_randomizer_get_target(spslr_u64 target, spslr_u64 *size, + spslr_u64 *fieldcnt) +{ + if (target >=3D spslr_target_cnt) + return -1; + + const struct spslr_target *t =3D &spslr_targets[target]; + + if (size) + *size =3D t->layout->size; + + if (fieldcnt) + *fieldcnt =3D t->layout->field_cnt; + + return 0; +} + +int spslr_randomizer_get_field(spslr_u64 target, spslr_u64 field, + int field_idx_mode, + struct spslr_randomizer_field_info *info) +{ + if (target >=3D spslr_target_cnt) + return -1; + + if (!info) + return 0; + + const struct spslr_target *t =3D &spslr_targets[target]; + + if (field >=3D t->layout->field_cnt) + return -1; + + const struct spslr_target_field *of =3D NULL; + const struct Field *rf =3D NULL; + + switch (field_idx_mode) { + case SPSLR_RANDOMIZER_FIELD_IDX_MODE_ORIGINAL: + of =3D meta_original_field(target, field); + rf =3D state_current_field( + target, state_current_field(target, field)->fidx); + break; + case SPSLR_RANDOMIZER_FIELD_IDX_MODE_FINAL: + of =3D meta_original_field( + target, state_current_field(target, field)->oidx); + rf =3D state_current_field(target, field); + break; + default: + return -1; + } + + info->size =3D of->size; + info->offset =3D rf->offset; + info->initial_offset =3D of->offset; + info->alignment =3D of->alignment; + info->flags =3D of->flags; + + return 0; +} + +int __init spslr_randomizer_validate_target(spslr_u64 target) +{ + spslr_u64 tsize, fieldcnt; + + if (spslr_randomizer_get_target(target, &tsize, &fieldcnt) < 0) + return -1; + + spslr_u64 cur_end =3D 0; + + for (spslr_u64 i =3D 0; i < fieldcnt; i++) { + struct spslr_randomizer_field_info finfo; + if (spslr_randomizer_get_field( + target, i, SPSLR_RANDOMIZER_FIELD_IDX_MODE_FINAL, + &finfo) < 0) + return -1; + + if (finfo.alignment =3D=3D 0) + return -1; + + if (finfo.offset % finfo.alignment !=3D 0) + return -1; + + if ((finfo.flags & SPSLR_FLAG_FIELD_FIXED) && + finfo.offset !=3D finfo.initial_offset) + return -1; + + if (finfo.offset > tsize) + return -1; + + /* Zero-sized metadata entries occupy no storage. */ + if (finfo.size =3D=3D 0) + continue; + + if (finfo.offset < cur_end) + return -1; + + /* Avoid overflow in offset + size. */ + if (finfo.size > tsize - finfo.offset) + return -1; + + cur_end =3D finfo.offset + finfo.size; + } + + return 0; +} + +// RANDOMIZATION CODE + +struct ShuffleRegion { + spslr_u64 begin; + spslr_u64 end; + spslr_u64 fill_begin; + spslr_u64 fill_end; +}; + +static spslr_u64 rand_u64(void); +static void get_origin_region(spslr_u64 target, spslr_u64 final_idx, + struct ShuffleRegion *region); +static int option_is_valid(spslr_u64 target, spslr_u64 origin_final_idx, + const struct ShuffleRegion *origin, + spslr_u64 offset); +static int pick_shuffle_option(spslr_u64 target, spslr_u64 origin_final_id= x, + const struct ShuffleRegion *origin, + spslr_u64 alignment, spslr_u64 *selected); +static void do_swap(spslr_u64 target, spslr_u64 origin_final_idx, + const struct ShuffleRegion *origin_region, + spslr_u64 new_offset); +static void shuffle_one_target(spslr_u64 target); +static void shuffle_target(spslr_u64 target); + +static spslr_u64 __init rand_u64(void) +{ + return spslr_env_random_u64(); +} + +static void __init get_origin_region(spslr_u64 target, spslr_u64 final_idx, + struct ShuffleRegion *region) +{ + const struct spslr_target *t =3D &spslr_targets[target]; + const struct Field *rf =3D state_current_field(target, final_idx); + const struct spslr_target_field *of =3D + meta_original_field(target, rf->oidx); + + region->fill_begin =3D rf->offset; + region->fill_end =3D region->fill_begin + of->size; + + if (final_idx =3D=3D 0) { + region->begin =3D 0; + } else { + const struct Field *pred_rf =3D + state_current_field(target, final_idx - 1); + const struct spslr_target_field *pred_of =3D + meta_original_field(target, pred_rf->oidx); + region->begin =3D pred_rf->offset + pred_of->size; + } + + if (final_idx + 1 >=3D t->layout->field_cnt) { + region->end =3D t->layout->size; + } else { + const struct Field *succ_rf =3D + state_current_field(target, final_idx + 1); + region->end =3D succ_rf->offset; + } +} + +/* + * Check whether a proposed field move preserves layout constraints. + * + * A move is valid only if displaced fields can be packed into the freed r= egion + * without violating alignment or moving fields marked fixed. + */ + +static int __init option_is_valid(spslr_u64 target, spslr_u64 origin_final= _idx, + const struct ShuffleRegion *origin, + spslr_u64 offset) +{ + const struct spslr_target *t =3D &spslr_targets[target]; + const struct spslr_target_field *origin_of =3D meta_original_field( + target, state_current_field(target, origin_final_idx)->oidx); + + // When placed at offset, field will occupy [offset, option_would_end) + spslr_u64 option_would_end =3D offset + origin_of->size; + if (option_would_end > t->layout->size) + return 0; + + // Field may overlap with origin region. Moving field to offset truly fre= es: + // [true_origin_region_begin, true_origin_region_end) + spslr_u64 true_origin_region_begin =3D origin->begin; + spslr_u64 true_origin_region_end =3D origin->end; + + if (offset <=3D origin->fill_begin && + option_would_end > true_origin_region_begin) + true_origin_region_begin =3D option_would_end; + + if (offset >=3D origin->fill_begin && offset < true_origin_region_end) + true_origin_region_end =3D offset; + + // Iterate over fields in target region [offset, option_would_end] and se= e if they fit into true origin region + spslr_u64 origin_region_ptr =3D true_origin_region_begin; + for (spslr_u64 it =3D 0; it < t->layout->field_cnt; it++) { + const struct Field *rf =3D state_current_field(target, it); + const struct spslr_target_field *of =3D + meta_original_field(target, rf->oidx); + + // The field being moved does not need to go into origin region + if (it =3D=3D origin_final_idx) + continue; + + /* + * Zero-sized metadata entries occupy no storage, but they still + * mark an ordering boundary. A moved field must neither straddle + * one nor start at one: equal-offset entries have an ordering + * relationship that do_swap() does not preserve while displacing + * fields. + */ + if (of->size =3D=3D 0) { + if (rf->offset >=3D offset && + rf->offset < option_would_end) + return 0; + + continue; + } + + // Field ends before target region -> must not be moved to origin region + if (rf->offset + of->size <=3D offset) + continue; + + // Field starts after target region -> must not be moved to origin region + if (rf->offset >=3D option_would_end) + break; + + // Fixed fields in target region unconditionally deny option + if (of->flags & SPSLR_FLAG_FIELD_FIXED) + return 0; + + // Field from target region must be moved to aligned position in origin = region + if (origin_region_ptr % of->alignment !=3D 0) + origin_region_ptr +=3D + of->alignment - + (origin_region_ptr % of->alignment); + + origin_region_ptr +=3D of->size; + + // Field does not fit into origin region -> option not possible + if (origin_region_ptr > true_origin_region_end) + return 0; + } + + return 1; +} + +static int __init pick_shuffle_option(spslr_u64 target, + spslr_u64 origin_final_idx, + const struct ShuffleRegion *origin, + spslr_u64 alignment, spslr_u64 *selected) +{ + const struct spslr_target *t =3D &spslr_targets[target]; + spslr_u64 seen =3D 0; + + /* + Note: Instead of looping over entire field array for each option, loops c= an be merged into one. + */ + + for (spslr_u64 offset =3D 0; offset < t->layout->size; + offset +=3D alignment) { + if (!option_is_valid(target, origin_final_idx, origin, offset)) + continue; + + // Reservoir sampling -> uniform distribution with O(1) memory consumpti= on + seen++; + if ((rand_u64() % seen) =3D=3D 0) + *selected =3D offset; + } + + return seen ? 0 : -1; +} + +/* + * Move one field into a new slot and repack the fields it displaced. + * + * This is not a simple pairwise swap: structure layout has byte ranges and + * alignment holes, so the displaced region may contain several fields. + */ + +static void __init do_swap(spslr_u64 target, spslr_u64 origin_idx, + const struct ShuffleRegion *origin_region, + spslr_u64 new_offset) +{ + const struct spslr_target *t =3D &spslr_targets[target]; + int pulled =3D 0; + + spslr_u64 option_fill_end =3D new_offset + (origin_region->fill_end - + origin_region->fill_begin); + + spslr_u64 true_origin_region_begin =3D origin_region->begin; + if (new_offset <=3D origin_region->fill_begin && + option_fill_end > true_origin_region_begin) + true_origin_region_begin =3D option_fill_end; + + spslr_u64 origin_oidx =3D state_current_field(target, origin_idx)->oidx; + + spslr_u64 origin_region_ptr =3D true_origin_region_begin; + for (spslr_u64 it =3D 0; it < t->layout->field_cnt; it++) { + struct Field *itf =3D state_current_field(target, it); + + if (itf->oidx =3D=3D origin_oidx) + continue; + + const struct spslr_target_field *itof =3D + meta_original_field(target, itf->oidx); + + // Zero-sized metadata entries occupy no storage. + if (itof->size =3D=3D 0) + continue; + + if (itf->offset + itof->size <=3D new_offset) + continue; + + if (itf->offset >=3D option_fill_end) + break; + + spslr_u64 falign =3D itof->alignment; + if (origin_region_ptr % falign !=3D 0) + origin_region_ptr +=3D + falign - (origin_region_ptr % falign); + + if (!pulled) { + pulled =3D 1; + + struct Field tmp =3D *state_current_field(target, it); + *state_current_field(target, it) =3D + *state_current_field(target, origin_idx); + *state_current_field(target, origin_idx) =3D tmp; + + state_current_field(target, it)->offset =3D new_offset; + + state_current_field(target, origin_idx)->offset =3D + origin_region_ptr; + origin_region_ptr +=3D + meta_original_field( + target, + state_current_field(target, origin_idx) + ->oidx) + ->size; + continue; + } + + { + struct Field tmp =3D *state_current_field(target, it); + + if (origin_idx >=3D it) { + for (spslr_u64 pull_it =3D it + 1; + pull_it <=3D origin_idx; pull_it++) + *state_current_field(target, + pull_it - 1) =3D + *state_current_field(target, + pull_it); + + *state_current_field(target, origin_idx) =3D tmp; + state_current_field(target, origin_idx)->offset =3D + origin_region_ptr; + origin_region_ptr +=3D + meta_original_field( + target, + state_current_field(target, + origin_idx) + ->oidx) + ->size; + + it--; // Must still look at the element now at it + } else { + for (spslr_u64 pull_it =3D it; + pull_it > origin_idx + (spslr_u64)pulled; + pull_it--) + *state_current_field(target, pull_it) =3D + *state_current_field( + target, pull_it - 1); + + *state_current_field( + target, + origin_idx + (spslr_u64)pulled) =3D tmp; + state_current_field( + target, origin_idx + (spslr_u64)pulled) + ->offset =3D origin_region_ptr; + origin_region_ptr +=3D + meta_original_field( + target, + state_current_field( + target, + origin_idx + + (spslr_u64) + pulled) + ->oidx) + ->size; + } + } + + pulled++; + } + + /* + * The selected destination may not overlap any other field. It may be an + * empty padding gap, or it may partially overlap the origin field itself + * when the field slides into adjacent padding. + * + * In either case the loop above never displaces another field and + * `pulled` remains zero. We still need to update the origin field's + * offset and reinsert it at the correct position in final-offset order so + * that the field array remains sorted. + */ + if (!pulled) { + struct Field origin =3D *state_current_field(target, origin_idx); + spslr_u64 insert_idx =3D t->layout->field_cnt; + + for (spslr_u64 it =3D 0; it < t->layout->field_cnt; it++) { + if (it =3D=3D origin_idx) + continue; + + if (state_current_field(target, it)->offset >=3D + new_offset) { + insert_idx =3D it; + break; + } + } + + if (insert_idx > origin_idx) + insert_idx--; + + if (origin_idx < insert_idx) { + for (spslr_u64 it =3D origin_idx + 1; it <=3D insert_idx; + it++) + *state_current_field(target, it - 1) =3D + *state_current_field(target, it); + } else if (origin_idx > insert_idx) { + for (spslr_u64 it =3D origin_idx; it > insert_idx; it--) + *state_current_field(target, it) =3D + *state_current_field(target, it - 1); + } + + *state_current_field(target, insert_idx) =3D origin; + state_current_field(target, insert_idx)->offset =3D new_offset; + } + + /* + * Rebuild original->final mapping for this target. + */ + for (spslr_u64 final_idx =3D 0; final_idx < t->layout->field_cnt; + final_idx++) { + struct Field *rf =3D state_current_field(target, final_idx); + state_current_field(target, rf->oidx)->fidx =3D final_idx; + } +} + +/* +Note: final version should not shuffle random fields but try to shuffle ea= ch original field idx once +*/ +static void __init shuffle_one_target(spslr_u64 target) +{ + const struct spslr_target *t =3D &spslr_targets[target]; + if (t->layout->field_cnt =3D=3D 0) + return; + + spslr_u64 origin_final_idx =3D rand_u64() % t->layout->field_cnt; + struct Field *origin_rf =3D state_current_field(target, origin_final_idx); + const struct spslr_target_field *origin_of =3D + meta_original_field(target, origin_rf->oidx); + + /* Zero-sized entries are metadata markers, not shuffleable storage. */ + if (origin_of->size =3D=3D 0) + return; + + if (origin_of->flags & SPSLR_FLAG_FIELD_FIXED) + return; + + struct ShuffleRegion origin_region; + spslr_u64 selected_option; + + get_origin_region(target, origin_final_idx, &origin_region); + + if (pick_shuffle_option(target, origin_final_idx, &origin_region, + origin_of->alignment, &selected_option) < 0) + return; + + do_swap(target, origin_final_idx, &origin_region, selected_option); +} + +static void __init shuffle_target(spslr_u64 target) +{ + const struct spslr_target *t =3D &spslr_targets[target]; + spslr_u64 shuffle_count =3D t->layout->field_cnt * 2; + + for (spslr_u64 i =3D 0; i < shuffle_count; i++) + shuffle_one_target(target); + + sanemaker_finish_layout(state_current_field_base(target), t->hash); +} + +int __init spslr_randomize(void) +{ + if (!fields) + return -1; + + for (spslr_u64 tidx =3D 0; tidx < spslr_target_cnt; tidx++) + shuffle_target(tidx); + + return 0; +} diff --git a/kernel/spslr/spslr_randomizer.h b/kernel/spslr/spslr_randomize= r.h new file mode 100644 index 000000000000..c360750894e7 --- /dev/null +++ b/kernel/spslr/spslr_randomizer.h @@ -0,0 +1,29 @@ +#ifndef SPSLR_RANDOMIZER_H +#define SPSLR_RANDOMIZER_H + +#include "spslr_env.h" +#include "pinpoint.h" + +#define SPSLR_RANDOMIZER_FIELD_IDX_MODE_ORIGINAL 1 +#define SPSLR_RANDOMIZER_FIELD_IDX_MODE_FINAL 2 + +struct spslr_randomizer_field_info { + spslr_u64 size; + spslr_u64 offset; + spslr_u64 initial_offset; + spslr_u64 alignment; + spslr_u64 flags; +}; + +int spslr_randomizer_init(void); +int spslr_randomize(void); + +int spslr_randomizer_get_target(spslr_u64 target, spslr_u64 *size, + spslr_u64 *fieldcnt); +int spslr_randomizer_get_field(spslr_u64 target, spslr_u64 field, + int field_idx_mode, + struct spslr_randomizer_field_info *info); + +int spslr_randomizer_validate_target(spslr_u64 target); + +#endif --=20 2.43.0 From nobody Sat Jul 25 01:54:02 2026 Received: from mail-wm1-f41.google.com (mail-wm1-f41.google.com [209.85.128.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 78A69394E91 for ; Mon, 20 Jul 2026 19:13:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784574797; cv=none; b=r51fFuYckW8FnfpNlyUL9F4rbGFAT77ra8S370mIBZ0nhTCewV0uYuMV7zLi/phNWlMY6v27UuIFMvdO+cCiWyxQV0dHQt5hRUFlxGDRxk4i2vRUWI0EsxBAD2cRHMpFBVYf12lzF1cEHGLbUCe/zcx013wcojU4qfA5afsy7co= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784574797; c=relaxed/simple; bh=N3MHexzpZGb34a2VrVrJbevD3q35nOnf2KzdfaFS0mc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ZfqXZ/wo1ruyJHURlJDmCCTzwjXbhld+2t1HshDoK1z5OK0SaGyomDwbiFthYVWDvtbpWspWqQyUa1QosZhkfb2kGvJ/20y6/ckGewka+g7UwpqgFD2AWSZeVm2CQxJ82AoCRmjOXpGZ6VH7vUJmjaxU578cGxknOXA7X5Yvfss= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=yjn-systems.com; spf=none smtp.mailfrom=yjn-systems.com; dkim=pass (2048-bit key) header.d=yjn-systems.com header.i=@yjn-systems.com header.b=gbSiG6V8; arc=none smtp.client-ip=209.85.128.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=yjn-systems.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=yjn-systems.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=yjn-systems.com header.i=@yjn-systems.com header.b="gbSiG6V8" Received: by mail-wm1-f41.google.com with SMTP id 5b1f17b1804b1-4921eed3fa2so80053855e9.0 for ; Mon, 20 Jul 2026 12:13:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yjn-systems.com; s=google; t=1784574792; x=1785179592; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=ke8s+Qt7PFdWHYg6VNVwDsTCuUnKvOxYTBtLz7+F8fI=; b=gbSiG6V8wd8tAb+DijDe6ZurzcOOLbLOcG68VjApf7xOO0GLc173XaPJjkXdlgpgvP tcAk6RFQNG5KKOev4xS1ZFY9wb0VuT0qbh79Z2VcicOhDeRJTvuVqtYxYzhLWiFUUmuy 4Rigx5MIYmIgtieTeXN/5smaR25lPv5As2aU2ZoGS7IAswacFgK90CuoXq7m6hdC5/nO ni8fx3PNto/JwUC1Kg2pWS/e2oKySoVaF5W2Mt8SCSdJYnDHGpkBIpt3na34LDrYNW/y kIdiEMalbuDOO5cBWMzyOupZTWCgErWqvAqtHtsNzoN4hnm8pnd6AngzQvM+JCCb6JI0 3Fsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784574792; x=1785179592; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=ke8s+Qt7PFdWHYg6VNVwDsTCuUnKvOxYTBtLz7+F8fI=; b=Fxs5oX5qspTJultceI+FI9KcinWfwRf9vvgtE5t02S+lVXSsC25S1dZSYn0lVMIZlE 3UM2RnSfRnS6rBG5d+AFz/1Rn9tMVuei/Pv8RIro2eo7qbiaszPWVh+jE6LcrlVQEOTC B/DWcEQfJygMttwIiJWUxD7IhknD9Uo0kz4Efu9pmiU4U4GwUukty3Xjob3SsPHREixp GPGaH4FOmXtpE4cYOQl5G19XnwLunu1F7Atwe32kdeV5B+k4hZ1M8QKWVdvCXt9EkQgE lSaxVbqoJbaeungsGRI5meR7tAI/JWE+oo2dbt0xka8XNi2zTQZWV6ygm0J9T3AyVPHU x4Hg== X-Gm-Message-State: AOJu0Yx1+oUBWXzDuBwroLfmk+os/FRcEnD1saYGvM/rf69251xAD0Ah 5hdqYNl2lvZofmKFxRkHLd5bLEXEBNKOy56CRZ7Q9yoRPx+ODLzllOmk/BCNFUoWuL/8 X-Gm-Gg: AfdE7cnmbpUI6SmmIj6lHZ0RpTQT2rEzy/FR1Zu1SCzUnavCWelOBSFDsEQYQJFIIYA gq8+X24FMN6DuQunn3lKnVGNHAeM+yr3PvnHoR/M7VklJmvMKL39eJk2lLH99UMZthB1dXnHdjE nfSAO/5AGMin278kGk7tiNm8z8Q2W1zHP44POXlYxc1OXv87ykYpc0Kg1tXXJTQURuyV8VDsnAp v8I1zUoFd7JpznzzEcMsQBxhjOkGsWdzxchoJAQ1eewhI/Hk8t3pFH2d987EYC7ts8eJI7OqGpG QHYwMMDph3yKCWWqcpq7NeAHKflP40s2YpY8nMmJ63bE+VF/xRGN9krfLDE659nE6GVi39+9NMa DjnuQWsCYIFjrJ1c4jsZrVbPmMXAbIJQoZWm5AIAGL4iltYuCjkVM28EBlrzfa2+VHAOE8mGwCH XcYjh0Fp0IvHrYqGdtcW6BgxHzBK9og7oTnzbK15gTLeaStaapLBJYgoen/uEFgdGGpF8x2jH15 d96YduI7jtdyCJnEdo= X-Received: by 2002:a05:600c:c10c:b0:495:56d4:3077 with SMTP id 5b1f17b1804b1-49556d432b4mr120347025e9.17.1784574792592; Mon, 20 Jul 2026 12:13:12 -0700 (PDT) Received: from yjn-Zenbook-UX3404VA-UX3404VA.. (p200300dcbf448c00c9c364f8ed993120.dip0.t-ipconnect.de. [2003:dc:bf44:8c00:c9c3:64f8:ed99:3120]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4956547323bsm8754025e9.4.2026.07.20.12.13.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Jul 2026 12:13:12 -0700 (PDT) From: York Jasper Niebuhr To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kees@kernel.org, franzen@sec.in.tum.de, ardb@kernel.org Subject: [RFC v3 3/5] SPSLR build integration Date: Mon, 20 Jul 2026 21:13:11 +0200 Message-ID: <20260720191311.21619-1-yjn@yjn-systems.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260720191146.21473-1-yjn@yjn-systems.com> References: <20260720191146.21473-1-yjn@yjn-systems.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Signed-off-by: York Jasper Niebuhr --- arch/x86/boot/startup/Makefile | 4 ++++ arch/x86/entry/vdso/common/Makefile.include | 2 +- arch/x86/kernel/vmlinux.lds.S | 23 +++++++++++++++++++ init/Kconfig | 13 +++++++++++ scripts/module.lds.S | 25 +++++++++++++++++++++ 5 files changed, 66 insertions(+), 1 deletion(-) diff --git a/arch/x86/boot/startup/Makefile b/arch/x86/boot/startup/Makefile index 5e499cfb29b5..1fa601781546 100644 --- a/arch/x86/boot/startup/Makefile +++ b/arch/x86/boot/startup/Makefile @@ -12,6 +12,10 @@ KBUILD_CFLAGS +=3D -D__DISABLE_EXPORTS -mcmodel=3Dsmall= -fPIC \ # disable ftrace hooks and LTO KBUILD_CFLAGS :=3D $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) KBUILD_CFLAGS :=3D $(filter-out $(CC_FLAGS_LTO),$(KBUILD_CFLAGS)) + +# Startup code executes before Bootpatch-SLR applies its runtime patches. +KBUILD_CFLAGS :=3D $(filter-out $(PINPOINT_PLUGIN_CFLAGS),$(KBUILD_CFLAGS)) + KASAN_SANITIZE :=3D n KCSAN_SANITIZE :=3D n KMSAN_SANITIZE :=3D n diff --git a/arch/x86/entry/vdso/common/Makefile.include b/arch/x86/entry/v= dso/common/Makefile.include index 687b3d89b40d..d8cc53cd9560 100644 --- a/arch/x86/entry/vdso/common/Makefile.include +++ b/arch/x86/entry/vdso/common/Makefile.include @@ -27,7 +27,7 @@ flags-remove-y +=3D \ -mfentry -pg \ $(RANDSTRUCT_CFLAGS) $(GCC_PLUGINS_CFLAGS) $(KSTACK_ERASE_CFLAGS) \ $(RETPOLINE_CFLAGS) $(CC_FLAGS_LTO) $(CC_FLAGS_CFI) \ - $(PADDING_CFLAGS) + $(PADDING_CFLAGS) $(PINPOINT_PLUGIN_CFLAGS) =20 # # Don't omit frame pointers for ease of userspace debugging, but do diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S index 74e336d7f9dd..aa71c2c2a756 100644 --- a/arch/x86/kernel/vmlinux.lds.S +++ b/arch/x86/kernel/vmlinux.lds.S @@ -200,6 +200,29 @@ SECTIONS /* rarely changed data like cpu maps */ READ_MOSTLY_DATA(INTERNODE_CACHE_BYTES) =20 +#ifdef CONFIG_SPSLR + __spslr_start =3D .; + + . =3D ALIGN(8); + __start_spslr_units =3D .; + KEEP(*(spslr_units)) + __stop_spslr_units =3D .; + + . =3D ALIGN(8); + __start_spslr_targets =3D .; + KEEP(*(spslr_targets)) + __stop_spslr_targets =3D .; + + . =3D ALIGN(8); + KEEP(*(spslr_target_layouts)) + KEEP(*(spslr_cu_target_refs)) + KEEP(*(spslr_ipins)) + KEEP(*(spslr_dpins)) + KEEP(*(spslr_strtab)) + + __spslr_end =3D .; +#endif + /* End of data section */ _edata =3D .; } :data diff --git a/init/Kconfig b/init/Kconfig index 5230d4879b1c..2110aff8a5c8 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -2316,3 +2316,16 @@ config ARCH_HAS_SYNC_CORE_BEFORE_USERMODE # . config ARCH_HAS_SYSCALL_WRAPPER def_bool n + +config SPSLR + bool "Selfpatch SLR prototype" + depends on X86_64 + depends on CC_IS_GCC + help + Experimental structure layout randomization prototype. + +config SANEMAKER + bool "Selfpatch SLR validation tooling" + depends on SPSLR + help + Experimental validation tooling for Selfpatch-SLR. diff --git a/scripts/module.lds.S b/scripts/module.lds.S index b62683061d79..e366be317115 100644 --- a/scripts/module.lds.S +++ b/scripts/module.lds.S @@ -62,6 +62,31 @@ SECTIONS { } =20 MOD_SEPARATE_CODETAG_SECTIONS() + +#ifdef CONFIG_SPSLR + .spslr : ALIGN(8) { + __spslr_start =3D .; + + . =3D ALIGN(8); + __start_spslr_units =3D .; + KEEP(*(spslr_units)) + __stop_spslr_units =3D .; + + . =3D ALIGN(8); + __start_spslr_targets =3D .; + KEEP(*(spslr_targets)) + __stop_spslr_targets =3D .; + + . =3D ALIGN(8); + KEEP(*(spslr_target_layouts)) + KEEP(*(spslr_cu_target_refs)) + KEEP(*(spslr_ipins)) + KEEP(*(spslr_dpins)) + KEEP(*(spslr_strtab)) + + __spslr_end =3D .; + } +#endif } =20 /* bring in arch-specific sections */ --=20 2.43.0 From nobody Sat Jul 25 01:54:02 2026 Received: from mail-wr1-f50.google.com (mail-wr1-f50.google.com [209.85.221.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8248D390CBF for ; Mon, 20 Jul 2026 19:13:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784574806; cv=none; b=G7KDTml56eM0yNEieBD3SfjT7URNjF9Uh8TPtsa1dpk/ihU/HRZsPeRTCe+e73nMKX6wMBeV4bXpuXyqwlON5InEa1cJ7E0rmV9zEvvYgfFIZQlLErZ8Mv49KpKg/3HmxIHKqcl3gqi/WxHdlDHIxISrOvAe43gB5lnwj4lmJZw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784574806; c=relaxed/simple; bh=2lK6gRroBoe4+bclCz1mfwWeoTeSwGscfR9tvghCLgQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=naQOejcpMOVKFEkqWYmhUU476+MRjln2Yk44IHWxq/vi8efZXmt5PU6Zh4tELEiJdsFXQltyIAMEi9fRe4kKBVeEcJi+cfHZL8xZvCB4BPbnBY5yzmiyDarJPH8BtQvABZ6b9Rp5R3FoAeuZaSMA2QgjBOxcPBmhqKbIxog41DI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=yjn-systems.com; spf=none smtp.mailfrom=yjn-systems.com; dkim=pass (2048-bit key) header.d=yjn-systems.com header.i=@yjn-systems.com header.b=lL2WFGIe; arc=none smtp.client-ip=209.85.221.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=yjn-systems.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=yjn-systems.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=yjn-systems.com header.i=@yjn-systems.com header.b="lL2WFGIe" Received: by mail-wr1-f50.google.com with SMTP id ffacd0b85a97d-47f7854678bso415512f8f.3 for ; Mon, 20 Jul 2026 12:13:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yjn-systems.com; s=google; t=1784574803; x=1785179603; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=mNy8FKI1rMtbV3QeF+PsHnoYXayigsA5C1ZOhTHQQls=; b=lL2WFGIel0zuS4G4SoGMeOu2nTUAQ7N7sNnrRZOnKFbBkpvHfVUaY6kC80GOaLhxxT iCZGcx4D+GE5eDZ6Tlb9lDUCKHLq8r5BrKTxQiVbtA7JPPiStJXZ6k/gG11Ud/K7SNVC v0zY5ctoAf5Jv/omfdAY3mrZu0yTEJHtdG1puzI5Ys8xbmhnQ5G+NCS6ULSol00JDnNw lkr1PDlT3gDBn29omqodJUzjKWWuhkUE7XOh9egJYZhTBwSRwz8IX9OkNp315Eg7NrVI n+x/adjQKqZt2Xj3cN2MpF+1gtgg3f5B7zSkqn0SodG3BBOsvZzSccdmg6ocWcStec4E zw/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784574803; x=1785179603; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=mNy8FKI1rMtbV3QeF+PsHnoYXayigsA5C1ZOhTHQQls=; b=cWpCv9tTFfDdPNITijpj1kpClQt5T+m0S1hVznnENZNpp/nSGofC0ZTf8WESlpFmv1 YIL5G4t0WBxhPgb4Tnz2XE8H8MmE/XP1KvruJSXvDP+PE6jsA6V58I5MC9Wz+dfpBAzd j9iJFj6u6TvFEnuJWyFobnQwgHYdiLtywLobhADOyRwAqHKrjazQJXThtZSxatFn7nWQ ZcFCIEdXs64ZN7oKFbp0uoE0twTt+iEfon8i4VyUb5eMBL25ixvpw/KI4AtNRht2Litq qBIJHl/pHFMTqiVtKy0djXiA3HfJOr0UxgTA991ZyJwQQHyVEIegb3zmqDFeyIG3MlUq b6iw== X-Gm-Message-State: AOJu0Yxi4eMs3IIkIsXJsYU+A2z3mq2MbVBcvUMKaroI18UU4oDaCzPH w0OzX7mVAiNOdimB5a0VMw4FA9RqzRsiCQI3TGNWTtcJ8P1AiZcapMPtbQ7ex1oRmRfz X-Gm-Gg: AR+sD11OSMPYdLAlofKUhSDylIENePLiHg07qQ5saMsBDw6QuKbEaGXIeF/zUaKtaEM v5ORJ1xc9i4qQH6Aljm0NxjCa6TWuibo66FGoV0Q0+7/rpHhPOIUtFFz4d2+ZkrWc0/7XUXse26 ls4DdpDx5ZqSVCnSRN/MJ4s1t4UgrWL4U9nX3I7+OGuAQ7eBZ44qPsAkh4+z5o+3oB1pUlZOlD9 ua4IXQyXeojM5uW9kmqc8yb8MeJ244zOaZifRMs8T3wqVZ6E/n0O6voga+rEHOLXx6yo5ojFcNo 8ojFUYbUknEPLfZRm1Gk7yw+1dapujwJLmNuG6Rqa21bWByKQ+wkgkXOyCmIS2+2YAoMdwBz34b hTy76iqKiNyvnmcyV34XCkwk2gAxHffQofSlEQ3qRKm7FsLLcR9PupPIdX5eIfcyOj2opTUnvcz sXLAIBOjV3glwj820RybVRgCr602arHcOnT3tcC/PEkNm4htYVq53XsVP3nk6X0MvOip1jO/gJ9 dB3NUKw+c7KA1liLl0= X-Received: by 2002:a5d:6f04:0:b0:47f:4508:585c with SMTP id ffacd0b85a97d-47f62306d29mr17874696f8f.11.1784574802823; Mon, 20 Jul 2026 12:13:22 -0700 (PDT) Received: from yjn-Zenbook-UX3404VA-UX3404VA.. (p200300dcbf448c00c9c364f8ed993120.dip0.t-ipconnect.de. [2003:dc:bf44:8c00:c9c3:64f8:ed99:3120]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-47f63e65644sm30747590f8f.16.2026.07.20.12.13.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Jul 2026 12:13:22 -0700 (PDT) From: York Jasper Niebuhr To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kees@kernel.org, franzen@sec.in.tum.de, ardb@kernel.org Subject: [RFC v3 4/5] SPSLR and Sanemaker source integration Date: Mon, 20 Jul 2026 21:13:21 +0200 Message-ID: <20260720191321.21635-1-yjn@yjn-systems.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260720191146.21473-1-yjn@yjn-systems.com> References: <20260720191146.21473-1-yjn@yjn-systems.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Signed-off-by: York Jasper Niebuhr --- include/linux/compiler_types.h | 8 ++ include/linux/sched.h | 58 ++++++------ init/main.c | 35 +++++++ kernel/fork.c | 10 +- kernel/module/main.c | 167 +++++++++++++++++++++++++++++++++ 5 files changed, 250 insertions(+), 28 deletions(-) diff --git a/include/linux/compiler_types.h b/include/linux/compiler_types.h index c5921f139007..3ef553fb2489 100644 --- a/include/linux/compiler_types.h +++ b/include/linux/compiler_types.h @@ -465,6 +465,14 @@ struct ftrace_likely_data { # define __latent_entropy #endif =20 +#if defined(__SPSLR__) +# define __spslr __attribute__((spslr)) +# define __spslr_field_fixed __attribute__((spslr_field_fixed)) +#else +# define __spslr +# define __spslr_field_fixed +#endif + #if defined(RANDSTRUCT) && !defined(__CHECKER__) # define __randomize_layout __designated_init __attribute__((randomize_lay= out)) # define __no_randomize_layout __attribute__((no_randomize_layout)) diff --git a/include/linux/sched.h b/include/linux/sched.h index 373bcc0598d1..8e6a87988d07 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -829,12 +829,12 @@ struct task_struct { * For reasons of header soup (see current_thread_info()), this * must be the first element of task_struct. */ - struct thread_info thread_info; + struct thread_info thread_info __spslr_field_fixed; #endif - unsigned int __state; + unsigned int __state __spslr_field_fixed; =20 /* saved state for "spinlock sleepers" */ - unsigned int saved_state; + unsigned int saved_state __spslr_field_fixed; =20 /* * This begins the randomizable portion of task_struct. Only @@ -877,12 +877,12 @@ struct task_struct { int normal_prio; unsigned int rt_priority; =20 - struct sched_entity se; - struct sched_rt_entity rt; + struct sched_entity se __spslr_field_fixed; + struct sched_rt_entity rt __spslr_field_fixed; struct sched_dl_entity dl; struct sched_dl_entity *dl_server; #ifdef CONFIG_SCHED_CLASS_EXT - struct sched_ext_entity scx; + struct sched_ext_entity scx __spslr_field_fixed; #endif const struct sched_class *sched_class; =20 @@ -919,7 +919,7 @@ struct task_struct { =20 #ifdef CONFIG_PREEMPT_NOTIFIERS /* List of struct preempt_notifier: */ - struct hlist_head preempt_notifiers; + struct hlist_head preempt_notifiers __spslr_field_fixed; #endif =20 #ifdef CONFIG_BLK_DEV_IO_TRACE @@ -931,15 +931,15 @@ struct task_struct { int nr_cpus_allowed; const cpumask_t *cpus_ptr; cpumask_t *user_cpus_ptr; - cpumask_t cpus_mask; + cpumask_t cpus_mask __spslr_field_fixed; void *migration_pending; unsigned short migration_disabled; unsigned short migration_flags; =20 #ifdef CONFIG_PREEMPT_RCU - int rcu_read_lock_nesting; - union rcu_special rcu_read_unlock_special; - struct list_head rcu_node_entry; + int rcu_read_lock_nesting __spslr_field_fixed; + union rcu_special rcu_read_unlock_special __spslr_field_fixed; + struct list_head rcu_node_entry __spslr_field_fixed; struct rcu_node *rcu_blocked_node; #endif /* #ifdef CONFIG_PREEMPT_RCU */ =20 @@ -948,9 +948,9 @@ struct task_struct { u8 rcu_tasks_holdout; u8 rcu_tasks_idx; int rcu_tasks_idle_cpu; - struct list_head rcu_tasks_holdout_list; + struct list_head rcu_tasks_holdout_list __spslr_field_fixed; int rcu_tasks_exit_cpu; - struct list_head rcu_tasks_exit_list; + struct list_head rcu_tasks_exit_list __spslr_field_fixed; #endif /* #ifdef CONFIG_TASKS_RCU */ =20 #ifdef CONFIG_TASKS_TRACE_RCU @@ -964,8 +964,8 @@ struct task_struct { =20 struct sched_info sched_info; =20 - struct list_head tasks; - struct plist_node pushable_tasks; + struct list_head tasks __spslr_field_fixed; + struct plist_node pushable_tasks __spslr_field_fixed; struct rb_node pushable_dl_tasks; =20 struct mm_struct *mm; @@ -1072,8 +1072,12 @@ struct task_struct { pid_t tgid; =20 #ifdef CONFIG_STACKPROTECTOR + /* Canary can not be randomized because of arch/x86/kernel/asm-offsets.c + * Pinpoint plugin could recognize context of instrumented accesses + * and e.g. hijack asm instructions that want to use them as constants. + */ /* Canary value for the -fstack-protector GCC feature: */ - unsigned long stack_canary; + unsigned long stack_canary __spslr_field_fixed; #endif /* * Pointers to the (original) parent process, youngest child, younger sib= ling, @@ -1090,8 +1094,8 @@ struct task_struct { /* * Children/sibling form the list of natural children: */ - struct list_head children; - struct list_head sibling; + struct list_head children __spslr_field_fixed; + struct list_head sibling __spslr_field_fixed; struct task_struct *group_leader; =20 /* @@ -1100,13 +1104,13 @@ struct task_struct { * This includes both natural children and PTRACE_ATTACH targets. * 'ptrace_entry' is this task's link on the p->parent->ptraced list. */ - struct list_head ptraced; - struct list_head ptrace_entry; + struct list_head ptraced __spslr_field_fixed; + struct list_head ptrace_entry __spslr_field_fixed; =20 /* PID/PID hash table linkage. */ struct pid *thread_pid; struct hlist_node pid_links[PIDTYPE_MAX]; - struct list_head thread_node; + struct list_head thread_node __spslr_field_fixed; =20 struct completion *vfork_done; =20 @@ -1211,7 +1215,7 @@ struct task_struct { sigset_t real_blocked; /* Restored if set_restore_sigmask() was used: */ sigset_t saved_sigmask; - struct sigpending pending; + struct sigpending pending __spslr_field_fixed; unsigned long sas_ss_sp; size_t sas_ss_size; unsigned int sas_ss_flags; @@ -1340,7 +1344,7 @@ struct task_struct { /* Control Group info protected by css_set_lock: */ struct css_set __rcu *cgroups; /* cg_list protected by css_set_lock and tsk->alloc_lock: */ - struct list_head cg_list; + struct list_head cg_list __spslr_field_fixed; #ifdef CONFIG_PREEMPT_RT struct llist_node cg_dead_lnode; #endif /* CONFIG_PREEMPT_RT */ @@ -1355,8 +1359,8 @@ struct task_struct { #ifdef CONFIG_PERF_EVENTS u8 perf_recursion[PERF_NR_CONTEXTS]; struct perf_event_context *perf_event_ctxp; - struct mutex perf_event_mutex; - struct list_head perf_event_list; + struct mutex perf_event_mutex __spslr_field_fixed; + struct list_head perf_event_list __spslr_field_fixed; struct perf_ctx_data __rcu *perf_ctx_data; #endif #ifdef CONFIG_DEBUG_PREEMPT @@ -1660,14 +1664,14 @@ struct task_struct { #endif =20 /* CPU-specific state of this task: */ - struct thread_struct thread; + struct thread_struct thread __spslr_field_fixed; =20 /* * New fields for task_struct should be added above here, so that * they are included in the randomized portion of task_struct. */ randomized_struct_fields_end -} __attribute__ ((aligned (64))); +} __spslr __attribute__ ((aligned (64))); =20 #ifdef CONFIG_SCHED_PROXY_EXEC DECLARE_STATIC_KEY_TRUE(__sched_proxy_exec); diff --git a/init/main.c b/init/main.c index e363232b428b..fbd2967d1530 100644 --- a/init/main.c +++ b/init/main.c @@ -119,6 +119,22 @@ =20 #include =20 +#include +#include + +#ifdef CONFIG_SPSLR + +bool spslr_enabled __ro_after_init =3D true; + +static int __init nospslr_setup(char *str) +{ + spslr_enabled =3D false; + return 0; +} +early_param("nospslr", nospslr_setup); + +#endif + static int kernel_init(void *); =20 /* @@ -974,6 +990,8 @@ void start_kernel(void) char *command_line; char *after_dashes; =20 + sanemaker_target_tag(&init_task, struct task_struct); + set_task_stack_end_magic(&init_task); smp_setup_processor_id(); debug_objects_early_init(); @@ -1023,6 +1041,23 @@ void start_kernel(void) /* Architectural and non-timekeeping rng init, before allocator init */ random_init_early(command_line); =20 +#ifdef CONFIG_SPSLR + if (spslr_enabled) { + /* Randomize structure layouts */ + struct spslr_status spslr_init_status =3D spslr_init(); + if (spslr_init_status.error !=3D SPSLR_OK) + panic("SPSLR initialization failed"); + + struct spslr_status spslr_selfpatch_status =3D spslr_selfpatch(); + if (spslr_selfpatch_status.error !=3D SPSLR_OK) + panic("SPSLR selfpatch failed"); + + pr_notice("Successfully applied SPSLR\n"); + } else { + pr_notice("SPSLR disabled\n"); + } +#endif + /* * These use large bootmem allocations and must precede * initalization of page allocator diff --git a/kernel/fork.c b/kernel/fork.c index f0e2e131a9a5..f07f89d8c55a 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -127,6 +127,9 @@ =20 #include =20 +#include +#include + /* * Minimum number of threads to boot the kernel */ @@ -185,11 +188,16 @@ static struct kmem_cache *task_struct_cachep; =20 static inline struct task_struct *alloc_task_struct_node(int node) { - return kmem_cache_alloc_node(task_struct_cachep, GFP_KERNEL, node); + struct task_struct *tsk =3D + kmem_cache_alloc_node(task_struct_cachep, GFP_KERNEL, node); + + sanemaker_target_tag(tsk, struct task_struct); + return tsk; } =20 static inline void free_task_struct(struct task_struct *tsk) { + sanemaker_target_untag(tsk); kmem_cache_free(task_struct_cachep, tsk); } =20 diff --git a/kernel/module/main.c b/kernel/module/main.c index 46dd8d25a605..1f314fc12930 100644 --- a/kernel/module/main.c +++ b/kernel/module/main.c @@ -66,6 +66,70 @@ #define CREATE_TRACE_POINTS #include =20 +#include +#include + +/* Sanemaker image (de)registration helpers */ + +static const void *sanemaker_module_image_base(const struct module *mod) +{ + unsigned long base =3D ULONG_MAX; + + if (mod->mem[MOD_TEXT].base && mod->mem[MOD_TEXT].size) + base =3D min(base, + (unsigned long)mod->mem[MOD_TEXT].base); + + if (mod->mem[MOD_INIT_TEXT].base && mod->mem[MOD_INIT_TEXT].size) + base =3D min(base, + (unsigned long)mod->mem[MOD_INIT_TEXT].base); + + return base =3D=3D ULONG_MAX ? NULL : (const void *)base; +} + +static void sanemaker_register_module_image(struct module *mod) +{ + const struct module_memory *text; + const void *image_base; + + image_base =3D sanemaker_module_image_base(mod); + if (!image_base) + return; + + sanemaker_new_image(mod->name, image_base); + + text =3D &mod->mem[MOD_TEXT]; + if (text->base && text->size) + sanemaker_new_image_text( + mod->name, + text->base, + (const char *)text->base + text->size); + + text =3D &mod->mem[MOD_INIT_TEXT]; + if (text->base && text->size) + sanemaker_new_image_text( + mod->name, + text->base, + (const char *)text->base + text->size); +} + +static void sanemaker_drop_module_init_text(struct module *mod) +{ + const struct module_memory *text =3D &mod->mem[MOD_INIT_TEXT]; + + if (!text->base || !text->size) + return; + + sanemaker_drop_image_text( + mod->name, + text->base, + (const char *)text->base + text->size); +} + +static void sanemaker_unregister_module_image(struct module *mod) +{ + sanemaker_drop_image(mod->name); +} + /* * Mutex protects: * 1) List of modules (also safely readable within RCU read section), @@ -1461,6 +1525,7 @@ static void free_module(struct module *mod) kfree(mod->args); percpu_modfree(mod); =20 + sanemaker_unregister_module_image(mod); free_mod_mem(mod); } =20 @@ -3100,10 +3165,22 @@ static noinline int do_init_module(struct module *m= od) freeinit->init_data =3D mod->mem[MOD_INIT_DATA].base; freeinit->init_rodata =3D mod->mem[MOD_INIT_RODATA].base; =20 + /* + * Constructors and mod->init are the first entry points into module text. + */ + sanemaker_register_module_image(mod); + do_mod_ctors(mod); /* Start the module */ if (mod->init !=3D NULL) ret =3D do_one_initcall(mod->init); + + /* + * mod->init() has returned, so no further execution should enter + * MOD_INIT_TEXT. This is independent of when the allocation is freed. + */ + sanemaker_drop_module_init_text(mod); + if (ret < 0) { /* * -EEXIST is reserved by [f]init_module() to signal to userspace that @@ -3415,6 +3492,87 @@ static int early_mod_check(struct load_info *info, i= nt flags) return err; } =20 +#ifdef CONFIG_SPSLR + +/* Find spslr symbol in module without kallsym */ +static unsigned long spslr_find_module_symbol(const struct load_info *info, + const char *name) +{ + Elf_Shdr *symsec =3D &info->sechdrs[info->index.sym]; + Elf_Sym *sym =3D (void *)symsec->sh_addr; + unsigned int i, n =3D symsec->sh_size / sizeof(*sym); + + for (i =3D 1; i < n; i++) { + const char *symname =3D info->strtab + sym[i].st_name; + + if (strcmp(symname, name) !=3D 0) + continue; + + /* Ignore undefined symbols just in case. */ + if (sym[i].st_shndx =3D=3D SHN_UNDEF) + return 0; + + return (unsigned long)sym[i].st_value; + } + + return 0; +} + +/* Apply structure layout randomization to module */ +static int __maybe_unused spslr_prepare_module(struct module *mod, + const struct load_info *info) +{ + struct spslr_ctx ctx =3D { }; + struct spslr_status st; + unsigned long workspace_size; + int err =3D 0; + + ctx.entry.start_units =3D (const void *)spslr_find_module_symbol(info, + __stringify(SPSLR_START_UNITS_SYM)); + if (!ctx.entry.start_units) { + pr_err("%s: SPSLR units start symbol missing\n", mod->name); + return -ENOEXEC; + } + + ctx.entry.stop_units =3D (const void *)spslr_find_module_symbol(info, + __stringify(SPSLR_STOP_UNITS_SYM)); + if (!ctx.entry.stop_units) { + pr_err("%s: SPSLR units stop symbol missing\n", mod->name); + return -ENOEXEC; + } + + ctx.entry.start_targets =3D (const void *)spslr_find_module_symbol(info, + __stringify(SPSLR_START_TARGETS_SYM)); + if (!ctx.entry.start_targets) { + pr_err("%s: SPSLR targets start symbol missing\n", mod->name); + return -ENOEXEC; + } + + ctx.entry.stop_targets =3D (const void *)spslr_find_module_symbol(info, + __stringify(SPSLR_STOP_TARGETS_SYM)); + if (!ctx.entry.stop_targets) { + pr_err("%s: SPSLR targets stop symbol missing\n", mod->name); + return -ENOEXEC; + } + + workspace_size =3D spslr_workspace_size(&ctx.entry); + ctx.workspace =3D kvmalloc(workspace_size, GFP_KERNEL); + if (!ctx.workspace) + return -ENOMEM; + + st =3D spslr_patch_module(&ctx); + if (st.viability !=3D SPSLR_VIABLE || st.error !=3D SPSLR_OK) { + pr_err("%s: SPSLR patch failed: viability=3D%d error=3D%d\n", + mod->name, st.viability, st.error); + err =3D -ENOEXEC; + } + + kvfree(ctx.workspace); + return err; +} + +#endif /* CONFIG_SPSLR */ + /* * Allocate and load the module: note that size of section 0 is always * zero, and we rely on this for optional sections. @@ -3520,6 +3678,15 @@ static int load_module(struct load_info *info, const= char __user *uargs, if (err < 0) goto free_modinfo; =20 +#ifdef CONFIG_SPSLR + if (spslr_enabled) { + /* SPSLR must happen after relocation and icache should be flushed after= wards */ + err =3D spslr_prepare_module(mod, info); + if (err < 0) + goto free_modinfo; + } +#endif + flush_module_icache(mod); =20 /* Now copy in args */ --=20 2.43.0 From nobody Sat Jul 25 01:54:02 2026 Received: from mail-wr1-f42.google.com (mail-wr1-f42.google.com [209.85.221.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4BB4D3A3E6F for ; Mon, 20 Jul 2026 19:13:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784574817; cv=none; b=RqNbtFlC6+gRcd/KCmrLdoon5F3oksqtkMC5GR442rGHNZRGhPt+yh+wixp2G/EZD3bU/Bz7HxJ2jlUkNMW8Wd0xskOanD7PUgB6W/74MSvlNNsoJOmxMQt00/b8CUvAhbtV/fFBNsCneGBlcZFtONHP6NrgV7P42nDRTLVN7tw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784574817; c=relaxed/simple; bh=VfOO6+7jMgNXfdsgzDu2gHlwmZ6txpGn5eOFifRinEY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ss+sTeeAiqzkZ1HHoPVghliCqlc56LymQf6yjdJcR7MMn0FjpxALXLEyLPO2a+5Z4GlfIadl3bDnmWbuAML7ZdFkVYrFX/RWzqxrdMMtzl7yMe9eFrSf5KiySIY2RRVHVMsi8DHuaF+69CPPzlH27q9aFU9DthzvDavg51ZikqQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=yjn-systems.com; spf=none smtp.mailfrom=yjn-systems.com; dkim=pass (2048-bit key) header.d=yjn-systems.com header.i=@yjn-systems.com header.b=K4LGom7J; arc=none smtp.client-ip=209.85.221.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=yjn-systems.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=yjn-systems.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=yjn-systems.com header.i=@yjn-systems.com header.b="K4LGom7J" Received: by mail-wr1-f42.google.com with SMTP id ffacd0b85a97d-47c6e9a694bso5845043f8f.1 for ; Mon, 20 Jul 2026 12:13:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yjn-systems.com; s=google; t=1784574814; x=1785179614; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=0HKAwADsjpeI6vXR3CdbOrXgFYwbzM256tbPggd8eZ0=; b=K4LGom7JW7Yrlmhsksg0IHpw8G1Qvrd82rIq8mU3UoqlkBj6SelZjnRKjVwZyWoxhP H9hVzhCfRcffORXyGc7I7zSWevV4uw9cXjC8Y3+7MeH7uTZIqsmYg23qZV429Ov9CYR9 a5me4lO89Wpu8zU5AVvCFpkmG31pAkZI9dnpLrjhauPd/qgUcL9GyoiUOQxRguMq4biZ ExBFV0tV/SqwurzfuxGzU8ZYHhIwTdOoCKGxVtN2AX2+2e7khdhsviRndWQIUgYgKCOn NztzP2zbnY0+YwSrBoxLXGrBztckWLDbcm/wg3UclZZ1ZZ1BTqqP4eOfrbriYu/GnMWk MF2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784574814; x=1785179614; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=0HKAwADsjpeI6vXR3CdbOrXgFYwbzM256tbPggd8eZ0=; b=lLMiI5b5Ze2LjLQW/Cto0APzle+3wuc6BhdCG7PqTJp5KJgdewX53Sj5nfhPXQYdpO TIKLYTMWfYFV1NTUDUX7s8cehG5f4xo0lToD5GKOQELRPB8fjpGdUwVKi/YmPABje3KI tkVV+8XWrVaEGjfahm0jDefTYegOoIB/uHmHqED7/GQM/HzahfiSx3yi3Q1T2VD4BE1y x8PseO20vXY/gBRUbNXHZTJcF4A2RGLdKh7oY9eOnwvS3jRIRFhgRCr/+cUz+tyVoDqz c8CnDQzBnbktzFPXeMSlrFQvccshs3Afc2Cnh2KuTz7ySJ+WPmzT0i3X2T2sYA+ETqzO qC7Q== X-Gm-Message-State: AOJu0YzF3qApF6PKXf9HKDc8eIIxEmfOTwTt/gBDlj+EaVA6c8OFTkEX 5USMf2XybighRuvQeRLf/z6VdSQRmcLLfg15Y9X8qIJBTk8NTfeZnqDpXhjFAenQWgbwaZXlsSH EyR3gMA== X-Gm-Gg: AR+sD12JYApnBwQvJgZ3ET+mU2yeVDHtE0/hYKPvLvZrDu5JSQvIZEtDPwAVdpzcRdp I39uPZcXWH1dtTLDyZs8bQ81AH228VTqmmlPoN1Ru5BJue+GgPKjdD5O+C8IJsnLCif66+GXi04 XbSAVJDcCljSSL97yPMbSOy4u2CeuLYacuDxdOjD/2TsPewBn0uT6UbLN2A4sH5vYXR2qYZi+ip O5PVwCh95wjZ3P/AD4juHmDeEApnQud02CoC7QwnPamP4YH1YKHriVoS/NMtVDDZ/USZiEJnfYc zxfRfl1JvbhKCCb7jUGxIuxg9K+EtJYIXHRD1S8fsirGqasTBwWwsXw8s2g1sF6eQ9oL058wFyZ dc6geCuh3YxXE3pi1gdTWykCJGu3PfdGk3FE2QoLMvGphosUfJPsCi/a0aNWPnJfXnkjTyRCggL kC4Id1WEl/DtxA0uNyUp5aoGZqceBR3E4POsBsTA6KiL2AoKFtJE0Do9F438kLwRTO38s0y887A YAcnsn6ZlvubgM04a8= X-Received: by 2002:a05:6000:1ac9:b0:47f:4ac9:98bc with SMTP id ffacd0b85a97d-47f6230dc51mr19072366f8f.24.1784574814520; Mon, 20 Jul 2026 12:13:34 -0700 (PDT) Received: from yjn-Zenbook-UX3404VA-UX3404VA.. (p200300dcbf448c00c9c364f8ed993120.dip0.t-ipconnect.de. [2003:dc:bf44:8c00:c9c3:64f8:ed99:3120]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-47f63ec7ea1sm33396206f8f.24.2026.07.20.12.13.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Jul 2026 12:13:34 -0700 (PDT) From: York Jasper Niebuhr To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kees@kernel.org, franzen@sec.in.tum.de, ardb@kernel.org Subject: [RFC v3 5/5] Tasklist sample module Date: Mon, 20 Jul 2026 21:13:33 +0200 Message-ID: <20260720191333.21652-1-yjn@yjn-systems.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260720191146.21473-1-yjn@yjn-systems.com> References: <20260720191146.21473-1-yjn@yjn-systems.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Signed-off-by: York Jasper Niebuhr --- samples/Kconfig | 3 ++ samples/Makefile | 1 + samples/spslr/Kconfig | 17 ++++++++ samples/spslr/Makefile | 1 + samples/spslr/tasklist/Makefile | 1 + samples/spslr/tasklist/tasklist.c | 67 +++++++++++++++++++++++++++++++ 6 files changed, 90 insertions(+) create mode 100644 samples/spslr/Kconfig create mode 100644 samples/spslr/Makefile create mode 100644 samples/spslr/tasklist/Makefile create mode 100644 samples/spslr/tasklist/tasklist.c diff --git a/samples/Kconfig b/samples/Kconfig index a75e8e78330d..8925820617c9 100644 --- a/samples/Kconfig +++ b/samples/Kconfig @@ -326,6 +326,8 @@ source "samples/rust/Kconfig" =20 source "samples/damon/Kconfig" =20 +source "samples/spslr/Kconfig" + endif # SAMPLES =20 config HAVE_SAMPLE_FTRACE_DIRECT @@ -333,3 +335,4 @@ config HAVE_SAMPLE_FTRACE_DIRECT =20 config HAVE_SAMPLE_FTRACE_DIRECT_MULTI bool + diff --git a/samples/Makefile b/samples/Makefile index 07641e177bd8..1b727ceb2efa 100644 --- a/samples/Makefile +++ b/samples/Makefile @@ -45,3 +45,4 @@ obj-$(CONFIG_SAMPLE_DAMON_PRCL) +=3D damon/ obj-$(CONFIG_SAMPLE_DAMON_MTIER) +=3D damon/ obj-$(CONFIG_SAMPLE_HUNG_TASK) +=3D hung_task/ obj-$(CONFIG_SAMPLE_TSM_MR) +=3D tsm-mr/ +obj-$(CONFIG_SAMPLES_SPSLR) +=3D spslr/ diff --git a/samples/spslr/Kconfig b/samples/spslr/Kconfig new file mode 100644 index 000000000000..ba163ea15006 --- /dev/null +++ b/samples/spslr/Kconfig @@ -0,0 +1,17 @@ +# SPDX-License-Identifier: GPL-2.0 + +menuconfig SAMPLES_SPSLR + bool "SPSLR samples" + depends on SPSLR + help + You can build sample SPSLR kernel code here. + + If unsure, say N. + +if SAMPLES_SPSLR + +config SAMPLE_SPSLR_TASKLIST + tristate "SPSLR tasklist module example" + depends on m + +endif # SAMPLES_SPSLR diff --git a/samples/spslr/Makefile b/samples/spslr/Makefile new file mode 100644 index 000000000000..2d330b105c95 --- /dev/null +++ b/samples/spslr/Makefile @@ -0,0 +1 @@ +obj-$(CONFIG_SAMPLE_SPSLR_TASKLIST) +=3D tasklist/ diff --git a/samples/spslr/tasklist/Makefile b/samples/spslr/tasklist/Makef= ile new file mode 100644 index 000000000000..6a5bc5b29b34 --- /dev/null +++ b/samples/spslr/tasklist/Makefile @@ -0,0 +1 @@ +obj-m +=3D tasklist.o diff --git a/samples/spslr/tasklist/tasklist.c b/samples/spslr/tasklist/tas= klist.c new file mode 100644 index 000000000000..ea021742c8e9 --- /dev/null +++ b/samples/spslr/tasklist/tasklist.c @@ -0,0 +1,67 @@ +#include +#include +#include +#include +#include +#include +#include +#include + +static const struct task_struct module_target_data =3D { .flags =3D 42 }; + +static int __init taskinfo_init(void) +{ + struct task_struct *p; + + pr_info("taskinfo: loaded\n"); + pr_info(" offsetof(task_struct, pid)=3D%zu\n", offsetof(struct task= _struct, pid)); + pr_info(" offsetof(task_struct, tgid)=3D%zu\n", offsetof(struct tas= k_struct, tgid)); + pr_info(" offsetof(task_struct, cred)=3D%zu\n", offsetof(struct tas= k_struct, cred)); + pr_info(" offsetof(task_struct, real_parent)=3D%zu\n", offsetof(str= uct task_struct, real_parent)); + pr_info(" offsetof(task_struct, comm)=3D%zu\n", offsetof(struct tas= k_struct, comm)); + pr_info(" offsetof(task_struct, __state)=3D%zu\n", offsetof(struct = task_struct, __state)); + pr_info(" offsetof(task_struct, flags)=3D%zu\n", offsetof(struct ta= sk_struct, flags)); + pr_info(" offsetof(task_struct, prio)=3D%zu\n", offsetof(struct tas= k_struct, prio)); + pr_info(" offsetof(task_struct, policy)=3D%zu\n", offsetof(struct t= ask_struct, policy)); + + pr_info("datapin flags value is %u (should be 42)\n", module_target_da= ta.flags); + + pr_info("=3D=3D=3D Task List =3D=3D=3D\n"); + + rcu_read_lock(); + + for_each_process(p) { + const struct cred *cred; + struct task_struct *parent; + + cred =3D rcu_dereference(p->cred); + parent =3D rcu_dereference(p->real_parent); + + pr_info("task: pid=3D%d tgid=3D%d ppid=3D%d uid=3D%u gid=3D%u comm= =3D%s state=3D%u flags=3D0x%x prio=3D%d policy=3D%u\n", + p->pid, + p->tgid, + parent ? parent->tgid : -1, + __kuid_val(cred->uid), + __kgid_val(cred->gid), + p->comm, + READ_ONCE(p->__state), + p->flags, + p->prio, + p->policy); + } + + rcu_read_unlock(); + + return 0; +} + +static void __exit taskinfo_exit(void) +{ + pr_info("taskinfo: unloaded\n"); +} + +module_init(taskinfo_init); +module_exit(taskinfo_exit); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("List modules and tasks (task_struct stress)"); --=20 2.43.0