From nobody Fri Dec 19 08:02:03 2025 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E717118024 for ; Sat, 12 Apr 2025 01:10:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744420218; cv=none; b=XIO7jrKkjoPB2P0dBq7oWrFraYkroAHGH+STOkaQJjjZSJrpjh806Aptnbtgdm9PN7WpdHWaa/PTTMMpNpHFYC5t09ERJ+qYPLJRAmjMlH4XXWnDgT9vVMHNRbugowDxfWV2mMWJvNV8fqqpCnMW7SLlNMG9HQlhjLw8i+zOzGE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744420218; c=relaxed/simple; bh=C5rogbPNUI6pHPavbIdCjxlMXwhFBMWl+E2se2tXjDY=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=n18GZsITvNjQHrJ/YE/vEUlZtTOBWGc/Z/5Z/ITZ5Q1hVYKy3oDzYmqd03NgZz2zeG7NEPwJpI0suIeVDLUd8SfRRA+uzXlLgk4J84GpaRzj97qW9SD/LkcQ27pE24gUwsCHO6dOLp+i49R2JsAqrGglssNP1jzEPmq8xsLsK2o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Kl02uR/f; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Kl02uR/f" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-2242ade807fso39242355ad.2 for ; Fri, 11 Apr 2025 18:10:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1744420216; x=1745025016; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=bWDHpUBhtTe2FgO8fryMDFRusp4rjFHy5/tTaSl+b7o=; b=Kl02uR/fSXOpeM/M9QoOvuqCFZIjOLg0onSJGa2SBgqgtJc7yy3WwAKyHT+YaSsTRE HQhrrfHXGvbfBBXKoaqjaYpxaPf/ndxqmpWSs/VPOn5NdcxW7+Ui0AGXQkO/Vy4TWOAh d9xfjaw7kEnQwLil/pDnSnypGSp9+Mu3F3IDqbpcm/HFa2NZB8p9DIJyekl1hWveS0Um XMqoksvna7Y7vEtzZnSoFQ+kVKvzGeNmUKF2UUZTUUXQ+WcZk/FQeQQ30yMC5Rmvuzdf MfALmHDEbvRP4npQKfJSiN2KVP8MAM/EgYn2u6HU2WPQNxLjeu6bF4221OFQ3UpABkvP Vfag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744420216; x=1745025016; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=bWDHpUBhtTe2FgO8fryMDFRusp4rjFHy5/tTaSl+b7o=; b=gJm3T5x/BwzLhAfUCoBuVr0E084nLLzJBpwJbYBmwfzSdia6dkCteMOOSGSfE1/gh4 2SYtsbqEygTpHbR6J/Ba2eTeGuAzHlpM0gzOLTT1hdCMBqcLyPEyTjGYyK3hAqAZ+2Xd WTM9rGVMq664ZwJCGPw8i+qHzOtt+e6HpUOJgUE4l71BpZlKibpwLXFwKXl+q6qg8aQ8 6l1tRQD2xYwf87S8o9DEuJffHbcpiUNNmYmrLOcuBCIdtH4Eo5h8AXtc/9oLhviA58R9 fz4LrUuUW0zNMMOqGhivhWY3yfeizwWHK/DdYROHJe6nBA01SbHiOlDtUwL2/6TAup88 bFQg== X-Forwarded-Encrypted: i=1; AJvYcCWEYX8PuD0TS5MQh0UE09MYVTWv5JbQC2uMmP1nTqHh2+toldRgATx62dzLuFT0QCz3cJGcbRd0pJSmLEE=@vger.kernel.org X-Gm-Message-State: AOJu0YyTPFKH/eEtI5ATG/JGGhLT6hyX6BCAW2Q82XaB8sMNEEM43S38 iC4ntcmUONT2fTacQDVV4Yrrw9SNJ/10RY5FHfVsOMIK0V/1T99UJoyuT3SceH2LFWC73Vej/vA ph/aBg9f8/BxpSjsZLD1rIQ== X-Google-Smtp-Source: AGHT+IHVLYuzqpZRTCktPu5rFIJT1KD3DgXm1NNnlt3I7puZQhJBEjVGAwIu34AVxzFmVqO3ENhw/4jf261UUYOPSg== X-Received: from pltf12.prod.google.com ([2002:a17:902:74cc:b0:224:2ae9:b271]) (user=dylanbhatch job=prod-delivery.src-stubby-dispatcher) by 2002:a17:903:1aa5:b0:223:377f:9795 with SMTP id d9443c01a7336-22bea3de1a8mr72996465ad.0.1744420216287; Fri, 11 Apr 2025 18:10:16 -0700 (PDT) Date: Sat, 12 Apr 2025 01:09:38 +0000 In-Reply-To: <20250412010940.1686376-1-dylanbhatch@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250412010940.1686376-1-dylanbhatch@google.com> X-Mailer: git-send-email 2.49.0.604.gff1f9ca942-goog Message-ID: <20250412010940.1686376-2-dylanbhatch@google.com> Subject: [PATCH v2 1/2] arm64: patching: Rename aarch64_insn_copy to text_poke. From: Dylan Hatch To: Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Puranjay Mohan , Xu Kuohai , "=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?=" , Catalin Marinas , Will Deacon , "Mike Rapoport (Microsoft)" , Arnd Bergmann , Geert Uytterhoeven , Luis Chamberlain , Andrew Morton , Song Liu , Ard Biesheuvel , Mark Rutland Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Dylan Hatch , Roman Gushchin Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Match the name and signature of the equivalent in the x86 text-poke API. Making the src pointer const also allows this function to be interchangeable with memcpy(). Signed-off-by: Dylan Hatch Acked-by: Song Liu --- arch/arm64/include/asm/text-patching.h | 2 +- arch/arm64/kernel/patching.c | 12 ++++++------ arch/arm64/net/bpf_jit_comp.c | 2 +- 3 files changed, 8 insertions(+), 8 deletions(-) diff --git a/arch/arm64/include/asm/text-patching.h b/arch/arm64/include/as= m/text-patching.h index 587bdb91ab7a6..450d806d11109 100644 --- a/arch/arm64/include/asm/text-patching.h +++ b/arch/arm64/include/asm/text-patching.h @@ -9,7 +9,7 @@ int aarch64_insn_write(void *addr, u32 insn); =20 int aarch64_insn_write_literal_u64(void *addr, u64 val); void *aarch64_insn_set(void *dst, u32 insn, size_t len); -void *aarch64_insn_copy(void *dst, void *src, size_t len); +void *text_poke(void *dst, const void *src, size_t len); =20 int aarch64_insn_patch_text_nosync(void *addr, u32 insn); int aarch64_insn_patch_text(void *addrs[], u32 insns[], int cnt); diff --git a/arch/arm64/kernel/patching.c b/arch/arm64/kernel/patching.c index 1041bc67a3eee..e07dc32620053 100644 --- a/arch/arm64/kernel/patching.c +++ b/arch/arm64/kernel/patching.c @@ -102,9 +102,9 @@ noinstr int aarch64_insn_write_literal_u64(void *addr, = u64 val) return ret; } =20 -typedef void text_poke_f(void *dst, void *src, size_t patched, size_t len); +typedef void text_poke_f(void *dst, const void *src, size_t patched, size_= t len); =20 -static void *__text_poke(text_poke_f func, void *addr, void *src, size_t l= en) +static void *__text_poke(text_poke_f func, void *addr, const void *src, si= ze_t len) { unsigned long flags; size_t patched =3D 0; @@ -132,12 +132,12 @@ static void *__text_poke(text_poke_f func, void *addr= , void *src, size_t len) return addr; } =20 -static void text_poke_memcpy(void *dst, void *src, size_t patched, size_t = len) +static void text_poke_memcpy(void *dst, const void *src, size_t patched, s= ize_t len) { copy_to_kernel_nofault(dst, src + patched, len); } =20 -static void text_poke_memset(void *dst, void *src, size_t patched, size_t = len) +static void text_poke_memset(void *dst, const void *src, size_t patched, s= ize_t len) { u32 c =3D *(u32 *)src; =20 @@ -145,14 +145,14 @@ static void text_poke_memset(void *dst, void *src, si= ze_t patched, size_t len) } =20 /** - * aarch64_insn_copy - Copy instructions into (an unused part of) RX memory + * text_poke - Copy instructions into (an unused part of) RX memory * @dst: address to modify * @src: source of the copy * @len: length to copy * * Useful for JITs to dump new code blocks into unused regions of RX memor= y. */ -noinstr void *aarch64_insn_copy(void *dst, void *src, size_t len) +noinstr void *text_poke(void *dst, const void *src, size_t len) { /* A64 instructions must be word aligned */ if ((uintptr_t)dst & 0x3) diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c index 70d7c89d3ac90..b5be90edff410 100644 --- a/arch/arm64/net/bpf_jit_comp.c +++ b/arch/arm64/net/bpf_jit_comp.c @@ -2047,7 +2047,7 @@ bool bpf_jit_supports_kfunc_call(void) =20 void *bpf_arch_text_copy(void *dst, void *src, size_t len) { - if (!aarch64_insn_copy(dst, src, len)) + if (!text_poke(dst, src, len)) return ERR_PTR(-EINVAL); return dst; } --=20 2.49.0.604.gff1f9ca942-goog From nobody Fri Dec 19 08:02:03 2025 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 32ED22AF04 for ; Sat, 12 Apr 2025 01:10:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744420228; cv=none; b=ZAnJHVpenSJz/ac7Utb+UvLDpbfNel47XdRAWJmtWdVvQHD1dRWlcZtxdJ9ar16o2yZWq9AZsNm+ugpCdFRKkQjELI/X+z9kH51Q1G25BsMC+pi8BRwaPHsU2Wy8UMbFADdP8ieMbu+1zimlH83NzVrOwa3M+ynd6nJjs0yF1/g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744420228; c=relaxed/simple; bh=CLWex6gdQgCeQ+j1Ylxw5hYzn0j5IyoX3BXUq3gS9XE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=U2FG8pL+cWxi9zv95sJF3IZOxVpGf32r9Oq04eX70x1eaBv1kHcTKJ+R12qylihrENX1559iMgRbIrgI1FHs0aWks/DO8LcucSpGdMbmT4EuTqYtuTKZ1wGzAIxjzO9b4Mj4p1DdJF6EJEEhdlwJ8FMmiY0Xm2Q0gHtpLCVs4p0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=4LQsC2wO; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="4LQsC2wO" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-7395d07a3dcso1836109b3a.3 for ; Fri, 11 Apr 2025 18:10:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1744420225; x=1745025025; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=UPE/w7OMV+R/BpERehO0lW1KzawWpcv3Vb+OACxnfog=; b=4LQsC2wO14y7Bf3/7k10jDCvWN6yfo+i7WjI4SzM2LB80SHKwPUhfsPpg5UVXdJbur bXK84iOZp1bJE4jXeyQNGFrteSYnprNqWPpe/HZ+0JUtrP92Q3sCYb6QpZ0CGCxnFPPw McAIGpQA5NIS/6o0xb8Rm1fuBlf4bajIvBVluwejQnCurQQf03BKmpXxQGS+Vy2xR52E BP5W2dZuX+nYYPkiMWFByENc2pRVTkzyscx8yWgd7uapnk0MvqkPypRfDlo8tgojRQrN QPT51ouJjPFbbLvrrTmp2SJMVxr4NifJ4To9TfVixF5ervCuf7a7IJHDOWY2zZad5mWd rs5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744420225; x=1745025025; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=UPE/w7OMV+R/BpERehO0lW1KzawWpcv3Vb+OACxnfog=; b=SUKbaucg2z4XV67qzjeMzF9VJctP9vgkjA8OZ//flbrgln8b0+wesLYkap8WcoPsyy PN1e1JYNfjB3xl0JTrirKws1Eenuq7yLo7lhFuzWpHhlbwjgXwaJ7VNyoqtUQvKjAxhN RiQcdf8ySuBTnBmkVgpoCDGOaau6g2CX5heohSOtOL4/FzRRzxc0r81+Jvp572winF07 Z/mxiM+BNZRA0I9VoHIqtiM9rneFu7aj7qrlYXJZMEl106tTeZ4xrjfLgJhf03i2K9el B13n0VSEGpKCG9N7NxmFhWAy+1eU4Df2pY5MknsRJ3QidfkzCsjMedAbNCKOJAR9X9zw qrzg== X-Forwarded-Encrypted: i=1; AJvYcCX/7E+3azL8yOr2rODpp0vOKfERhxHOMGsK5GYtNESXpaCuVcu4Vj9/C348l9A+yv5TiYWyPXn55C1TyH0=@vger.kernel.org X-Gm-Message-State: AOJu0YwWDOgtvvx81CusTCEDEcwUX6VGNyIsISMs6ZucRs0vfz9C35By vMpnfVdvN7+RLtsSRR+yCdYbxh3mVMaO/+5Yi9mIN0miusrMBwn0ufyZXkr5am9ArAvV9mTGSVI MqRcC47elybYjtsTgNg+e0A== X-Google-Smtp-Source: AGHT+IGCCoJ+p4ug1h6JLtcNhK91NiKYTxwJFv7sDgsGsshWBvKN29loXFFysazsL5Y0Mr82eB8IfOe4BG34Ie+snA== X-Received: from pfgt19.prod.google.com ([2002:a05:6a00:1393:b0:739:56be:f58c]) (user=dylanbhatch job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:9f8f:b0:1f5:6c94:2cc1 with SMTP id adf61e73a8af0-201797b7eeemr6841826637.21.1744420225268; Fri, 11 Apr 2025 18:10:25 -0700 (PDT) Date: Sat, 12 Apr 2025 01:09:39 +0000 In-Reply-To: <20250412010940.1686376-1-dylanbhatch@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250412010940.1686376-1-dylanbhatch@google.com> X-Mailer: git-send-email 2.49.0.604.gff1f9ca942-goog Message-ID: <20250412010940.1686376-3-dylanbhatch@google.com> Subject: [PATCH v2 2/2] arm64/module: Use text-poke API for late relocations. From: Dylan Hatch To: Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Puranjay Mohan , Xu Kuohai , "=?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?=" , Catalin Marinas , Will Deacon , "Mike Rapoport (Microsoft)" , Arnd Bergmann , Geert Uytterhoeven , Luis Chamberlain , Andrew Morton , Song Liu , Ard Biesheuvel , Mark Rutland Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Dylan Hatch , Roman Gushchin Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" To enable late module patching, livepatch modules need to be able to apply some of their relocations well after being loaded. In this scenario, use the text-poking API to allow this, even with STRICT_MODULE_RWX. This patch is largely based off commit 88fc078a7a8f6 ("x86/module: Use text_poke() for late relocations"). Signed-off-by: Dylan Hatch Acked-by: Song Liu --- arch/arm64/kernel/module.c | 129 ++++++++++++++++++++++++------------- 1 file changed, 83 insertions(+), 46 deletions(-) diff --git a/arch/arm64/kernel/module.c b/arch/arm64/kernel/module.c index 06bb680bfe975..0703502d9dc37 100644 --- a/arch/arm64/kernel/module.c +++ b/arch/arm64/kernel/module.c @@ -18,11 +18,13 @@ #include #include #include +#include =20 #include #include #include #include +#include =20 enum aarch64_reloc_op { RELOC_OP_NONE, @@ -48,7 +50,8 @@ static u64 do_reloc(enum aarch64_reloc_op reloc_op, __le3= 2 *place, u64 val) return 0; } =20 -static int reloc_data(enum aarch64_reloc_op op, void *place, u64 val, int = len) +static int reloc_data(enum aarch64_reloc_op op, void *place, u64 val, int = len, + void *(*write)(void *dest, const void *src, size_t len)) { s64 sval =3D do_reloc(op, place, val); =20 @@ -66,7 +69,7 @@ static int reloc_data(enum aarch64_reloc_op op, void *pla= ce, u64 val, int len) =20 switch (len) { case 16: - *(s16 *)place =3D sval; + write(place, &sval, sizeof(s16)); switch (op) { case RELOC_OP_ABS: if (sval < 0 || sval > U16_MAX) @@ -82,7 +85,7 @@ static int reloc_data(enum aarch64_reloc_op op, void *pla= ce, u64 val, int len) } break; case 32: - *(s32 *)place =3D sval; + write(place, &sval, sizeof(s32)); switch (op) { case RELOC_OP_ABS: if (sval < 0 || sval > U32_MAX) @@ -98,7 +101,7 @@ static int reloc_data(enum aarch64_reloc_op op, void *pl= ace, u64 val, int len) } break; case 64: - *(s64 *)place =3D sval; + write(place, &sval, sizeof(s64)); break; default: pr_err("Invalid length (%d) for data relocation\n", len); @@ -113,11 +116,13 @@ enum aarch64_insn_movw_imm_type { }; =20 static int reloc_insn_movw(enum aarch64_reloc_op op, __le32 *place, u64 va= l, - int lsb, enum aarch64_insn_movw_imm_type imm_type) + int lsb, enum aarch64_insn_movw_imm_type imm_type, + void *(*write)(void *dest, const void *src, size_t len)) { u64 imm; s64 sval; u32 insn =3D le32_to_cpu(*place); + __le32 le_insn; =20 sval =3D do_reloc(op, place, val); imm =3D sval >> lsb; @@ -145,7 +150,8 @@ static int reloc_insn_movw(enum aarch64_reloc_op op, __= le32 *place, u64 val, =20 /* Update the instruction with the new encoding. */ insn =3D aarch64_insn_encode_immediate(AARCH64_INSN_IMM_16, insn, imm); - *place =3D cpu_to_le32(insn); + le_insn =3D cpu_to_le32(insn); + write(place, &le_insn, sizeof(le_insn)); =20 if (imm > U16_MAX) return -ERANGE; @@ -154,11 +160,13 @@ static int reloc_insn_movw(enum aarch64_reloc_op op, = __le32 *place, u64 val, } =20 static int reloc_insn_imm(enum aarch64_reloc_op op, __le32 *place, u64 val, - int lsb, int len, enum aarch64_insn_imm_type imm_type) + int lsb, int len, enum aarch64_insn_imm_type imm_type, + void *(*write)(void *dest, const void *src, size_t len)) { u64 imm, imm_mask; s64 sval; u32 insn =3D le32_to_cpu(*place); + __le32 le_insn; =20 /* Calculate the relocation value. */ sval =3D do_reloc(op, place, val); @@ -170,7 +178,8 @@ static int reloc_insn_imm(enum aarch64_reloc_op op, __l= e32 *place, u64 val, =20 /* Update the instruction's immediate field. */ insn =3D aarch64_insn_encode_immediate(imm_type, insn, imm); - *place =3D cpu_to_le32(insn); + le_insn =3D cpu_to_le32(insn); + write(place, &le_insn, sizeof(le_insn)); =20 /* * Extract the upper value bits (including the sign bit) and @@ -189,17 +198,19 @@ static int reloc_insn_imm(enum aarch64_reloc_op op, _= _le32 *place, u64 val, } =20 static int reloc_insn_adrp(struct module *mod, Elf64_Shdr *sechdrs, - __le32 *place, u64 val) + __le32 *place, u64 val, + void *(*write)(void *dest, const void *src, size_t len)) { u32 insn; + __le32 le_insn; =20 if (!is_forbidden_offset_for_adrp(place)) return reloc_insn_imm(RELOC_OP_PAGE, place, val, 12, 21, - AARCH64_INSN_IMM_ADR); + AARCH64_INSN_IMM_ADR, write); =20 /* patch ADRP to ADR if it is in range */ if (!reloc_insn_imm(RELOC_OP_PREL, place, val & ~0xfff, 0, 21, - AARCH64_INSN_IMM_ADR)) { + AARCH64_INSN_IMM_ADR, write)) { insn =3D le32_to_cpu(*place); insn &=3D ~BIT(31); } else { @@ -211,15 +222,17 @@ static int reloc_insn_adrp(struct module *mod, Elf64_= Shdr *sechdrs, AARCH64_INSN_BRANCH_NOLINK); } =20 - *place =3D cpu_to_le32(insn); + le_insn =3D cpu_to_le32(insn); + write(place, &le_insn, sizeof(le_insn)); return 0; } =20 -int apply_relocate_add(Elf64_Shdr *sechdrs, +static int __apply_relocate_add(Elf64_Shdr *sechdrs, const char *strtab, unsigned int symindex, unsigned int relsec, - struct module *me) + struct module *me, + void *(*write)(void *dest, const void *src, size_t len)) { unsigned int i; int ovf; @@ -255,23 +268,23 @@ int apply_relocate_add(Elf64_Shdr *sechdrs, /* Data relocations. */ case R_AARCH64_ABS64: overflow_check =3D false; - ovf =3D reloc_data(RELOC_OP_ABS, loc, val, 64); + ovf =3D reloc_data(RELOC_OP_ABS, loc, val, 64, write); break; case R_AARCH64_ABS32: - ovf =3D reloc_data(RELOC_OP_ABS, loc, val, 32); + ovf =3D reloc_data(RELOC_OP_ABS, loc, val, 32, write); break; case R_AARCH64_ABS16: - ovf =3D reloc_data(RELOC_OP_ABS, loc, val, 16); + ovf =3D reloc_data(RELOC_OP_ABS, loc, val, 16, write); break; case R_AARCH64_PREL64: overflow_check =3D false; - ovf =3D reloc_data(RELOC_OP_PREL, loc, val, 64); + ovf =3D reloc_data(RELOC_OP_PREL, loc, val, 64, write); break; case R_AARCH64_PREL32: - ovf =3D reloc_data(RELOC_OP_PREL, loc, val, 32); + ovf =3D reloc_data(RELOC_OP_PREL, loc, val, 32, write); break; case R_AARCH64_PREL16: - ovf =3D reloc_data(RELOC_OP_PREL, loc, val, 16); + ovf =3D reloc_data(RELOC_OP_PREL, loc, val, 16, write); break; =20 /* MOVW instruction relocations. */ @@ -280,88 +293,88 @@ int apply_relocate_add(Elf64_Shdr *sechdrs, fallthrough; case R_AARCH64_MOVW_UABS_G0: ovf =3D reloc_insn_movw(RELOC_OP_ABS, loc, val, 0, - AARCH64_INSN_IMM_MOVKZ); + AARCH64_INSN_IMM_MOVKZ, write); break; case R_AARCH64_MOVW_UABS_G1_NC: overflow_check =3D false; fallthrough; case R_AARCH64_MOVW_UABS_G1: ovf =3D reloc_insn_movw(RELOC_OP_ABS, loc, val, 16, - AARCH64_INSN_IMM_MOVKZ); + AARCH64_INSN_IMM_MOVKZ, write); break; case R_AARCH64_MOVW_UABS_G2_NC: overflow_check =3D false; fallthrough; case R_AARCH64_MOVW_UABS_G2: ovf =3D reloc_insn_movw(RELOC_OP_ABS, loc, val, 32, - AARCH64_INSN_IMM_MOVKZ); + AARCH64_INSN_IMM_MOVKZ, write); break; case R_AARCH64_MOVW_UABS_G3: /* We're using the top bits so we can't overflow. */ overflow_check =3D false; ovf =3D reloc_insn_movw(RELOC_OP_ABS, loc, val, 48, - AARCH64_INSN_IMM_MOVKZ); + AARCH64_INSN_IMM_MOVKZ, write); break; case R_AARCH64_MOVW_SABS_G0: ovf =3D reloc_insn_movw(RELOC_OP_ABS, loc, val, 0, - AARCH64_INSN_IMM_MOVNZ); + AARCH64_INSN_IMM_MOVNZ, write); break; case R_AARCH64_MOVW_SABS_G1: ovf =3D reloc_insn_movw(RELOC_OP_ABS, loc, val, 16, - AARCH64_INSN_IMM_MOVNZ); + AARCH64_INSN_IMM_MOVNZ, write); break; case R_AARCH64_MOVW_SABS_G2: ovf =3D reloc_insn_movw(RELOC_OP_ABS, loc, val, 32, - AARCH64_INSN_IMM_MOVNZ); + AARCH64_INSN_IMM_MOVNZ, write); break; case R_AARCH64_MOVW_PREL_G0_NC: overflow_check =3D false; ovf =3D reloc_insn_movw(RELOC_OP_PREL, loc, val, 0, - AARCH64_INSN_IMM_MOVKZ); + AARCH64_INSN_IMM_MOVKZ, write); break; case R_AARCH64_MOVW_PREL_G0: ovf =3D reloc_insn_movw(RELOC_OP_PREL, loc, val, 0, - AARCH64_INSN_IMM_MOVNZ); + AARCH64_INSN_IMM_MOVNZ, write); break; case R_AARCH64_MOVW_PREL_G1_NC: overflow_check =3D false; ovf =3D reloc_insn_movw(RELOC_OP_PREL, loc, val, 16, - AARCH64_INSN_IMM_MOVKZ); + AARCH64_INSN_IMM_MOVKZ, write); break; case R_AARCH64_MOVW_PREL_G1: ovf =3D reloc_insn_movw(RELOC_OP_PREL, loc, val, 16, - AARCH64_INSN_IMM_MOVNZ); + AARCH64_INSN_IMM_MOVNZ, write); break; case R_AARCH64_MOVW_PREL_G2_NC: overflow_check =3D false; ovf =3D reloc_insn_movw(RELOC_OP_PREL, loc, val, 32, - AARCH64_INSN_IMM_MOVKZ); + AARCH64_INSN_IMM_MOVKZ, write); break; case R_AARCH64_MOVW_PREL_G2: ovf =3D reloc_insn_movw(RELOC_OP_PREL, loc, val, 32, - AARCH64_INSN_IMM_MOVNZ); + AARCH64_INSN_IMM_MOVNZ, write); break; case R_AARCH64_MOVW_PREL_G3: /* We're using the top bits so we can't overflow. */ overflow_check =3D false; ovf =3D reloc_insn_movw(RELOC_OP_PREL, loc, val, 48, - AARCH64_INSN_IMM_MOVNZ); + AARCH64_INSN_IMM_MOVNZ, write); break; =20 /* Immediate instruction relocations. */ case R_AARCH64_LD_PREL_LO19: ovf =3D reloc_insn_imm(RELOC_OP_PREL, loc, val, 2, 19, - AARCH64_INSN_IMM_19); + AARCH64_INSN_IMM_19, write); break; case R_AARCH64_ADR_PREL_LO21: ovf =3D reloc_insn_imm(RELOC_OP_PREL, loc, val, 0, 21, - AARCH64_INSN_IMM_ADR); + AARCH64_INSN_IMM_ADR, write); break; case R_AARCH64_ADR_PREL_PG_HI21_NC: overflow_check =3D false; fallthrough; case R_AARCH64_ADR_PREL_PG_HI21: - ovf =3D reloc_insn_adrp(me, sechdrs, loc, val); + ovf =3D reloc_insn_adrp(me, sechdrs, loc, val, write); if (ovf && ovf !=3D -ERANGE) return ovf; break; @@ -369,46 +382,46 @@ int apply_relocate_add(Elf64_Shdr *sechdrs, case R_AARCH64_LDST8_ABS_LO12_NC: overflow_check =3D false; ovf =3D reloc_insn_imm(RELOC_OP_ABS, loc, val, 0, 12, - AARCH64_INSN_IMM_12); + AARCH64_INSN_IMM_12, write); break; case R_AARCH64_LDST16_ABS_LO12_NC: overflow_check =3D false; ovf =3D reloc_insn_imm(RELOC_OP_ABS, loc, val, 1, 11, - AARCH64_INSN_IMM_12); + AARCH64_INSN_IMM_12, write); break; case R_AARCH64_LDST32_ABS_LO12_NC: overflow_check =3D false; ovf =3D reloc_insn_imm(RELOC_OP_ABS, loc, val, 2, 10, - AARCH64_INSN_IMM_12); + AARCH64_INSN_IMM_12, write); break; case R_AARCH64_LDST64_ABS_LO12_NC: overflow_check =3D false; ovf =3D reloc_insn_imm(RELOC_OP_ABS, loc, val, 3, 9, - AARCH64_INSN_IMM_12); + AARCH64_INSN_IMM_12, write); break; case R_AARCH64_LDST128_ABS_LO12_NC: overflow_check =3D false; ovf =3D reloc_insn_imm(RELOC_OP_ABS, loc, val, 4, 8, - AARCH64_INSN_IMM_12); + AARCH64_INSN_IMM_12, write); break; case R_AARCH64_TSTBR14: ovf =3D reloc_insn_imm(RELOC_OP_PREL, loc, val, 2, 14, - AARCH64_INSN_IMM_14); + AARCH64_INSN_IMM_14, write); break; case R_AARCH64_CONDBR19: ovf =3D reloc_insn_imm(RELOC_OP_PREL, loc, val, 2, 19, - AARCH64_INSN_IMM_19); + AARCH64_INSN_IMM_19, write); break; case R_AARCH64_JUMP26: case R_AARCH64_CALL26: ovf =3D reloc_insn_imm(RELOC_OP_PREL, loc, val, 2, 26, - AARCH64_INSN_IMM_26); + AARCH64_INSN_IMM_26, write); if (ovf =3D=3D -ERANGE) { val =3D module_emit_plt_entry(me, sechdrs, loc, &rel[i], sym); if (!val) return -ENOEXEC; ovf =3D reloc_insn_imm(RELOC_OP_PREL, loc, val, 2, - 26, AARCH64_INSN_IMM_26); + 26, AARCH64_INSN_IMM_26, write); } break; =20 @@ -431,6 +444,30 @@ int apply_relocate_add(Elf64_Shdr *sechdrs, return -ENOEXEC; } =20 +int apply_relocate_add(Elf64_Shdr *sechdrs, + const char *strtab, + unsigned int symindex, + unsigned int relsec, + struct module *me) +{ + int ret; + bool early =3D me->state =3D=3D MODULE_STATE_UNFORMED; + void *(*write)(void *, const void *, size_t) =3D memcpy; + + if (!early) { + write =3D text_poke; + mutex_lock(&text_mutex); + } + + ret =3D __apply_relocate_add(sechdrs, strtab, symindex, relsec, me, + write); + + if (!early) + mutex_unlock(&text_mutex); + + return ret; +} + static inline void __init_plt(struct plt_entry *plt, unsigned long addr) { *plt =3D get_plt_entry(addr, plt); --=20 2.49.0.604.gff1f9ca942-goog