From nobody Sun Feb 8 22:34:46 2026 Received: from mail-wm1-f51.google.com (mail-wm1-f51.google.com [209.85.128.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DB0561CAAA for ; Sat, 16 Mar 2024 23:21:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710631285; cv=none; b=MuGRGdWEUq6ZP6m0Fy4yW295nr86UBGOhS8EmFNO2LfuQrnG40KKkMYG+5mcDx/DDo8G1lcstGSt2+YxF/M1IXxBZiCAsGA7uRNNOTA8ynqaihUwFw/rGeK/eoXL+swRL3pPvFiOyH6fapZLvRkn4zMTR5RrgwRZHh8zJGH6mp4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710631285; c=relaxed/simple; bh=8I+9Xh1B8ca2rFZocr4NbZZsAyhD5pZEaMy/G/2Cfmk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=WWYLyHzaj27FYSEIEiwQPHri2hcxmVl2IA2eblSbYdJHZOukt5ULjvywW8+rVVonWarmt02bcM5hVFBPxckBa5Y4cT44dI3xw3RbS9wRBcIwRZQ9SVsXamwjB/hhV5H4zDy5QH8crhGim1spFQ5g54FhqK3tdfCv4JoRSt9M8FY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Y0zUzSmD; arc=none smtp.client-ip=209.85.128.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Y0zUzSmD" Received: by mail-wm1-f51.google.com with SMTP id 5b1f17b1804b1-41405d77c7bso8481915e9.3 for ; Sat, 16 Mar 2024 16:21:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1710631282; x=1711236082; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FmX3xpzhkH4xRY1ryBz8V2RojP635fvXqLUXYTnRCuU=; b=Y0zUzSmDbp23U6+gbyAEzzjOkgA9HTXsmo6rSLdqGNM3p6xvpUPVfSp/5DfTmykwIH sDJMCiG9af20tXdz54MYD/9w5ZN0vt94hag3OdY5d3fElXskFyHagLrG5Q8hmbrZxqzw OjiFD6QmilDlW7ydCSAVjMN1/rvQ0sp7R3xsUyGMJwm5eeq6cnVSBXq3F+Vb3bMK1p7o kssof3uct9MemvNTw+woVnEX2wvN4ANcanISqGUKyClyq1dLueMuaB68vcmkisJHqiz3 QChf6hH9ZCt/Gkm9X5mdnFIbzbqc8fr1PvPWCYn2oATEslJcbPqeENptDgXn5QAQ+RRH asQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710631282; x=1711236082; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FmX3xpzhkH4xRY1ryBz8V2RojP635fvXqLUXYTnRCuU=; b=bwgCxlwIWR3WMcNUfiOsldKj7p78s2PZo+ekmjyt0Q4kLuIRWsz8ghws2yVHo5MMLK H56jojbOC/zfS8qgfE4UalSB9U5L+N4Kao0q4MGoydPyEMmIT23cN9QjZdDZLPZPeFCP p66EpzoZp6NDzx/z3g0CafVIEGIlreG0ygiWJ+U3ejw/NHBw2+7BNkWvi6K0TkScj3JO fSLy19WxPO54cZI+3biZ+fQbtvy3wvGagZ8jfN2k4gaMVkugfGVrby2JSyPXS/aT5Esv 9d0pquOJWPwiU0XVnftIal0VNoyqMmJhHhMNuLgaf7GOo+3v+hfZ1EMVkqOZrxcxftus mhew== X-Forwarded-Encrypted: i=1; AJvYcCXeoMC684aH0YpJLWr1mNz0+PPMGKwRZzodrxglhPfUHtbmCrS2ErhwfwNUtE5RXdjVOsKMoM3yZLTfV9eOJohQN2syb0VUEQVzzEpm X-Gm-Message-State: AOJu0YzQEAgUwl8WrBs1SZkVxT0+9HZtNOTMLMfkLxdw3Zp8wccj+5fQ IYdyIB9LHgwy+i5Ce4uMIcMkTcN3mHy5GhSGhJxQ9EhlWvURY4jK X-Google-Smtp-Source: AGHT+IGFwm9bPqWxJCgs8iwJ3DYOZlNg5oMSi29NqNYG0e739fpFWf1qV3fjQJrjPsh3Xva7P6OsCw== X-Received: by 2002:a05:600c:5124:b0:412:ebde:8873 with SMTP id o36-20020a05600c512400b00412ebde8873mr6822309wms.30.1710631282256; Sat, 16 Mar 2024 16:21:22 -0700 (PDT) Received: from solport.. (82.red-83-42-63.dynamicip.rima-tde.net. [83.42.63.82]) by smtp.gmail.com with ESMTPSA id fm13-20020a05600c0c0d00b004140a13f998sm1597028wmb.11.2024.03.16.16.21.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 16 Mar 2024 16:21:21 -0700 (PDT) From: =?UTF-8?q?Joan=20Bruguera=20Mic=C3=B3?= To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Uros Bizjak , Borislav Petkov , Dave Hansen , Peter Zijlstra , "H. Peter Anvin" , =?UTF-8?q?Joan=20Bruguera=20Mic=C3=B3?= Subject: [PATCH 1/2] x86/bpf: Fix IP after emitting call depth accounting Date: Sat, 16 Mar 2024 23:21:03 +0000 Message-ID: <20240316232104.368561-2-joanbrugueram@gmail.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240316232104.368561-1-joanbrugueram@gmail.com> References: <20240316232104.368561-1-joanbrugueram@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Adjust the IP passed to `emit_patch` so it calculates the correct offset for the CALL instruction if `x86_call_depth_emit_accounting` emits code. Otherwise we will skip some instructions and most likely crash. Fixes: b2e9dfe54be4 ("x86/bpf: Emit call depth accounting if required") Link: https://lore.kernel.org/lkml/20230105214922.250473-1-joanbrugueram@gm= ail.com/ Signed-off-by: Joan Bruguera Mic=C3=B3 --- arch/x86/net/bpf_jit_comp.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c index a7ba8e178645..09f7dc9d4d65 100644 --- a/arch/x86/net/bpf_jit_comp.c +++ b/arch/x86/net/bpf_jit_comp.c @@ -479,9 +479,10 @@ static int emit_call(u8 **pprog, void *func, void *ip) =20 static int emit_rsb_call(u8 **pprog, void *func, void *ip) { + void *adjusted_ip; OPTIMIZER_HIDE_VAR(func); - x86_call_depth_emit_accounting(pprog, func); - return emit_patch(pprog, func, ip, 0xE8); + adjusted_ip =3D ip + x86_call_depth_emit_accounting(pprog, func); + return emit_patch(pprog, func, adjusted_ip, 0xE8); } =20 static int emit_jump(u8 **pprog, void *func, void *ip) --=20 2.44.0 From nobody Sun Feb 8 22:34:46 2026 Received: from mail-wm1-f53.google.com (mail-wm1-f53.google.com [209.85.128.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D8ABC433D8 for ; Sat, 16 Mar 2024 23:21:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710631287; cv=none; b=oyW5Mpzg0S3emk9GTTRqCUqkvcLEkiuXv1KwrzPiDAKCaEdS1wSR+ZaBzEdmTKwOiWul+hkjQmJD+CesL4jszDEx0dMaAMobpRwkOinDmtJvhhha7owZ/gqMeTk3G236Z/UH/SIJBwj1fcNR75ZCvwGqPlsGz4hQ5j82oj665YY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710631287; c=relaxed/simple; bh=n8w1ZtzBd5/Q5wTKnX7AEhzagI2H+aqaDS9Fye2Kfhs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=sNyAuB++kK9fbo6bDIt1bL3fQCENMqHXor+UAiZKLtYTUTy7Fi9AsQM37+pibTpQQrKAX7tVjCgenRQg9Czxf3uHs/ATH7mPuKz231b21Awq8SpW7iO8HusmUN8uo91QN/P94Nv1PVD8lnPgOUYkhRLL/LuS/VIjuK1rwrgnFlE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=YfLvQtjk; arc=none smtp.client-ip=209.85.128.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="YfLvQtjk" Received: by mail-wm1-f53.google.com with SMTP id 5b1f17b1804b1-413fea148c9so16116015e9.2 for ; Sat, 16 Mar 2024 16:21:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1710631284; x=1711236084; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=eyYnPtGjqDeDPkvkILQi659Kg/uKdJDkB+U0D+eZfJw=; b=YfLvQtjkuIlyCpWbKYpYWedAIEHc7P7mxwSvUsOcJ/D9KbhCTVn/klkC+Ao5lCf5gY g1AxMmh6ARyu+dLfc6dLiUBuWZ6w2Maj96dWZiAy9+dc5leyJsd1tziMMMtiGxFRPD70 +JIIBjNGF19xk0JUEJlwhta6ec62xfXBBnZki021Tb+gPgBn1elJmUNOOHZTD/sJOJgV SHMLknexhDIjkdS/0fI4anW0fkwqQ+4Hy+PHEybEflQrERUNQpysKhXYUyGTLK5VTdrI NbUFyg3EVqtUlQFLROeTyJdGEahp7YZqHN1WAghPYgo66hxKbjEWDTkuQIPrLNlHBj3+ u8FQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710631284; x=1711236084; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eyYnPtGjqDeDPkvkILQi659Kg/uKdJDkB+U0D+eZfJw=; b=AYD0RsDKYv2KGJcqckQX71TF6DalIkU2vo3h4rD4y7cqcQvWNkxDvXJgIavXbFy0sH wU6iTZKEChj/+FnGPeKLpVFnfNdUlB7vo0cztxlJ9kEQBnZ3MiCF6lpxqbkY39+t9mYL 59h8brmv4vDSrQ/x3DjMWqSBMA3ccW0lHV35LP7yDg81fmEIuVcVDuezFLSr3mReiWj1 hHE9RkJeSnV0l/E7iIBunODza1k1OHrvmAAb37NXnjCUDc4Ez/sYvnxhNIR2bro0tw/K m/OKRLYFoigGl0HEZMy6Mng/Cn7nn+YPM6XDMG5d7kedTfAhLieY3jnCaaPgKDx5w4Pm FMTQ== X-Forwarded-Encrypted: i=1; AJvYcCUOyBvnvO/WlFnb77isNRRRL5pZckQCQL46zpcPEbKeLJ3MJseB3z/EwWhMj41/Rc1kFqEdjkqnqLWjk6byKywj3GrfUAgWCpwySWVh X-Gm-Message-State: AOJu0YwJ9NIsYnMu6zukZOQ1OeFjgQI7bOThGCMBt3KGNcAbrlYGTNkS Vd91d/WZYnDm3EDDtSZVassw7FfpvKtBXiDSeauUkA07JS3MsYui X-Google-Smtp-Source: AGHT+IHEoz7YU5ogG/DA3gRILG6NmbkQL/3zAPF4zLOMgtqak+7iBU+huSDPe1KYz2uITMxmy/M+wg== X-Received: by 2002:a05:600c:458e:b0:413:812:ce7b with SMTP id r14-20020a05600c458e00b004130812ce7bmr6483817wmo.24.1710631284007; Sat, 16 Mar 2024 16:21:24 -0700 (PDT) Received: from solport.. (82.red-83-42-63.dynamicip.rima-tde.net. [83.42.63.82]) by smtp.gmail.com with ESMTPSA id fm13-20020a05600c0c0d00b004140a13f998sm1597028wmb.11.2024.03.16.16.21.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 16 Mar 2024 16:21:23 -0700 (PDT) From: =?UTF-8?q?Joan=20Bruguera=20Mic=C3=B3?= To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Thomas Gleixner , Ingo Molnar , Uros Bizjak , Borislav Petkov , Dave Hansen , Peter Zijlstra , "H. Peter Anvin" , =?UTF-8?q?Joan=20Bruguera=20Mic=C3=B3?= Subject: [PATCH 2/2] x86/bpf: Fix IP for relocating call depth accounting Date: Sat, 16 Mar 2024 23:21:04 +0000 Message-ID: <20240316232104.368561-3-joanbrugueram@gmail.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240316232104.368561-1-joanbrugueram@gmail.com> References: <20240316232104.368561-1-joanbrugueram@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable The recently introduced support for %rip-relative relocations in the call thunk template assumes that the code is being patched in-place, so the destination of the relocation matches the address of the code. This is not true for the call depth accounting emitted by the BPF JIT, so the calculated address is wrong and usually causes a page fault. Pass the destination IP when the BPF JIT emits call depth accounting. Fixes: 17bce3b2ae2d ("x86/callthunks: Handle %rip-relative relocations in c= all thunk template") Signed-off-by: Joan Bruguera Mic=C3=B3 Reviewed-by: Uros Bizjak --- arch/x86/include/asm/alternative.h | 4 ++-- arch/x86/kernel/callthunks.c | 4 ++-- arch/x86/net/bpf_jit_comp.c | 19 ++++++++----------- 3 files changed, 12 insertions(+), 15 deletions(-) diff --git a/arch/x86/include/asm/alternative.h b/arch/x86/include/asm/alte= rnative.h index fcd20c6dc7f9..67b68d0d17d1 100644 --- a/arch/x86/include/asm/alternative.h +++ b/arch/x86/include/asm/alternative.h @@ -117,7 +117,7 @@ extern void callthunks_patch_builtin_calls(void); extern void callthunks_patch_module_calls(struct callthunk_sites *sites, struct module *mod); extern void *callthunks_translate_call_dest(void *dest); -extern int x86_call_depth_emit_accounting(u8 **pprog, void *func); +extern int x86_call_depth_emit_accounting(u8 **pprog, void *func, void *ip= ); #else static __always_inline void callthunks_patch_builtin_calls(void) {} static __always_inline void @@ -128,7 +128,7 @@ static __always_inline void *callthunks_translate_call_= dest(void *dest) return dest; } static __always_inline int x86_call_depth_emit_accounting(u8 **pprog, - void *func) + void *func, void *ip) { return 0; } diff --git a/arch/x86/kernel/callthunks.c b/arch/x86/kernel/callthunks.c index 30335182b6b0..e92ff0c11db8 100644 --- a/arch/x86/kernel/callthunks.c +++ b/arch/x86/kernel/callthunks.c @@ -314,7 +314,7 @@ static bool is_callthunk(void *addr) return !bcmp(pad, insn_buff, tmpl_size); } =20 -int x86_call_depth_emit_accounting(u8 **pprog, void *func) +int x86_call_depth_emit_accounting(u8 **pprog, void *func, void *ip) { unsigned int tmpl_size =3D SKL_TMPL_SIZE; u8 insn_buff[MAX_PATCH_LEN]; @@ -327,7 +327,7 @@ int x86_call_depth_emit_accounting(u8 **pprog, void *fu= nc) return 0; =20 memcpy(insn_buff, skl_call_thunk_template, tmpl_size); - apply_relocation(insn_buff, tmpl_size, *pprog, + apply_relocation(insn_buff, tmpl_size, ip, skl_call_thunk_template, tmpl_size); =20 memcpy(*pprog, insn_buff, tmpl_size); diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c index 09f7dc9d4d65..f2e8769f5eee 100644 --- a/arch/x86/net/bpf_jit_comp.c +++ b/arch/x86/net/bpf_jit_comp.c @@ -481,7 +481,7 @@ static int emit_rsb_call(u8 **pprog, void *func, void *= ip) { void *adjusted_ip; OPTIMIZER_HIDE_VAR(func); - adjusted_ip =3D ip + x86_call_depth_emit_accounting(pprog, func); + adjusted_ip =3D ip + x86_call_depth_emit_accounting(pprog, func, ip); return emit_patch(pprog, func, adjusted_ip, 0xE8); } =20 @@ -1973,20 +1973,17 @@ st: if (is_imm8(insn->off)) =20 /* call */ case BPF_JMP | BPF_CALL: { - int offs; + u8 *ip =3D image + addrs[i - 1]; =20 func =3D (u8 *) __bpf_call_base + imm32; if (tail_call_reachable) { RESTORE_TAIL_CALL_CNT(bpf_prog->aux->stack_depth); - if (!imm32) - return -EINVAL; - offs =3D 7 + x86_call_depth_emit_accounting(&prog, func); - } else { - if (!imm32) - return -EINVAL; - offs =3D x86_call_depth_emit_accounting(&prog, func); + ip +=3D 7; } - if (emit_call(&prog, func, image + addrs[i - 1] + offs)) + if (!imm32) + return -EINVAL; + ip +=3D x86_call_depth_emit_accounting(&prog, func, ip); + if (emit_call(&prog, func, ip)) return -EINVAL; break; } @@ -2836,7 +2833,7 @@ static int __arch_prepare_bpf_trampoline(struct bpf_t= ramp_image *im, void *rw_im * Direct-call fentry stub, as such it needs accounting for the * __fentry__ call. */ - x86_call_depth_emit_accounting(&prog, NULL); + x86_call_depth_emit_accounting(&prog, NULL, image); } EMIT1(0x55); /* push rbp */ EMIT3(0x48, 0x89, 0xE5); /* mov rbp, rsp */ --=20 2.44.0