From nobody Sat Oct 11 08:12:45 2025 Received: from mailgw.kylinos.cn (mailgw.kylinos.cn [124.126.103.232]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8F88620C485 for ; Wed, 11 Jun 2025 05:36:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=124.126.103.232 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749620204; cv=none; b=Khl7nQujtfZ/buY/xy4MhfeJj4D3lwpd4fi6xg22S7326Xn434dAOz9bXxaGwjWffMNKl/UB91abx4g11O2xB3o3C+xLcdjhuXf18n1802lbzdsWHoKnXJd+P7vvJB8hLYSTM+xUOUwjNx4sZNaqq1q40s7lAw322YNnSMVKXY4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749620204; c=relaxed/simple; bh=5RwXiP9db3ZiMTTmqPZMvskn3c84oxlF0u3TQMV3CdY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Gm1upBNcM2oB8Ck56WI+ETC3YiGvrnTdbmkTLqRkV5oRbTJ+fkn52x569d0+TUVb6wTjNFPbhsvm6tENt6qD7Zw2Qwi38lwKq0WyYjjkKosPAS5pN38J2aajlYe77uCbAPcGgH4cJe9ec2dIBCQeOBcJHhadoRMaNU6a8/9xqJY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kylinos.cn; spf=pass smtp.mailfrom=kylinos.cn; arc=none smtp.client-ip=124.126.103.232 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kylinos.cn Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kylinos.cn X-UUID: 092a3808468611f0b29709d653e92f7d-20250611 X-CTIC-Tags: HR_CC_COUNT, HR_CC_DOMAIN_COUNT, HR_CC_NAME, HR_CTE_8B, HR_CTT_MISS HR_DATE_H, HR_DATE_WKD, HR_DATE_ZONE, HR_FROM_NAME, HR_SJ_DIGIT_LEN HR_SJ_LANG, HR_SJ_LEN, HR_SJ_LETTER, HR_SJ_NOR_SYM, HR_SJ_PHRASE HR_SJ_PHRASE_LEN, HR_SJ_WS, HR_TO_COUNT, HR_TO_DOMAIN_COUNT, HR_TO_NO_NAME IP_UNTRUSTED, SRC_UNTRUSTED, IP_LOWREP, SRC_LOWREP, DN_TRUSTED SRC_TRUSTED, SA_EXISTED, SN_UNTRUSTED, SN_UNFAMILIAR, SPF_NOPASS DKIM_NOPASS, DMARC_NOPASS, CIE_BAD, CIE_GOOD, CIE_GOOD_SPF GTI_FG_BS, GTI_RG_INFO, GTI_C_BU, AMN_T1, AMN_GOOD AMN_C_TI, AMN_C_BU, ABX_MISS_RDNS X-CID-P-RULE: Release_Ham X-CID-O-INFO: VERSION:1.1.45,REQID:4f519acc-bb8a-436a-83e6-1b7c8d1a8648,IP:10, URL:0,TC:0,Content:0,EDM:0,RT:0,SF:-5,FILE:0,BULK:0,RULE:Release_Ham,ACTIO N:release,TS:5 X-CID-INFO: VERSION:1.1.45,REQID:4f519acc-bb8a-436a-83e6-1b7c8d1a8648,IP:10,UR L:0,TC:0,Content:0,EDM:0,RT:0,SF:-5,FILE:0,BULK:0,RULE:Release_Ham,ACTION: release,TS:5 X-CID-META: VersionHash:6493067,CLOUDID:dd49578951bb38abcaeae2c14f3778fd,BulkI D:250611133637X39LCBZ6,BulkQuantity:0,Recheck:0,SF:17|19|25|38|45|66|78|81 |82|102,TC:nil,Content:0|50,EDM:-3,IP:-2,URL:0,File:nil,RT:nil,Bulk:nil,QS :nil,BEC:nil,COL:0,OSI:0,OSA:0,AV:0,LES:1,SPR:NO,DKR:0,DKP:0,BRR:0,BRE:0,A RC:0 X-CID-BVR: 0,NGT X-CID-BAS: 0,NGT,0,_ X-CID-FACTOR: TF_CID_SPAM_FSD,TF_CID_SPAM_FSI,TF_CID_SPAM_SNR,TF_CID_SPAM_FAS X-UUID: 092a3808468611f0b29709d653e92f7d-20250611 X-User: duanchenghao@kylinos.cn Received: from localhost.localdomain [(223.104.40.103)] by mailgw.kylinos.cn (envelope-from ) (Generic MTA) with ESMTP id 227271751; Wed, 11 Jun 2025 13:36:34 +0800 From: Chenghao Duan To: loongarch@lists.linux.dev, linux-kernel@vger.kernel.org Cc: George Guo , Chenghao Duan Subject: [PATCH v1 1/5] LoongArch: Support fixmap Date: Wed, 11 Jun 2025 13:36:21 +0800 Message-Id: <20250611053625.352091-2-duanchenghao@kylinos.cn> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20250611053625.352091-1-duanchenghao@kylinos.cn> References: <20250611035952.111182-3-duanchenghao@kylinos.cn> <20250611053625.352091-1-duanchenghao@kylinos.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: George Guo Support fixmap for Loongarch. In the functions patch_map and patch_unmap, __set_fixmap are used. Therefore, remove the __init identifier from these functions. Signed-off-by: Chenghao Duan Signed-off-by: George Guo --- arch/loongarch/include/asm/fixmap.h | 2 + arch/loongarch/kernel/setup.c | 1 + arch/loongarch/mm/init.c | 111 ++++++++++++++++++++++++++-- 3 files changed, 108 insertions(+), 6 deletions(-) diff --git a/arch/loongarch/include/asm/fixmap.h b/arch/loongarch/include/a= sm/fixmap.h index d2e55ae55..b579ad2be 100644 --- a/arch/loongarch/include/asm/fixmap.h +++ b/arch/loongarch/include/asm/fixmap.h @@ -13,6 +13,7 @@ enum fixed_addresses { FIX_HOLE, FIX_EARLYCON_MEM_BASE, + FIX_TEXT_POKE0, __end_of_fixed_addresses }; =20 @@ -20,6 +21,7 @@ enum fixed_addresses { #define FIXADDR_START (FIXADDR_TOP - FIXADDR_SIZE) #define FIXMAP_PAGE_IO PAGE_KERNEL_SUC =20 +void __init early_fixmap_init(void); extern void __set_fixmap(enum fixed_addresses idx, phys_addr_t phys, pgprot_t flags); =20 diff --git a/arch/loongarch/kernel/setup.c b/arch/loongarch/kernel/setup.c index 56934fe58..368786952 100644 --- a/arch/loongarch/kernel/setup.c +++ b/arch/loongarch/kernel/setup.c @@ -597,6 +597,7 @@ void __init setup_arch(char **cmdline_p) memblock_init(); pagetable_init(); bootcmdline_init(cmdline_p); + early_fixmap_init(); parse_early_param(); reserve_initrd_mem(); =20 diff --git a/arch/loongarch/mm/init.c b/arch/loongarch/mm/init.c index 188b52bbb..68abb7bad 100644 --- a/arch/loongarch/mm/init.c +++ b/arch/loongarch/mm/init.c @@ -36,6 +36,109 @@ #include #include =20 +#define SPAN_NR_ENTRIES(vstart, vend, shift) \ + ((((vend) - 1) >> (shift)) - ((vstart) >> (shift)) + 1) + +#define NR_BM_PTE_TABLES \ + SPAN_NR_ENTRIES(FIXADDR_START, FIXADDR_TOP, PMD_SHIFT) +#define NR_BM_PMD_TABLES \ + SPAN_NR_ENTRIES(FIXADDR_START, FIXADDR_TOP, PUD_SHIFT) + +static_assert(NR_BM_PMD_TABLES =3D=3D 1); + +#define __BM_TABLE_IDX(addr, shift) \ + (((addr) >> (shift)) - (FIXADDR_START >> (shift))) + +#define BM_PTE_TABLE_IDX(addr) __BM_TABLE_IDX(addr, PMD_SHIFT) + +static pte_t bm_pte[NR_BM_PTE_TABLES][PTRS_PER_PTE] __page_aligned_bss; +static pmd_t bm_pmd[PTRS_PER_PMD] __page_aligned_bss __maybe_unused; +static pud_t bm_pud[PTRS_PER_PUD] __page_aligned_bss __maybe_unused; + +static inline pte_t *fixmap_pte(unsigned long addr) +{ + return &bm_pte[BM_PTE_TABLE_IDX(addr)][pte_index(addr)]; +} + +static void __init early_fixmap_init_pte(pmd_t *pmdp, unsigned long addr) +{ + pmd_t pmd =3D READ_ONCE(*pmdp); + pte_t *ptep; + + if (!pmd_present(pmd)) { + ptep =3D bm_pte[BM_PTE_TABLE_IDX(addr)]; + pmd_populate_kernel(&init_mm, pmdp, ptep); + kernel_pte_init(ptep); + } +} + +static void __init early_fixmap_init_pmd(pud_t *pudp, unsigned long addr, + unsigned long end) +{ + unsigned long next; + pud_t pud =3D READ_ONCE(*pudp); + pmd_t *pmdp; + + if (pud_none(pud)) + pud_populate(&init_mm, pudp, bm_pmd); + + pmdp =3D pmd_offset(pudp, addr); + +#ifndef __PAGETABLE_PMD_FOLDED + pmd_init(pmdp); +#endif + do { + next =3D pmd_addr_end(addr, end); + early_fixmap_init_pte(pmdp, addr); + } while (pmdp++, addr =3D next, addr !=3D end); +} + +static void __init early_fixmap_init_pud(p4d_t *p4dp, unsigned long addr, + unsigned long end) +{ + p4d_t p4d =3D READ_ONCE(*p4dp); + pud_t *pudp; + +#ifndef __PAGETABLE_PUD_FOLDED + if (CONFIG_PGTABLE_LEVELS > 3 && !p4d_none(p4d) && + p4d_phys(p4d) !=3D __pa_symbol(bm_pud)) { + /* + * We only end up here if the kernel mapping and the fixmap + * share the top level pgd entry, which should only happen on + * 16k/4 levels configurations. + */ + BUG_ON(!IS_ENABLED(CONFIG_PAGE_SIZE_16KB)); + } +#endif + + if (p4d_none(p4d)) + p4d_populate(&init_mm, p4dp, bm_pud); + + pudp =3D pud_offset(p4dp, addr); + +#ifndef __PAGETABLE_PUD_FOLDED + pud_init(pudp); +#endif + early_fixmap_init_pmd(pudp, addr, end); +} + +/* + * The p*d_populate functions call virt_to_phys implicitly so they can't b= e used + * directly on kernel symbols (bm_p*d). This function is called too early = to use + * lm_alias so __p*d_populate functions must be used to populate with the + * physical address from __pa_symbol. + */ +void __init early_fixmap_init(void) +{ + unsigned long addr =3D FIXADDR_START; + unsigned long end =3D FIXADDR_TOP; + + pgd_t *pgdp =3D pgd_offset_k(addr); + p4d_t *p4dp =3D p4d_offset(pgdp, addr); + + early_fixmap_init_pud(p4dp, addr, end); +} + unsigned long empty_zero_page[PAGE_SIZE / sizeof(unsigned long)] __page_al= igned_bss; EXPORT_SYMBOL(empty_zero_page); =20 @@ -209,7 +312,7 @@ pte_t * __init populate_kernel_pte(unsigned long addr) return pte_offset_kernel(pmd, addr); } =20 -void __init __set_fixmap(enum fixed_addresses idx, +void __set_fixmap(enum fixed_addresses idx, phys_addr_t phys, pgprot_t flags) { unsigned long addr =3D __fix_to_virt(idx); @@ -217,11 +320,7 @@ void __init __set_fixmap(enum fixed_addresses idx, =20 BUG_ON(idx <=3D FIX_HOLE || idx >=3D __end_of_fixed_addresses); =20 - ptep =3D populate_kernel_pte(addr); - if (!pte_none(ptep_get(ptep))) { - pte_ERROR(*ptep); - return; - } + ptep =3D fixmap_pte(addr); =20 if (pgprot_val(flags)) set_pte(ptep, pfn_pte(phys >> PAGE_SHIFT, flags)); --=20 2.25.1 From nobody Sat Oct 11 08:12:45 2025 Received: from mailgw.kylinos.cn (mailgw.kylinos.cn [124.126.103.232]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 95ED8226883 for ; Wed, 11 Jun 2025 05:36:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=124.126.103.232 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749620204; cv=none; b=mYpskglwW2UicT41Vorno2wegrFLa03pSVmct5R9ut7lr6Fa84hYFsnQi6uO6/ofXSLL9Q1GHHylLTvu/1deg6U4FQfX5y7UvtbB/nVkT1PqVC/xE3rgC9BwYbXxj6ygIdLGSz4if8EzBeZRmbYC+5YmMFouJNeV5VnxsSjXbmg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749620204; c=relaxed/simple; bh=6hil3jme6RmuqIKWHKQsZ2pdj9ZHIGCX1CiT7H80zY0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=P9Hd50+4LXCmx6NY1eItNlOU1dkuXYNldjfxPPf6Ir5Plel5d+ihSSE1OIS6PQyp3wyz8or98cp9f2wtDRJ9ifbtnfElYfa2Cl2dt8BvqXDMwnXtCXU9bbjO6anqjIVyC5Rq5lEEzUfdpt7obKueWHlYc0XS+QHrXahv43IepGs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kylinos.cn; spf=pass smtp.mailfrom=kylinos.cn; arc=none smtp.client-ip=124.126.103.232 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kylinos.cn Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kylinos.cn X-UUID: 09afac90468611f0b29709d653e92f7d-20250611 X-CTIC-Tags: HR_CC_COUNT, HR_CC_DOMAIN_COUNT, HR_CC_NAME, HR_CTE_8B, HR_CTT_MISS HR_DATE_H, HR_DATE_WKD, HR_DATE_ZONE, HR_FROM_NAME, HR_SJ_DIGIT_LEN HR_SJ_LANG, HR_SJ_LEN, HR_SJ_LETTER, HR_SJ_NOR_SYM, HR_SJ_PHRASE HR_SJ_PHRASE_LEN, HR_SJ_WS, HR_TO_COUNT, HR_TO_DOMAIN_COUNT, HR_TO_NO_NAME IP_UNTRUSTED, SRC_UNTRUSTED, IP_LOWREP, SRC_LOWREP, DN_TRUSTED SRC_TRUSTED, SA_EXISTED, SN_UNTRUSTED, SN_UNFAMILIAR, SPF_NOPASS DKIM_NOPASS, DMARC_NOPASS, CIE_BAD, CIE_GOOD, CIE_GOOD_SPF GTI_FG_BS, GTI_RG_INFO, GTI_C_BU, AMN_T1, AMN_GOOD AMN_C_TI, AMN_C_BU, ABX_MISS_RDNS X-CID-P-RULE: Release_Ham X-CID-O-INFO: VERSION:1.1.45,REQID:d0832168-f478-4a19-be73-fb229c5342f8,IP:10, URL:0,TC:0,Content:0,EDM:-25,RT:0,SF:-5,FILE:0,BULK:0,RULE:Release_Ham,ACT ION:release,TS:-20 X-CID-INFO: VERSION:1.1.45,REQID:d0832168-f478-4a19-be73-fb229c5342f8,IP:10,UR L:0,TC:0,Content:0,EDM:-25,RT:0,SF:-5,FILE:0,BULK:0,RULE:EDM_GE969F26,ACTI ON:release,TS:-20 X-CID-META: VersionHash:6493067,CLOUDID:7ee8bebc6a0d2c6ddbc33a2613e38ca8,BulkI D:250611133637GVWAGH1C,BulkQuantity:0,Recheck:0,SF:17|19|25|38|45|66|78|81 |82|102,TC:nil,Content:0|50,EDM:1,IP:-2,URL:0,File:nil,RT:nil,Bulk:nil,QS: nil,BEC:nil,COL:0,OSI:0,OSA:0,AV:0,LES:1,SPR:NO,DKR:0,DKP:0,BRR:0,BRE:0,AR C:0 X-CID-BVR: 0 X-CID-BAS: 0,_,0,_ X-CID-FACTOR: TF_CID_SPAM_SNR,TF_CID_SPAM_FAS,TF_CID_SPAM_FSD,TF_CID_SPAM_FSI X-UUID: 09afac90468611f0b29709d653e92f7d-20250611 X-User: duanchenghao@kylinos.cn Received: from localhost.localdomain [(223.104.40.103)] by mailgw.kylinos.cn (envelope-from ) (Generic MTA) with ESMTP id 1249612687; Wed, 11 Jun 2025 13:36:35 +0800 From: Chenghao Duan To: loongarch@lists.linux.dev, linux-kernel@vger.kernel.org Cc: Chenghao Duan , George Guo , Youling Tang Subject: [PATCH v1 2/5] LoongArch: BPF: The operation commands needed to add a trampoline Date: Wed, 11 Jun 2025 13:36:22 +0800 Message-Id: <20250611053625.352091-3-duanchenghao@kylinos.cn> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20250611053625.352091-1-duanchenghao@kylinos.cn> References: <20250611035952.111182-3-duanchenghao@kylinos.cn> <20250611053625.352091-1-duanchenghao@kylinos.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add instrctions: emit_NOP emit_BREAK Add branch jump function: larch_insn_gen_beq larch_insn_gen_bne Add instruction copy function: larch_insn_text_copy The implementation of larch_insn_text_copy uses the fixmap FIX_TEXT_POKE0. Signed-off-by: George Guo Signed-off-by: Youling Tang Signed-off-by: Chenghao Duan --- arch/loongarch/include/asm/inst.h | 19 +++++++ arch/loongarch/kernel/inst.c | 85 +++++++++++++++++++++++++++++++ 2 files changed, 104 insertions(+) diff --git a/arch/loongarch/include/asm/inst.h b/arch/loongarch/include/asm= /inst.h index 3089785ca..dd6e07781 100644 --- a/arch/loongarch/include/asm/inst.h +++ b/arch/loongarch/include/asm/inst.h @@ -497,6 +497,7 @@ void arch_simulate_insn(union loongarch_instruction ins= n, struct pt_regs *regs); int larch_insn_read(void *addr, u32 *insnp); int larch_insn_write(void *addr, u32 insn); int larch_insn_patch_text(void *addr, u32 insn); +int larch_insn_text_copy(void *dst, void *src, size_t len); =20 u32 larch_insn_gen_nop(void); u32 larch_insn_gen_b(unsigned long pc, unsigned long dest); @@ -511,6 +512,8 @@ u32 larch_insn_gen_lu12iw(enum loongarch_gpr rd, int im= m); u32 larch_insn_gen_lu32id(enum loongarch_gpr rd, int imm); u32 larch_insn_gen_lu52id(enum loongarch_gpr rd, enum loongarch_gpr rj, in= t imm); u32 larch_insn_gen_jirl(enum loongarch_gpr rd, enum loongarch_gpr rj, int = imm); +u32 larch_insn_gen_beq(enum loongarch_gpr rd, enum loongarch_gpr rj, int i= mm); +u32 larch_insn_gen_bne(enum loongarch_gpr rd, enum loongarch_gpr rj, int i= mm); =20 static inline bool signed_imm_check(long val, unsigned int bit) { @@ -778,6 +781,22 @@ static inline void emit_##NAME(union loongarch_instruc= tion *insn, \ =20 DEF_EMIT_REG3SA2_FORMAT(alsld, alsld_op) =20 +#define DEF_EMIT_NOP(NAME) \ +static inline void emit_##NAME(union loongarch_instruction *insn) \ +{ \ + insn->word =3D INSN_NOP; \ +} + +DEF_EMIT_NOP(NOP) + +#define DEF_EMIT_BREAK(NAME) \ +static inline void emit_##NAME(union loongarch_instruction *insn) \ +{ \ + insn->word =3D INSN_BREAK; \ +} + +DEF_EMIT_BREAK(BREAK) + struct pt_regs; =20 void emulate_load_store_insn(struct pt_regs *regs, void __user *addr, unsi= gned int *pc); diff --git a/arch/loongarch/kernel/inst.c b/arch/loongarch/kernel/inst.c index 14d7d700b..a47dc3575 100644 --- a/arch/loongarch/kernel/inst.c +++ b/arch/loongarch/kernel/inst.c @@ -10,6 +10,33 @@ =20 static DEFINE_RAW_SPINLOCK(patch_lock); =20 +static bool is_image_text(unsigned long addr) +{ + return core_kernel_text(addr); +} + +static void *patch_map(void *addr, int fixmap) +{ + unsigned long uintaddr =3D (uintptr_t)addr; + bool image =3D is_image_text(uintaddr); + struct page *page; + phys_addr_t phys; + + if (image) + phys =3D __pa_symbol(addr); + else { + page =3D vmalloc_to_page(addr); + phys =3D page_to_phys(page) + offset_in_page(addr); + } + + return (void *)set_fixmap_offset(fixmap, phys); +} + +static void patch_unmap(int fixmap) +{ + clear_fixmap(fixmap); +} + void simu_pc(struct pt_regs *regs, union loongarch_instruction insn) { unsigned long pc =3D regs->csr_era; @@ -218,6 +245,36 @@ int larch_insn_patch_text(void *addr, u32 insn) return ret; } =20 +int larch_insn_text_copy(void *dst, void *src, size_t len) +{ + unsigned long flags; + size_t wlen =3D 0; + size_t size; + void *waddr; + void *ptr; + int ret =3D 0; + + raw_spin_lock_irqsave(&patch_lock, flags); + while (wlen < len) { + ptr =3D dst + wlen; + size =3D min_t(size_t, PAGE_SIZE - offset_in_page(ptr), + len - wlen); + + waddr =3D patch_map(ptr, FIX_TEXT_POKE0); + ret =3D copy_to_kernel_nofault(waddr, src + wlen, size); + patch_unmap(FIX_TEXT_POKE0); + + if (ret) { + pr_err("%s: operation failed\n", __func__); + break; + } + wlen +=3D size; + } + raw_spin_unlock_irqrestore(&patch_lock, flags); + + return ret; +} + u32 larch_insn_gen_nop(void) { return INSN_NOP; @@ -336,3 +393,31 @@ u32 larch_insn_gen_jirl(enum loongarch_gpr rd, enum lo= ongarch_gpr rj, int imm) =20 return insn.word; } + +u32 larch_insn_gen_beq(enum loongarch_gpr rd, enum loongarch_gpr rj, int i= mm) +{ + union loongarch_instruction insn; + + if ((imm & 3) || imm < -SZ_128K || imm >=3D SZ_128K) { + pr_warn("The generated beq instruction is out of range.\n"); + return INSN_BREAK; + } + + emit_beq(&insn, rd, rj, imm >> 2); + + return insn.word; +} + +u32 larch_insn_gen_bne(enum loongarch_gpr rd, enum loongarch_gpr rj, int i= mm) +{ + union loongarch_instruction insn; + + if ((imm & 3) || imm < -SZ_128K || imm >=3D SZ_128K) { + pr_warn("The generated bne instruction is out of range.\n"); + return INSN_BREAK; + } + + emit_bne(&insn, rj, rd, imm >> 2); + + return insn.word; +} --=20 2.25.1 From nobody Sat Oct 11 08:12:45 2025 Received: from mailgw.kylinos.cn (mailgw.kylinos.cn [124.126.103.232]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6509A20102D for ; Wed, 11 Jun 2025 05:36:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=124.126.103.232 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749620205; cv=none; b=uRugBWPRI2wy0gxLqnNn3GhPlqdnQpCOZn0m2hYadxyH+87/z733L2Cc1sqRzyLvZ07FyFHtzq+yci8Wazvg7Lr0m7T9W9wIVwWRtN85iTjopjuS8UKo4QMRpI8WtpRyFpYNSX9KM3a2xL1eWba3EiIx0zm7GNZ9M+dF+m2y/HQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749620205; c=relaxed/simple; bh=9CSl90umod+66WIXaDcpwnyOGW72220eFIEQOqdjezg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=i5zOE20if4bN336jQloD6mt4ocySI/CPvaWfwMqQ1qnHFBtikluvSotfwvc8GNy2E6R11MGQF1QBxY2DCRjSo9NK+1AJnyP1aVadm8kMa9UnIGhtDYiHJQNnS2zK93RdW/dXZiNKoH+YPKQlBfNQbqPC6xXWgnBXi7turZb+hv0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kylinos.cn; spf=pass smtp.mailfrom=kylinos.cn; arc=none smtp.client-ip=124.126.103.232 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kylinos.cn Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kylinos.cn X-UUID: 0a208636468611f0b29709d653e92f7d-20250611 X-CTIC-Tags: HR_CC_COUNT, HR_CC_DOMAIN_COUNT, HR_CC_NAME, HR_CTE_8B, HR_CTT_MISS HR_DATE_H, HR_DATE_WKD, HR_DATE_ZONE, HR_FROM_NAME, HR_SJ_DIGIT_LEN HR_SJ_LANG, HR_SJ_LEN, HR_SJ_LETTER, HR_SJ_NOR_SYM, HR_SJ_PHRASE HR_SJ_PHRASE_LEN, HR_SJ_WS, HR_TO_COUNT, HR_TO_DOMAIN_COUNT, HR_TO_NO_NAME IP_UNTRUSTED, SRC_UNTRUSTED, IP_LOWREP, SRC_LOWREP, DN_TRUSTED SRC_TRUSTED, SA_EXISTED, SN_UNTRUSTED, SN_UNFAMILIAR, SPF_NOPASS DKIM_NOPASS, DMARC_NOPASS, CIE_BAD, CIE_GOOD, CIE_GOOD_SPF GTI_FG_BS, GTI_RG_INFO, GTI_C_BU, AMN_T1, AMN_GOOD AMN_C_TI, AMN_C_BU, ABX_MISS_RDNS X-CID-P-RULE: Release_Ham X-CID-O-INFO: VERSION:1.1.45,REQID:66bc2441-98d1-43bc-b80e-c82c20b30dd5,IP:10, URL:0,TC:0,Content:0,EDM:0,RT:0,SF:-5,FILE:0,BULK:0,RULE:Release_Ham,ACTIO N:release,TS:5 X-CID-INFO: VERSION:1.1.45,REQID:66bc2441-98d1-43bc-b80e-c82c20b30dd5,IP:10,UR L:0,TC:0,Content:0,EDM:0,RT:0,SF:-5,FILE:0,BULK:0,RULE:Release_Ham,ACTION: release,TS:5 X-CID-META: VersionHash:6493067,CLOUDID:6a6d1c745aa5a83c38564995450c95b6,BulkI D:2506111336373ZLSE6US,BulkQuantity:0,Recheck:0,SF:17|19|25|38|45|66|78|81 |82|102,TC:nil,Content:0|50,EDM:-3,IP:-2,URL:0,File:nil,RT:nil,Bulk:nil,QS :nil,BEC:nil,COL:0,OSI:0,OSA:0,AV:0,LES:1,SPR:NO,DKR:0,DKP:0,BRR:0,BRE:0,A RC:0 X-CID-BVR: 0 X-CID-BAS: 0,_,0,_ X-CID-FACTOR: TF_CID_SPAM_FAS,TF_CID_SPAM_FSD,TF_CID_SPAM_FSI,TF_CID_SPAM_SNR X-UUID: 0a208636468611f0b29709d653e92f7d-20250611 X-User: duanchenghao@kylinos.cn Received: from localhost.localdomain [(223.104.40.103)] by mailgw.kylinos.cn (envelope-from ) (Generic MTA) with ESMTP id 117676519; Wed, 11 Jun 2025 13:36:35 +0800 From: Chenghao Duan To: loongarch@lists.linux.dev, linux-kernel@vger.kernel.org Cc: Chenghao Duan , George Guo Subject: [PATCH v1 3/5] LoongArch: BPF: Add bpf_arch_text_poke support for Loongarch Date: Wed, 11 Jun 2025 13:36:23 +0800 Message-Id: <20250611053625.352091-4-duanchenghao@kylinos.cn> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20250611053625.352091-1-duanchenghao@kylinos.cn> References: <20250611035952.111182-3-duanchenghao@kylinos.cn> <20250611053625.352091-1-duanchenghao@kylinos.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Implement the bpf_arch_text_poke function for the LoongArch architecture. On LoongArch, since symbol addresses in the direct mapping region cannot be reached via relative jump instructions from the paged mapping region, we use the move_imm+jirl instruction pair as absolute jump instructions. These require 2-5 instructions, so we reserve 5 NOP instructions in the program as placeholders for function jumps. Signed-off-by: George Guo Signed-off-by: Chenghao Duan --- arch/loongarch/net/bpf_jit.c | 64 ++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) diff --git a/arch/loongarch/net/bpf_jit.c b/arch/loongarch/net/bpf_jit.c index ea357a3ed..0e31d4d66 100644 --- a/arch/loongarch/net/bpf_jit.c +++ b/arch/loongarch/net/bpf_jit.c @@ -4,6 +4,7 @@ * * Copyright (C) 2022 Loongson Technology Corporation Limited */ +#include #include "bpf_jit.h" =20 #define REG_TCC LOONGARCH_GPR_A6 @@ -1351,3 +1352,66 @@ bool bpf_jit_supports_subprog_tailcalls(void) { return true; } + +static int emit_jump_and_link(struct jit_ctx *ctx, u8 rd, u64 ip, u64 targ= et) +{ + s64 offset =3D (s64)(target - ip); + + if (offset && (offset >=3D -SZ_128M && offset < SZ_128M)) { + emit_insn(ctx, bl, offset >> 2); + } else { + move_imm(ctx, LOONGARCH_GPR_T1, target, false); + emit_insn(ctx, jirl, rd, LOONGARCH_GPR_T1, 0); + } + + return 0; +} + +static int gen_jump_or_nops(void *target, void *ip, u32 *insns, bool is_ca= ll) +{ + s64 off =3D 0; + struct jit_ctx ctx; + + ctx.idx =3D 0; + ctx.image =3D (union loongarch_instruction *)insns; + + if (!target) { + emit_insn((&ctx), NOP); + emit_insn((&ctx), NOP); + return 0; + } + + off =3D (s64)(target - ip); + return emit_jump_and_link(&ctx, is_call ? LOONGARCH_GPR_T0 : LOONGARCH_GP= R_ZERO, + (unsigned long)ip, (unsigned long)target); +} + +int bpf_arch_text_poke(void *ip, enum bpf_text_poke_type poke_type, + void *old_addr, void *new_addr) +{ + u32 old_insns[5] =3D {[0 ... 4] =3D INSN_NOP}; + u32 new_insns[5] =3D {[0 ... 4] =3D INSN_NOP}; + bool is_call =3D poke_type =3D=3D BPF_MOD_CALL; + int ret; + + if (!is_kernel_text((unsigned long)ip) && + !is_bpf_text_address((unsigned long)ip)) + return -ENOTSUPP; + + ret =3D gen_jump_or_nops(old_addr, ip, old_insns, is_call); + if (ret) + return ret; + + if (memcmp(ip, old_insns, 5 * 4)) + return -EFAULT; + + ret =3D gen_jump_or_nops(new_addr, ip, new_insns, is_call); + if (ret) + return ret; + + mutex_lock(&text_mutex); + if (memcmp(ip, new_insns, 5 * 4)) + ret =3D larch_insn_text_copy(ip, new_insns, 5 * 4); + mutex_unlock(&text_mutex); + return ret; +} --=20 2.25.1 From nobody Sat Oct 11 08:12:45 2025 Received: from mailgw.kylinos.cn (mailgw.kylinos.cn [124.126.103.232]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EAEAA225A23 for ; Wed, 11 Jun 2025 05:36:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=124.126.103.232 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749620204; cv=none; b=KsJyC3JywoQXiq5Rl1LG8o1xpghZ6u+owlpraPl/1xHAu4sLJLc+xyZqUBySglBvhO2MvdYe74dpnNODTtlV4ZRXDyck8Ad6q50iXYdak9/j2x0Riem36UdqFm1poIe6Ueo6PqdMPT9PZe0GdMQfTiQ0mogw6GHf0JtsQTXXKVY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749620204; c=relaxed/simple; bh=49AKq9ID6BBuWfyisxuAwvD/4ETkV0WHbvy5s/TxhHE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=WnexzCOsGRpvk14Scr4UY4BRJGZhFrjL/U7MGgBZ1bs4nCDabzxbCoXlb5uJ7fWh2YeEPANDU4eq8ax5kSug1Qcc27kbXj2rK0dxdsi6kbcKbsk5as5utbT7MyxzP7ygiyt40+0vrl2J8Gu4B6THyJl429Uu32ZoLj6D7nmqakw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kylinos.cn; spf=pass smtp.mailfrom=kylinos.cn; arc=none smtp.client-ip=124.126.103.232 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kylinos.cn Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kylinos.cn X-UUID: 0a9478f2468611f0b29709d653e92f7d-20250611 X-CTIC-Tags: HR_CC_COUNT, HR_CC_DOMAIN_COUNT, HR_CC_NAME, HR_CTE_8B, HR_CTT_MISS HR_DATE_H, HR_DATE_WKD, HR_DATE_ZONE, HR_FROM_NAME, HR_SJ_DIGIT_LEN HR_SJ_LANG, HR_SJ_LEN, HR_SJ_LETTER, HR_SJ_NOR_SYM, HR_SJ_PHRASE HR_SJ_PHRASE_LEN, HR_SJ_WS, HR_TO_COUNT, HR_TO_DOMAIN_COUNT, HR_TO_NO_NAME IP_UNTRUSTED, SRC_UNTRUSTED, IP_LOWREP, SRC_LOWREP, DN_TRUSTED SRC_TRUSTED, SA_EXISTED, SN_UNTRUSTED, SN_UNFAMILIAR, SPF_NOPASS DKIM_NOPASS, DMARC_NOPASS, CIE_BAD, CIE_GOOD, CIE_GOOD_SPF GTI_FG_BS, GTI_RG_INFO, GTI_C_BU, AMN_T1, AMN_GOOD AMN_C_TI, AMN_C_BU, ABX_MISS_RDNS X-CID-P-RULE: Release_Ham X-CID-O-INFO: VERSION:1.1.45,REQID:c964e10c-97a5-4e69-bff2-d52d016613c5,IP:10, URL:0,TC:0,Content:0,EDM:0,RT:0,SF:-5,FILE:0,BULK:0,RULE:Release_Ham,ACTIO N:release,TS:5 X-CID-INFO: VERSION:1.1.45,REQID:c964e10c-97a5-4e69-bff2-d52d016613c5,IP:10,UR L:0,TC:0,Content:0,EDM:0,RT:0,SF:-5,FILE:0,BULK:0,RULE:Release_Ham,ACTION: release,TS:5 X-CID-META: VersionHash:6493067,CLOUDID:8fc30d15678a98b10ffcd9ed8dc1fb57,BulkI D:2506111336373KLPVV1F,BulkQuantity:0,Recheck:0,SF:17|19|25|38|45|66|78|81 |82|102,TC:nil,Content:0|50,EDM:-3,IP:-2,URL:0,File:nil,RT:nil,Bulk:nil,QS :nil,BEC:nil,COL:0,OSI:0,OSA:0,AV:0,LES:1,SPR:NO,DKR:0,DKP:0,BRR:0,BRE:0,A RC:0 X-CID-BVR: 0 X-CID-BAS: 0,_,0,_ X-CID-FACTOR: TF_CID_SPAM_SNR,TF_CID_SPAM_FAS,TF_CID_SPAM_FSD,TF_CID_SPAM_FSI X-UUID: 0a9478f2468611f0b29709d653e92f7d-20250611 X-User: duanchenghao@kylinos.cn Received: from localhost.localdomain [(223.104.40.103)] by mailgw.kylinos.cn (envelope-from ) (Generic MTA) with ESMTP id 1579172774; Wed, 11 Jun 2025 13:36:36 +0800 From: Chenghao Duan To: loongarch@lists.linux.dev, linux-kernel@vger.kernel.org Cc: Chenghao Duan , George Guo Subject: [PATCH v1 4/5] LoongArch: BPF: Add bpf trampoline support for Loongarch Date: Wed, 11 Jun 2025 13:36:24 +0800 Message-Id: <20250611053625.352091-5-duanchenghao@kylinos.cn> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20250611053625.352091-1-duanchenghao@kylinos.cn> References: <20250611035952.111182-3-duanchenghao@kylinos.cn> <20250611053625.352091-1-duanchenghao@kylinos.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" BPF trampoline is the critical infrastructure of the BPF subsystem, acting as a mediator between kernel functions and BPF programs. Numerous important features, such as using BPF program for zero overhead kernel introspection, rely on this key component. We can't wait to support bpf trampoline on Loongarch. The related tests have passed, Including the following technical points: 1. fentry 2. fmod_ret 3. fexit Signed-off-by: George Guo Signed-off-by: Chenghao Duan --- arch/loongarch/net/bpf_jit.c | 423 +++++++++++++++++++++++++++++++++++ arch/loongarch/net/bpf_jit.h | 6 + 2 files changed, 429 insertions(+) diff --git a/arch/loongarch/net/bpf_jit.c b/arch/loongarch/net/bpf_jit.c index 0e31d4d66..cdcb73f93 100644 --- a/arch/loongarch/net/bpf_jit.c +++ b/arch/loongarch/net/bpf_jit.c @@ -7,6 +7,10 @@ #include #include "bpf_jit.h" =20 +#define LOONGARCH_MAX_REG_ARGS 8 +#define LOONGARCH_FENTRY_NINSNS 2 +#define LOONGARCH_FENTRY_NBYTES (LOONGARCH_FENTRY_NINSNS * 4) + #define REG_TCC LOONGARCH_GPR_A6 #define TCC_SAVED LOONGARCH_GPR_S5 =20 @@ -1386,6 +1390,20 @@ static int gen_jump_or_nops(void *target, void *ip, = u32 *insns, bool is_call) (unsigned long)ip, (unsigned long)target); } =20 +static int emit_call(struct jit_ctx *ctx, u64 addr) +{ + u64 ip; + s64 off =3D 0; + + if (addr && ctx->image && ctx->ro_image) { + ip =3D (u64)(ctx->image + ctx->idx); + off =3D addr - ip; + } + + return emit_jump_and_link(ctx, LOONGARCH_GPR_RA, ip, addr); +} + + int bpf_arch_text_poke(void *ip, enum bpf_text_poke_type poke_type, void *old_addr, void *new_addr) { @@ -1415,3 +1433,408 @@ int bpf_arch_text_poke(void *ip, enum bpf_text_poke= _type poke_type, mutex_unlock(&text_mutex); return ret; } + +static void store_args(struct jit_ctx *ctx, int nargs, int args_off) +{ + int i =3D 0; + + for (i =3D 0; i < nargs; i++) { + emit_insn(ctx, std, LOONGARCH_GPR_A0 + i, LOONGARCH_GPR_FP, -args_off); + args_off -=3D 8; + } +} + +static void restore_args(struct jit_ctx *ctx, int nargs, int args_off) +{ + int i =3D 0; + + for (i =3D 0; i < nargs; i++) { + emit_insn(ctx, ldd, LOONGARCH_GPR_A0 + i, LOONGARCH_GPR_FP, -args_off); + args_off -=3D 8; + } +} + +static int invoke_bpf_prog(struct jit_ctx *ctx, struct bpf_tramp_link *l, + int args_off, int retval_off, + int run_ctx_off, bool save_ret) +{ + int ret; + u32 *branch; + struct bpf_prog *p =3D l->link.prog; + int cookie_off =3D offsetof(struct bpf_tramp_run_ctx, bpf_cookie); + + if (l->cookie) { + move_imm(ctx, LOONGARCH_GPR_T1, l->cookie, false); + emit_insn(ctx, std, LOONGARCH_GPR_T1, LOONGARCH_GPR_FP, -run_ctx_off + c= ookie_off); + } else { + emit_insn(ctx, std, LOONGARCH_GPR_ZERO, LOONGARCH_GPR_FP, + -run_ctx_off + cookie_off); + } + + /* arg1: prog */ + move_imm(ctx, LOONGARCH_GPR_A0, (const s64)p, false); + /* arg2: &run_ctx */ + emit_insn(ctx, addid, LOONGARCH_GPR_A1, LOONGARCH_GPR_FP, -run_ctx_off); + ret =3D emit_call(ctx, (const u64)bpf_trampoline_enter(p)); + if (ret) + return ret; + + /* store prog start time */ + move_reg(ctx, LOONGARCH_GPR_S1, LOONGARCH_GPR_A0); + + /* if (__bpf_prog_enter(prog) =3D=3D 0) + * goto skip_exec_of_prog; + * + */ + branch =3D (u32 *)ctx->image + ctx->idx; + /* NOP reserved for conditional jump */ + emit_insn(ctx, NOP); + + /* arg1: &args_off */ + emit_insn(ctx, addid, LOONGARCH_GPR_A0, LOONGARCH_GPR_FP, -args_off); + if (!p->jited) + move_imm(ctx, LOONGARCH_GPR_A1, (const s64)p->insnsi, false); + ret =3D emit_call(ctx, (const u64)p->bpf_func); + if (ret) + return ret; + + if (save_ret) { + emit_insn(ctx, std, LOONGARCH_GPR_A0, LOONGARCH_GPR_FP, -retval_off); + emit_insn(ctx, std, regmap[BPF_REG_0], LOONGARCH_GPR_FP, -(retval_off - = 8)); + } + + /* update branch with beqz */ + if (ctx->image) { + int offset =3D (void *)(u32 *)(&ctx->image[ctx->idx]) - (void *)branch; + *branch =3D larch_insn_gen_beq(LOONGARCH_GPR_A0, LOONGARCH_GPR_ZERO, off= set); + } + + /* arg1: prog */ + move_imm(ctx, LOONGARCH_GPR_A0, (const s64)p, false); + /* arg2: prog start time */ + move_reg(ctx, LOONGARCH_GPR_A1, LOONGARCH_GPR_S1); + /* arg3: &run_ctx */ + emit_insn(ctx, addid, LOONGARCH_GPR_A2, LOONGARCH_GPR_FP, -run_ctx_off); + ret =3D emit_call(ctx, (const u64)bpf_trampoline_exit(p)); + + return ret; +} + +static void invoke_bpf_mod_ret(struct jit_ctx *ctx, struct bpf_tramp_links= *tl, + int args_off, int retval_off, int run_ctx_off, u32 **branches) +{ + int i; + + emit_insn(ctx, std, LOONGARCH_GPR_ZERO, LOONGARCH_GPR_FP, -retval_off); + for (i =3D 0; i < tl->nr_links; i++) { + invoke_bpf_prog(ctx, tl->links[i], args_off, retval_off, + run_ctx_off, true); + emit_insn(ctx, ldd, LOONGARCH_GPR_T1, LOONGARCH_GPR_FP, -retval_off); + branches[i] =3D (u32 *)ctx->image + ctx->idx; + emit_insn(ctx, NOP); + } +} + +void *bpf_arch_text_copy(void *dst, void *src, size_t len) +{ + if (larch_insn_text_copy(dst, src, len)) + return ERR_PTR(-EINVAL); + + return dst; +} + +u64 bpf_jit_alloc_exec_limit(void) +{ + return VMALLOC_END - VMALLOC_START; +} + +void *arch_alloc_bpf_trampoline(unsigned int size) +{ + void *image; + + if (WARN_ON_ONCE(size > PAGE_SIZE)) + return NULL; + image =3D bpf_jit_alloc_exec(PAGE_SIZE); + if (image) { + jit_fill_hole(image, size); + set_vm_flush_reset_perms(image); + } + + return image; +} + +void arch_free_bpf_trampoline(void *image, unsigned int size) +{ + WARN_ON_ONCE(size > PAGE_SIZE); + bpf_jit_free_exec(image); +} + +int arch_protect_bpf_trampoline(void *image, unsigned int size) +{ + return 0; +} + +int bpf_arch_text_invalidate(void *dst, size_t len) +{ + jit_fill_hole(dst, len); + return 0; +} + +static int __arch_prepare_bpf_trampoline(struct jit_ctx *ctx, struct bpf_t= ramp_image *im, + const struct btf_func_model *m, + struct bpf_tramp_links *tlinks, + void *func_addr, u32 flags) +{ + int i; + int stack_size =3D 0, nargs =3D 0; + int retval_off, args_off, nargs_off, ip_off, run_ctx_off, sreg_off; + struct bpf_tramp_links *fentry =3D &tlinks[BPF_TRAMP_FENTRY]; + struct bpf_tramp_links *fexit =3D &tlinks[BPF_TRAMP_FEXIT]; + struct bpf_tramp_links *fmod_ret =3D &tlinks[BPF_TRAMP_MODIFY_RETURN]; + int ret, save_ret; + void *orig_call =3D func_addr; + u32 **branches =3D NULL; + + if (flags & (BPF_TRAMP_F_ORIG_STACK | BPF_TRAMP_F_SHARE_IPMODIFY)) + return -ENOTSUPP; + + /* + * FP + 8 [ RA to parent func ] return address to parent + * function + * FP + 0 [ FP of parent func ] frame pointer of parent + * function + * FP - 8 [ T0 to traced func ] return address of traced + * function + * FP - 16 [ FP of traced func ] frame pointer of traced + * function + * + * FP - retval_off [ return value ] BPF_TRAMP_F_CALL_ORIG or + * BPF_TRAMP_F_RET_FENTRY_RET + * [ argN ] + * [ ... ] + * FP - args_off [ arg1 ] + * + * FP - nargs_off [ regs count ] + * + * FP - ip_off [ traced func ] BPF_TRAMP_F_IP_ARG + * + * FP - run_ctx_off [ bpf_tramp_run_ctx ] + * + * FP - sreg_off [ callee saved reg ] + * + */ + + if (m->nr_args > LOONGARCH_MAX_REG_ARGS) + return -ENOTSUPP; + + if (flags & (BPF_TRAMP_F_ORIG_STACK | BPF_TRAMP_F_SHARE_IPMODIFY)) + return -ENOTSUPP; + + stack_size =3D 0; + + /* room of trampoline frame to store return address and frame pointer */ + stack_size +=3D 16; + + save_ret =3D flags & (BPF_TRAMP_F_CALL_ORIG | BPF_TRAMP_F_RET_FENTRY_RET); + if (save_ret) { + /* Save BPF R0 and A0 */ + stack_size +=3D 16; + retval_off =3D stack_size; + } + + /* room of trampoline frame to store args */ + nargs =3D m->nr_args; + stack_size +=3D nargs * 8; + args_off =3D stack_size; + /* room of trampoline frame to store args number */ + stack_size +=3D 8; + nargs_off =3D stack_size; + + /* room of trampoline frame to store ip address */ + if (flags & BPF_TRAMP_F_IP_ARG) { + stack_size +=3D 8; + ip_off =3D stack_size; + } + + /* room of trampoline frame to store struct bpf_tramp_run_ctx */ + stack_size +=3D round_up(sizeof(struct bpf_tramp_run_ctx), 8); + run_ctx_off =3D stack_size; + + stack_size +=3D 8; + sreg_off =3D stack_size; + + stack_size =3D round_up(stack_size, 16); + + /* For the trampoline called from function entry */ + /* RA and FP for parent function*/ + emit_insn(ctx, addid, LOONGARCH_GPR_SP, LOONGARCH_GPR_SP, -16); + emit_insn(ctx, std, LOONGARCH_GPR_RA, LOONGARCH_GPR_SP, 8); + emit_insn(ctx, std, LOONGARCH_GPR_FP, LOONGARCH_GPR_SP, 0); + emit_insn(ctx, addid, LOONGARCH_GPR_FP, LOONGARCH_GPR_SP, 16); + + /* RA and FP for traced function*/ + emit_insn(ctx, addid, LOONGARCH_GPR_SP, LOONGARCH_GPR_SP, -stack_size); + emit_insn(ctx, std, LOONGARCH_GPR_T0, LOONGARCH_GPR_SP, stack_size - 8); + emit_insn(ctx, std, LOONGARCH_GPR_FP, LOONGARCH_GPR_SP, stack_size - 16); + emit_insn(ctx, addid, LOONGARCH_GPR_FP, LOONGARCH_GPR_SP, stack_size); + + /* callee saved register S1 to pass start time */ + emit_insn(ctx, std, LOONGARCH_GPR_S1, LOONGARCH_GPR_FP, -sreg_off); + + /* store ip address of the traced function */ + if (flags & BPF_TRAMP_F_IP_ARG) { + move_imm(ctx, LOONGARCH_GPR_T1, (const s64)func_addr, false); + emit_insn(ctx, std, LOONGARCH_GPR_T1, LOONGARCH_GPR_FP, -ip_off); + } + + /* store nargs number*/ + move_imm(ctx, LOONGARCH_GPR_T1, nargs, false); + emit_insn(ctx, std, LOONGARCH_GPR_T1, LOONGARCH_GPR_FP, -nargs_off); + + store_args(ctx, nargs, args_off); + + /* To traced function */ + orig_call +=3D LOONGARCH_FENTRY_NBYTES; + if (flags & BPF_TRAMP_F_CALL_ORIG) { + move_imm(ctx, LOONGARCH_GPR_A0, (const s64)im, false); + ret =3D emit_call(ctx, (const u64)__bpf_tramp_enter); + if (ret) + return ret; + } + + for (i =3D 0; i < fentry->nr_links; i++) { + ret =3D invoke_bpf_prog(ctx, fentry->links[i], args_off, retval_off, + run_ctx_off, flags & BPF_TRAMP_F_RET_FENTRY_RET); + if (ret) + return ret; + } + + if (fmod_ret->nr_links) { + branches =3D kcalloc(fmod_ret->nr_links, sizeof(u32 *), GFP_KERNEL); + if (!branches) + return -ENOMEM; + + invoke_bpf_mod_ret(ctx, fmod_ret, args_off, retval_off, + run_ctx_off, branches); + } + + if (flags & BPF_TRAMP_F_CALL_ORIG) { + restore_args(ctx, m->nr_args, args_off); + ret =3D emit_call(ctx, (const u64)orig_call); + if (ret) + goto out; + emit_insn(ctx, std, LOONGARCH_GPR_A0, LOONGARCH_GPR_FP, -retval_off); + emit_insn(ctx, std, regmap[BPF_REG_0], LOONGARCH_GPR_FP, -(retval_off - = 8)); + im->ip_after_call =3D ctx->ro_image + ctx->idx; + /* Reserve space for the move_imm + jirl instruction */ + emit_insn(ctx, NOP); + emit_insn(ctx, NOP); + emit_insn(ctx, NOP); + emit_insn(ctx, NOP); + emit_insn(ctx, NOP); + } + + for (i =3D 0; ctx->image && i < fmod_ret->nr_links; i++) { + int offset =3D (u32 *)&ctx->image[ctx->idx] - branches[i]; + *branches[i] =3D larch_insn_gen_bne(LOONGARCH_GPR_T1, LOONGARCH_GPR_ZERO= , offset); + } + + for (i =3D 0; i < fexit->nr_links; i++) { + ret =3D invoke_bpf_prog(ctx, fexit->links[i], args_off, retval_off, + run_ctx_off, false); + if (ret) + goto out; + } + + if (flags & BPF_TRAMP_F_CALL_ORIG) { + im->ip_epilogue =3D ctx->ro_image + ctx->idx; + move_imm(ctx, LOONGARCH_GPR_A0, (const s64)im, false); + ret =3D emit_call(ctx, (const u64)__bpf_tramp_exit); + if (ret) + goto out; + } + + if (flags & BPF_TRAMP_F_RESTORE_REGS) + restore_args(ctx, m->nr_args, args_off); + + if (save_ret) { + emit_insn(ctx, ldd, LOONGARCH_GPR_A0, LOONGARCH_GPR_FP, -retval_off); + emit_insn(ctx, ldd, regmap[BPF_REG_0], LOONGARCH_GPR_FP, -(retval_off - = 8)); + } + + emit_insn(ctx, ldd, LOONGARCH_GPR_S1, LOONGARCH_GPR_FP, -sreg_off); + + /* trampoline called from function entry */ + emit_insn(ctx, ldd, LOONGARCH_GPR_T0, LOONGARCH_GPR_SP, stack_size - 8); + emit_insn(ctx, ldd, LOONGARCH_GPR_FP, LOONGARCH_GPR_SP, stack_size - 16); + emit_insn(ctx, addid, LOONGARCH_GPR_SP, LOONGARCH_GPR_SP, stack_size); + + emit_insn(ctx, ldd, LOONGARCH_GPR_RA, LOONGARCH_GPR_SP, 8); + emit_insn(ctx, ldd, LOONGARCH_GPR_FP, LOONGARCH_GPR_SP, 0); + emit_insn(ctx, addid, LOONGARCH_GPR_SP, LOONGARCH_GPR_SP, 16); + + if (flags & BPF_TRAMP_F_SKIP_FRAME) + /* return to parent function */ + emit_insn(ctx, jirl, LOONGARCH_GPR_ZERO, LOONGARCH_GPR_RA, 0); + else + /* return to traced function */ + emit_insn(ctx, jirl, LOONGARCH_GPR_ZERO, LOONGARCH_GPR_T0, 0); + + ret =3D ctx->idx; +out: + kfree(branches); + + return ret; +} + +int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *ro_image, + void *ro_image_end, const struct btf_func_model *m, + u32 flags, struct bpf_tramp_links *tlinks, + void *func_addr) +{ + int ret; + void *image, *tmp; + u32 size =3D ro_image_end - ro_image; + + image =3D kvmalloc(size, GFP_KERNEL); + if (!image) + return -ENOMEM; + + struct jit_ctx ctx =3D { + .image =3D (union loongarch_instruction *)image, + .ro_image =3D (union loongarch_instruction *)ro_image, + .idx =3D 0, + }; + + jit_fill_hole(image, (unsigned int)(ro_image_end - ro_image)); + ret =3D __arch_prepare_bpf_trampoline(&ctx, im, m, tlinks, func_addr, fla= gs); + if (ret > 0 && validate_code(&ctx) < 0) { + ret =3D -EINVAL; + goto out; + } + + tmp =3D bpf_arch_text_copy(ro_image, image, size); + if (IS_ERR(tmp)) { + ret =3D PTR_ERR(tmp); + goto out; + } + + bpf_flush_icache(ro_image, ro_image_end); +out: + kvfree(image); + return ret < 0 ? ret : size; +} + +int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags, + struct bpf_tramp_links *tlinks, void *func_addr) +{ + struct bpf_tramp_image im; + struct jit_ctx ctx; + int ret; + + ctx.image =3D NULL; + ctx.idx =3D 0; + + ret =3D __arch_prepare_bpf_trampoline(&ctx, &im, m, tlinks, func_addr, fl= ags); + return ret < 0 ? ret : ret * LOONGARCH_INSN_SIZE; +} diff --git a/arch/loongarch/net/bpf_jit.h b/arch/loongarch/net/bpf_jit.h index 68586338e..51a12b809 100644 --- a/arch/loongarch/net/bpf_jit.h +++ b/arch/loongarch/net/bpf_jit.h @@ -18,6 +18,7 @@ struct jit_ctx { u32 *offset; int num_exentries; union loongarch_instruction *image; + union loongarch_instruction *ro_image; u32 stack_size; }; =20 @@ -303,3 +304,8 @@ static inline int emit_tailcall_jmp(struct jit_ctx *ctx= , u8 cond, enum loongarch =20 return -EINVAL; } + +static inline void bpf_flush_icache(void *start, void *end) +{ + flush_icache_range((unsigned long)start, (unsigned long)end); +} --=20 2.25.1 From nobody Sat Oct 11 08:12:45 2025 Received: from mailgw.kylinos.cn (mailgw.kylinos.cn [124.126.103.232]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4F6C6227B81 for ; Wed, 11 Jun 2025 05:36:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=124.126.103.232 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749620206; cv=none; b=LewvN1OXmB9ax8bVrfiKSwGM9yxBl6uaNtjUJQWtO88mYD/g7gNynllfEXIZlf8wxhSDxtVObwdq+KY53qSss5M2MVAjqM4J9YdEWKWdXKOq6VfoXGfEcm0U5LJkVzIO2//oABZJnalaE/96losm9Qzw+LwhGZIj13YmGyY2itE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749620206; c=relaxed/simple; bh=/HowC9kFKFSbycgjpOeF3oDREZHhIxzYt6lJ4cBZjNI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Y2+6zXFcQM+IESbTJl6aXhF9DzmPiM4KsLfXVg0wjizMq/t6HcUA+qZX97K0wbt06wDLzGt9qgaP2ZdtfR6+peJ/J8Xh5gGgCqtdV4voNhzPVVb96WZ0bmDH9Xsv+a/gUjPe+I1U6zq4a4E1ta06uYHO1gddUyYxyvT9OULze98= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kylinos.cn; spf=pass smtp.mailfrom=kylinos.cn; arc=none smtp.client-ip=124.126.103.232 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kylinos.cn Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kylinos.cn X-UUID: 0b087856468611f0b29709d653e92f7d-20250611 X-CTIC-Tags: HR_CC_COUNT, HR_CC_DOMAIN_COUNT, HR_CC_NAME, HR_CTE_8B, HR_CTT_MISS HR_DATE_H, HR_DATE_WKD, HR_DATE_ZONE, HR_FROM_NAME, HR_SJ_DIGIT_LEN HR_SJ_LANG, HR_SJ_LEN, HR_SJ_LETTER, HR_SJ_NOR_SYM, HR_SJ_PHRASE HR_SJ_PHRASE_LEN, HR_SJ_WS, HR_TO_COUNT, HR_TO_DOMAIN_COUNT, HR_TO_NO_NAME IP_UNTRUSTED, SRC_UNTRUSTED, IP_LOWREP, SRC_LOWREP, DN_TRUSTED SRC_TRUSTED, SA_EXISTED, SN_UNTRUSTED, SN_UNFAMILIAR, SPF_NOPASS DKIM_NOPASS, DMARC_NOPASS, CIE_BAD, CIE_GOOD, CIE_GOOD_SPF GTI_FG_BS, GTI_RG_INFO, GTI_C_BU, AMN_T1, AMN_GOOD AMN_C_TI, AMN_C_BU, ABX_MISS_RDNS X-CID-P-RULE: Release_Ham X-CID-O-INFO: VERSION:1.1.45,REQID:1acf6f43-1cc6-4e0e-bd89-499e73413c62,IP:10, URL:0,TC:0,Content:0,EDM:0,RT:0,SF:-5,FILE:0,BULK:0,RULE:Release_Ham,ACTIO N:release,TS:5 X-CID-INFO: VERSION:1.1.45,REQID:1acf6f43-1cc6-4e0e-bd89-499e73413c62,IP:10,UR L:0,TC:0,Content:0,EDM:0,RT:0,SF:-5,FILE:0,BULK:0,RULE:Release_Ham,ACTION: release,TS:5 X-CID-META: VersionHash:6493067,CLOUDID:715a3887ef34a629ff90b7f74980c6ac,BulkI D:250611133640944RVAVP,BulkQuantity:0,Recheck:0,SF:17|19|25|38|45|66|78|81 |82|102,TC:nil,Content:0|50,EDM:-3,IP:-2,URL:0,File:nil,RT:nil,Bulk:nil,QS :nil,BEC:nil,COL:0,OSI:0,OSA:0,AV:0,LES:1,SPR:NO,DKR:0,DKP:0,BRR:0,BRE:0,A RC:0 X-CID-BVR: 0,NGT X-CID-BAS: 0,NGT,0,_ X-CID-FACTOR: TF_CID_SPAM_SNR,TF_CID_SPAM_FAS,TF_CID_SPAM_FSD,TF_CID_SPAM_FSI X-UUID: 0b087856468611f0b29709d653e92f7d-20250611 X-User: duanchenghao@kylinos.cn Received: from localhost.localdomain [(223.104.40.103)] by mailgw.kylinos.cn (envelope-from ) (Generic MTA) with ESMTP id 1664631378; Wed, 11 Jun 2025 13:36:37 +0800 From: Chenghao Duan To: loongarch@lists.linux.dev, linux-kernel@vger.kernel.org Cc: Chenghao Duan , George Guo Subject: [PATCH v1 5/5] LoongArch: BPF: Update the code to rename validate_code to validate_ctx. Date: Wed, 11 Jun 2025 13:36:25 +0800 Message-Id: <20250611053625.352091-6-duanchenghao@kylinos.cn> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20250611053625.352091-1-duanchenghao@kylinos.cn> References: <20250611035952.111182-3-duanchenghao@kylinos.cn> <20250611053625.352091-1-duanchenghao@kylinos.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Update the code to rename validate_code to validate_ctx. validate_code is used to check the validity of code. validate_ctx is used to check both code validity and table entry correctness. Signed-off-by: George Guo Signed-off-by: Chenghao Duan --- arch/loongarch/net/bpf_jit.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/arch/loongarch/net/bpf_jit.c b/arch/loongarch/net/bpf_jit.c index cdcb73f93..ee38a9824 100644 --- a/arch/loongarch/net/bpf_jit.c +++ b/arch/loongarch/net/bpf_jit.c @@ -1177,6 +1177,14 @@ static int validate_code(struct jit_ctx *ctx) return -1; } =20 + return 0; +} + +static int validate_ctx(struct jit_ctx *ctx) +{ + if (validate_code(ctx)) + return -1; + if (WARN_ON_ONCE(ctx->num_exentries !=3D ctx->prog->aux->num_exentries)) return -1; =20 @@ -1285,7 +1293,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog = *prog) build_epilogue(&ctx); =20 /* 3. Extra pass to validate JITed code */ - if (validate_code(&ctx)) { + if (validate_ctx(&ctx)) { bpf_jit_binary_free(header); prog =3D orig_prog; goto out_offset; --=20 2.25.1