From nobody Sun Feb 8 10:34:00 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC80EC7EE29 for ; Fri, 9 Jun 2023 18:37:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231534AbjFIShI (ORCPT ); Fri, 9 Jun 2023 14:37:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49670 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231479AbjFIShC (ORCPT ); Fri, 9 Jun 2023 14:37:02 -0400 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A660035B5 for ; Fri, 9 Jun 2023 11:36:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1686335819; x=1717871819; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=0EsEMi/iv7xFak5m4YpFIXQtablbSyEfe89CDk9g/WM=; b=g+7cW+k5VxytIm8zCD28lq3kcnhJSDE99k4k/Mzeb2NG/99AWjlhUPuV H0VZkTRd4jvFJySXZoqtWGDcX0iqkdRH9tcXkRjXHWC0Sv7x8b3j6Ludf yntYtGsnQlduv7Y+eLFghXfW6dRSkJb0S04zJ9kmCkV3XGtK2EoKHVIa+ kabGmP7j5tyoICGU3OoJqT2ixxQrnCQ7aMcwDKvJV1gGVTRyeODj5jC6+ DY8IZu154Yy3fcmZkRDanS3DxzX5ADbgJkP7relablDorbo/sdgrPpaeu z3nCUllFcWY8HyUxkkFya6P/Do3HPNFLI8/8ynYSWjLOs27hmkT0l7z+R g==; X-IronPort-AV: E=McAfee;i="6600,9927,10736"; a="338022083" X-IronPort-AV: E=Sophos;i="6.00,230,1681196400"; d="scan'208";a="338022083" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Jun 2023 11:36:59 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10736"; a="710443951" X-IronPort-AV: E=Sophos;i="6.00,230,1681196400"; d="scan'208";a="710443951" Received: from black.fi.intel.com (HELO black.fi.intel.com.) ([10.237.72.28]) by orsmga002.jf.intel.com with ESMTP; 09 Jun 2023 11:36:55 -0700 From: Alexander Shishkin To: linux-kernel@vger.kernel.org, x86@kernel.org, Andy Lutomirski , Dave Hansen , Ravi Shankar , Tony Luck , Sohil Mehta Cc: Peter Zijlstra , Alexander Shishkin Subject: [PATCH v3 02/12] x86/asm: Introduce inline memcpy and memset Date: Fri, 9 Jun 2023 21:36:22 +0300 Message-Id: <20230609183632.48706-3-alexander.shishkin@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230609183632.48706-1-alexander.shishkin@linux.intel.com> References: <20230609183632.48706-1-alexander.shishkin@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Peter Zijlstra Provide inline memcpy and memset functions that can be used instead of the GCC builtins whenever necessary. Code posted by Peter Zijlstra . Link: https://lore.kernel.org/lkml/Y759AJ%2F0N9fqwDED@hirez.programming.kic= ks-ass.net/ [Missing Signed-off-by from PeterZ] Signed-off-by: Sohil Mehta Signed-off-by: Alexander Shishkin --- arch/x86/include/asm/string_32.h | 21 +++++++++++++++++++++ arch/x86/include/asm/string_64.h | 21 +++++++++++++++++++++ 2 files changed, 42 insertions(+) diff --git a/arch/x86/include/asm/string_32.h b/arch/x86/include/asm/string= _32.h index 32c0d981a82a..8896270e5eda 100644 --- a/arch/x86/include/asm/string_32.h +++ b/arch/x86/include/asm/string_32.h @@ -151,6 +151,16 @@ extern void *memcpy(void *, const void *, size_t); =20 #endif /* !CONFIG_FORTIFY_SOURCE */ =20 +static __always_inline void *__inline_memcpy(void *to, const void *from, s= ize_t len) +{ + void *ret =3D to; + + asm volatile("rep movsb" + : "+D" (to), "+S" (from), "+c" (len) + : : "memory"); + return ret; +} + #define __HAVE_ARCH_MEMMOVE void *memmove(void *dest, const void *src, size_t n); =20 @@ -195,6 +205,17 @@ extern void *memset(void *, int, size_t); #define memset(s, c, count) __builtin_memset(s, c, count) #endif /* !CONFIG_FORTIFY_SOURCE */ =20 +static __always_inline void *__inline_memset(void *s, int v, size_t n) +{ + void *ret =3D s; + + asm volatile("rep stosb" + : "+D" (s), "+c" (n) + : "a" ((uint8_t)v) + : "memory"); + return ret; +} + #define __HAVE_ARCH_MEMSET16 static inline void *memset16(uint16_t *s, uint16_t v, size_t n) { diff --git a/arch/x86/include/asm/string_64.h b/arch/x86/include/asm/string= _64.h index 857d364b9888..ea51e2d73265 100644 --- a/arch/x86/include/asm/string_64.h +++ b/arch/x86/include/asm/string_64.h @@ -18,10 +18,31 @@ extern void *memcpy(void *to, const void *from, size_t len); extern void *__memcpy(void *to, const void *from, size_t len); =20 +static __always_inline void *__inline_memcpy(void *to, const void *from, s= ize_t len) +{ + void *ret =3D to; + + asm volatile("rep movsb" + : "+D" (to), "+S" (from), "+c" (len) + : : "memory"); + return ret; +} + #define __HAVE_ARCH_MEMSET void *memset(void *s, int c, size_t n); void *__memset(void *s, int c, size_t n); =20 +static __always_inline void *__inline_memset(void *s, int v, size_t n) +{ + void *ret =3D s; + + asm volatile("rep stosb" + : "+D" (s), "+c" (n) + : "a" ((uint8_t)v) + : "memory"); + return ret; +} + /* * KMSAN needs to instrument as much code as possible. Use C versions of * memsetXX() from lib/string.c under KMSAN. --=20 2.39.2