From nobody Wed Apr 29 08:07:43 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 88DD1C433EF for ; Sun, 12 Jun 2022 09:00:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235311AbiFLJAs (ORCPT ); Sun, 12 Jun 2022 05:00:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38994 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235218AbiFLJAh (ORCPT ); Sun, 12 Jun 2022 05:00:37 -0400 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C012549F3E for ; Sun, 12 Jun 2022 02:00:35 -0700 (PDT) Received: by mail-wr1-x429.google.com with SMTP id a15so3704917wrh.2 for ; Sun, 12 Jun 2022 02:00:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=MH4Lv8K62JKUOJxVHdgE/oEdWi4lVPPJtSl5KwJG+1g=; b=p23NZpYkNV6Mw0OmtGi312n7HxeU9JJ/KfuYg25OWavQonbuxP2PlxFs7ZNZZmpppx e8oH+SylS9UYulExeaAjCPDHKx7nTbs1/SxiYfEWenUQ605TBPwdT41rY6WtmUWw4NTE mJsDBZYyuxn9oqyK0A9GWbS6xXQ3e+sQCMySI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=MH4Lv8K62JKUOJxVHdgE/oEdWi4lVPPJtSl5KwJG+1g=; b=O3D8XrJpvhkQbLEAy/DgLH0GOSr9s4jAY63yYIbChqrQzKdnAAK3r5MOEnfJYv3HaH M2oMKHoqzeswjDw5pMoRzgcJ/7eRqdGMoEIe0JwOkNHOlrCGkltIZlfp6pfHdlUkNMX7 e/jwZUvX0ycU/Dtr0gy8F90D7+uwipW2Qpq3EcEvoMKJHhtN+0W2JnQKduj6azwLGgG6 viAtoRyP2aqLshQ/MS33axvY+gZ+1y/sC+TW9n/Om0tqH7utQd5HIyomKZhapww33xSr /I4+Z2wEwhruhJtUvBJX9W2Dyu/zpKAt3KJZPc7EyFE+hK3C9fgWL4B02flGBng+/XKZ sQmQ== X-Gm-Message-State: AOAM5324h4Xpqz3z1caIW6EwqKid8qc0cBo93WNDkK9FD8/1bUojc9O7 rwdHxcWO1ZQdivsMkjmADXJABw== X-Google-Smtp-Source: ABdhPJzPfVvDi7oTmyqgIAHJ2AcFuN31YYqfCHQZc8+YvM1Z23odoVoyM/EwdMd9H9m+5olvSCaDUQ== X-Received: by 2002:a5d:4c49:0:b0:210:353c:1c91 with SMTP id n9-20020a5d4c49000000b00210353c1c91mr52527060wrt.159.1655024434317; Sun, 12 Jun 2022 02:00:34 -0700 (PDT) Received: from localhost.localdomain ([178.130.153.185]) by smtp.gmail.com with ESMTPSA id d34-20020a05600c4c2200b0039c5b4ab1b0sm4798603wmp.48.2022.06.12.02.00.31 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 12 Jun 2022 02:00:32 -0700 (PDT) From: Joe Damato To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , linux-kernel@vger.kernel.org Cc: Joe Damato Subject: [RFC,x86 v2 1/8] arch, x86, uaccess: Add nontemporal copy functions Date: Sun, 12 Jun 2022 01:57:50 -0700 Message-Id: <1655024280-23827-2-git-send-email-jdamato@fastly.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1655024280-23827-1-git-send-email-jdamato@fastly.com> References: <1655024280-23827-1-git-send-email-jdamato@fastly.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Add a generic non-temporal wrapper to uaccess which can be overridden by arches that support non-temporal copies. An implementation is added for x86 which wraps an existing non-temporal copy in the kernel. Signed-off-by: Joe Damato --- arch/x86/include/asm/uaccess_64.h | 6 ++++++ include/linux/uaccess.h | 6 ++++++ 2 files changed, 12 insertions(+) diff --git a/arch/x86/include/asm/uaccess_64.h b/arch/x86/include/asm/uacce= ss_64.h index 45697e0..ed41dba 100644 --- a/arch/x86/include/asm/uaccess_64.h +++ b/arch/x86/include/asm/uaccess_64.h @@ -65,6 +65,12 @@ extern long __copy_user_flushcache(void *dst, const void= __user *src, unsigned s extern void memcpy_page_flushcache(char *to, struct page *page, size_t off= set, size_t len); =20 +static inline unsigned long +__copy_from_user_nocache(void *dst, const void __user *src, unsigned long = size) +{ + return (unsigned long)__copy_user_nocache(dst, src, (unsigned int) size, = 0); +} + static inline int __copy_from_user_inatomic_nocache(void *dst, const void __user *src, unsigned size) diff --git a/include/linux/uaccess.h b/include/linux/uaccess.h index 5a328cf..6612c37 100644 --- a/include/linux/uaccess.h +++ b/include/linux/uaccess.h @@ -256,6 +256,12 @@ static inline size_t probe_subpage_writeable(char __us= er *uaddr, size_t size) #ifndef ARCH_HAS_NOCACHE_UACCESS =20 static inline __must_check unsigned long +__copy_from_user_nocache(void *to, const void __user *from, unsigned long = n) +{ + return __copy_from_user(to, from, n); +} + +static inline __must_check unsigned long __copy_from_user_inatomic_nocache(void *to, const void __user *from, unsigned long n) { --=20 2.7.4 From nobody Wed Apr 29 08:07:43 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7B84AC43334 for ; Sun, 12 Jun 2022 09:00:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235258AbiFLJAu (ORCPT ); Sun, 12 Jun 2022 05:00:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39814 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235233AbiFLJAo (ORCPT ); Sun, 12 Jun 2022 05:00:44 -0400 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 993B4517FF for ; Sun, 12 Jun 2022 02:00:37 -0700 (PDT) Received: by mail-wr1-x429.google.com with SMTP id a15so3704917wrh.2 for ; Sun, 12 Jun 2022 02:00:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=DtboLJnQWlaQx3cZn/5rtwDm8UnHa+IXUF0Drf8tf3c=; b=cr93iVQUbdBVCRI6c2GBMIxrKddY7Fx5VpjMCKS70SP0k7Oaz4l5ex5HXzKO7ThBgL 8qaWw+ktuAEQxYMG1k3ZIbbZpJ2ZqhRKTj+L0lGBnbYTL3blvzDvdZqscET7qs3md+q4 vwhG7ShyCieA5DF/O4ym8nSdvXFeSa2wDJ9VU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=DtboLJnQWlaQx3cZn/5rtwDm8UnHa+IXUF0Drf8tf3c=; b=8LUscIJAucslQiYNiBD7BLCfDCqsmYEFvOrhTrCS7k7ZrwUsAgsP1lX29HkFjlZcIS ait0wrVLRl2hT6boOO/u0D0mjIz7V+D0Y5Jo7h5P75uR2CAx0KBUqb+TU2WtmToESbdh S7+cYNwUxcC18jBUHupxGvp21SI/e2l9ixYKa7i7WKjTMGbyE04slGhVQsp7MNWnBmlq FX/d/tucHMFHLIrYlQsq35EgzpTHNTJV6g1alGv3nI3isGGqunsNwsYfxYmcJT5rMs/G uiqZ9xHtAUukVenFVPqC/Ismn1yba2uScS/QPuWZqc75YLS3nUdwWa7X54dqVh/Xbbx3 YF8g== X-Gm-Message-State: AOAM5329nrrooR8KWMckzpwHnMxqL6yZ2onbayodaCcsMp5Pqg51gdgX /HrAAzsDxRgFpYC4baHz+0zJIQ== X-Google-Smtp-Source: ABdhPJzOh6DJzmbgNNA/kSYpX0YaIat9g7wOlQtREkJUQ3J+bznP3DbPy7OfehfCY6oDmCLaRHJJ4g== X-Received: by 2002:a5d:64ac:0:b0:211:7f3b:a0d4 with SMTP id m12-20020a5d64ac000000b002117f3ba0d4mr52839238wrp.490.1655024437221; Sun, 12 Jun 2022 02:00:37 -0700 (PDT) Received: from localhost.localdomain ([178.130.153.185]) by smtp.gmail.com with ESMTPSA id d34-20020a05600c4c2200b0039c5b4ab1b0sm4798603wmp.48.2022.06.12.02.00.35 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 12 Jun 2022 02:00:35 -0700 (PDT) From: Joe Damato To: Alexander Viro , linux-kernel@vger.kernel.org Cc: Joe Damato Subject: [RFC,iov_iter v2 2/8] iov_iter: Introduce iter_copy_type Date: Sun, 12 Jun 2022 01:57:51 -0700 Message-Id: <1655024280-23827-3-git-send-email-jdamato@fastly.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1655024280-23827-1-git-send-email-jdamato@fastly.com> References: <1655024280-23827-1-git-send-email-jdamato@fastly.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" struct iov_iter has a new member: iter_copy_type. This field holds a value designating which type of copy to use: a regular temporal copy (ITER_COPY) or a non-temporal copy (ITER_NOCACHE_COPY). iov_iter initializers have been updated to set the default ITER_COPY type. Signed-off-by: Joe Damato --- include/linux/uio.h | 17 +++++++++++++++++ lib/iov_iter.c | 6 ++++++ 2 files changed, 23 insertions(+) diff --git a/include/linux/uio.h b/include/linux/uio.h index 739285f..59573ee 100644 --- a/include/linux/uio.h +++ b/include/linux/uio.h @@ -28,6 +28,12 @@ enum iter_type { ITER_DISCARD, }; =20 +enum iter_copy_type { + /* iter copy types */ + ITER_COPY, + ITER_NOCACHE_COPY, +}; + struct iov_iter_state { size_t iov_offset; size_t count; @@ -35,6 +41,7 @@ struct iov_iter_state { }; =20 struct iov_iter { + u8 iter_copy_type; u8 iter_type; bool nofault; bool data_source; @@ -62,6 +69,11 @@ static inline enum iter_type iov_iter_type(const struct = iov_iter *i) return i->iter_type; } =20 +static inline enum iter_copy_type iov_iter_copy_type(const struct iov_iter= *i) +{ + return i->iter_copy_type; +} + static inline void iov_iter_save_state(struct iov_iter *iter, struct iov_iter_state *state) { @@ -95,6 +107,11 @@ static inline bool iov_iter_is_discard(const struct iov= _iter *i) return iov_iter_type(i) =3D=3D ITER_DISCARD; } =20 +static inline bool iov_iter_copy_is_nt(const struct iov_iter *i) +{ + return iov_iter_copy_type(i) =3D=3D ITER_NOCACHE_COPY; +} + static inline bool iov_iter_is_xarray(const struct iov_iter *i) { return iov_iter_type(i) =3D=3D ITER_XARRAY; diff --git a/lib/iov_iter.c b/lib/iov_iter.c index 6dd5330..d32d7e5 100644 --- a/lib/iov_iter.c +++ b/lib/iov_iter.c @@ -511,6 +511,7 @@ void iov_iter_init(struct iov_iter *i, unsigned int dir= ection, { WARN_ON(direction & ~(READ | WRITE)); *i =3D (struct iov_iter) { + .iter_copy_type =3D ITER_COPY, .iter_type =3D ITER_IOVEC, .nofault =3D false, .data_source =3D direction, @@ -1175,6 +1176,7 @@ void iov_iter_kvec(struct iov_iter *i, unsigned int d= irection, { WARN_ON(direction & ~(READ | WRITE)); *i =3D (struct iov_iter){ + .iter_copy_type =3D ITER_COPY, .iter_type =3D ITER_KVEC, .data_source =3D direction, .kvec =3D kvec, @@ -1191,6 +1193,7 @@ void iov_iter_bvec(struct iov_iter *i, unsigned int d= irection, { WARN_ON(direction & ~(READ | WRITE)); *i =3D (struct iov_iter){ + .iter_copy_type =3D ITER_COPY, .iter_type =3D ITER_BVEC, .data_source =3D direction, .bvec =3D bvec, @@ -1208,6 +1211,7 @@ void iov_iter_pipe(struct iov_iter *i, unsigned int d= irection, BUG_ON(direction !=3D READ); WARN_ON(pipe_full(pipe->head, pipe->tail, pipe->ring_size)); *i =3D (struct iov_iter){ + .iter_copy_type =3D ITER_COPY, .iter_type =3D ITER_PIPE, .data_source =3D false, .pipe =3D pipe, @@ -1237,6 +1241,7 @@ void iov_iter_xarray(struct iov_iter *i, unsigned int= direction, { BUG_ON(direction & ~1); *i =3D (struct iov_iter) { + .iter_copy_type =3D ITER_COPY, .iter_type =3D ITER_XARRAY, .data_source =3D direction, .xarray =3D xarray, @@ -1260,6 +1265,7 @@ void iov_iter_discard(struct iov_iter *i, unsigned in= t direction, size_t count) { BUG_ON(direction !=3D READ); *i =3D (struct iov_iter){ + .iter_copy_type =3D ITER_COPY, .iter_type =3D ITER_DISCARD, .data_source =3D false, .count =3D count, --=20 2.7.4 From nobody Wed Apr 29 08:07:43 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E13F8C43334 for ; Sun, 12 Jun 2022 09:00:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235347AbiFLJAx (ORCPT ); Sun, 12 Jun 2022 05:00:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39886 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235156AbiFLJAp (ORCPT ); Sun, 12 Jun 2022 05:00:45 -0400 Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8BC1751E50 for ; Sun, 12 Jun 2022 02:00:41 -0700 (PDT) Received: by mail-wm1-x331.google.com with SMTP id n185so1469706wmn.4 for ; Sun, 12 Jun 2022 02:00:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=BSwA4NeVgnxLtuLfBKKExOEIpE9041JXV4ItPWNPtss=; b=Phou/fv0D1wJ4Nk2pfva5TbGOkuRvzB5hCvplrKRFOl/E/aof+dvqIV0hEg7B8yWA2 boGSuUryK0hJY/CJr9pfeCKgO8aBpITFUNTkAfI2yfs4h8oPM5j7bbtdchgPKTxhphMi REgT3rfdCzXU5GZMDjqVC2Rut3LVv5IF5cjDs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=BSwA4NeVgnxLtuLfBKKExOEIpE9041JXV4ItPWNPtss=; b=yD0iMgxSRV8WQxFWVUwUe+yWZ1I/Glxn+vAtXBEXtGQWaGurb4Vh0GrYlbkBvlcdnY gOLRcSig7BtlBJzFtUhPVCOU9DFSbnJkRqmrDJzw8WpV3FOd7dqjNXEOLgSNTOEO/CRs EYvqx62nlBNQtnoo8syj7QCeqOva8BcExrtkQJ+MC7c3LX+LmL86vgFPYdJbHOOB9R39 jxJAX73FBoRSVnyTelUWMhufFuM7qs/wOWwlGz/xXcJUjvOdF4PPfznqTHKS6g0+b0Jl 6yuAnswHCgsMk5xfXNbWM/li6wDuheJGhNOuH8EONWog0/qxV0BV5X8FgChvuQw7MB08 lQ9g== X-Gm-Message-State: AOAM530b1FxWU6DzhukJwZTFfjfaQC8VSG9Rtdd0qW5FNW2ZRijrSUz7 S76JaAL+3XjXzS7iw4Yqp5xAOw== X-Google-Smtp-Source: ABdhPJwNsBIpVclTS0sPJeAUmur+L0XJo1yT8PTf/ts7qsn4fvkqtJUHn+dBs1UinPtS7am6gjEEiw== X-Received: by 2002:a05:600c:358f:b0:39c:7fe7:cbe2 with SMTP id p15-20020a05600c358f00b0039c7fe7cbe2mr8463834wmq.46.1655024439801; Sun, 12 Jun 2022 02:00:39 -0700 (PDT) Received: from localhost.localdomain ([178.130.153.185]) by smtp.gmail.com with ESMTPSA id d34-20020a05600c4c2200b0039c5b4ab1b0sm4798603wmp.48.2022.06.12.02.00.37 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 12 Jun 2022 02:00:38 -0700 (PDT) From: Joe Damato To: Alexander Viro , linux-kernel@vger.kernel.org Cc: Joe Damato Subject: [RFC,iov_iter v2 3/8] iov_iter: add copyin_iovec helper Date: Sun, 12 Jun 2022 01:57:52 -0700 Message-Id: <1655024280-23827-4-git-send-email-jdamato@fastly.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1655024280-23827-1-git-send-email-jdamato@fastly.com> References: <1655024280-23827-1-git-send-email-jdamato@fastly.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" copyin_iovec is a helper which wraps copyin and selects the right copy method based on the iter_copy_type. Signed-off-by: Joe Damato --- lib/iov_iter.c | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/lib/iov_iter.c b/lib/iov_iter.c index d32d7e5..6720cb2 100644 --- a/lib/iov_iter.c +++ b/lib/iov_iter.c @@ -168,6 +168,15 @@ static int copyin(void *to, const void __user *from, s= ize_t n) return n; } =20 +static int copyin_iovec(void *to, const void __user *from, size_t n, + struct iov_iter *i) +{ + if (unlikely(iov_iter_copy_is_nt(i))) + return __copy_from_user_nocache(to, from, n); + else + return copyin(to, from, n); +} + static size_t copy_page_to_iter_iovec(struct page *page, size_t offset, si= ze_t bytes, struct iov_iter *i) { @@ -278,7 +287,7 @@ static size_t copy_page_from_iter_iovec(struct page *pa= ge, size_t offset, size_t to =3D kaddr + offset; =20 /* first chunk, usually the only one */ - left =3D copyin(to, buf, copy); + left =3D copyin_iovec(to, buf, copy, i); copy -=3D left; skip +=3D copy; to +=3D copy; @@ -288,7 +297,7 @@ static size_t copy_page_from_iter_iovec(struct page *pa= ge, size_t offset, size_t iov++; buf =3D iov->iov_base; copy =3D min(bytes, iov->iov_len); - left =3D copyin(to, buf, copy); + left =3D copyin_iovec(to, buf, copy, i); copy -=3D left; skip =3D copy; to +=3D copy; @@ -307,7 +316,7 @@ static size_t copy_page_from_iter_iovec(struct page *pa= ge, size_t offset, size_t =20 kaddr =3D kmap(page); to =3D kaddr + offset; - left =3D copyin(to, buf, copy); + left =3D copyin_iovec(to, buf, copy, i); copy -=3D left; skip +=3D copy; to +=3D copy; @@ -316,7 +325,7 @@ static size_t copy_page_from_iter_iovec(struct page *pa= ge, size_t offset, size_t iov++; buf =3D iov->iov_base; copy =3D min(bytes, iov->iov_len); - left =3D copyin(to, buf, copy); + left =3D copyin_iovec(to, buf, copy, i); copy -=3D left; skip =3D copy; to +=3D copy; @@ -766,7 +775,7 @@ size_t _copy_from_iter(void *addr, size_t bytes, struct= iov_iter *i) if (iter_is_iovec(i)) might_fault(); iterate_and_advance(i, bytes, base, len, off, - copyin(addr + off, base, len), + copyin_iovec(addr + off, base, len, i), memcpy(addr + off, base, len) ) =20 --=20 2.7.4 From nobody Wed Apr 29 08:07:43 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81145C43334 for ; Sun, 12 Jun 2022 09:00:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235377AbiFLJA5 (ORCPT ); Sun, 12 Jun 2022 05:00:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39862 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235279AbiFLJAr (ORCPT ); Sun, 12 Jun 2022 05:00:47 -0400 Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 106AA51E4E for ; Sun, 12 Jun 2022 02:00:44 -0700 (PDT) Received: by mail-wm1-x336.google.com with SMTP id x6-20020a1c7c06000000b003972dfca96cso1602097wmc.4 for ; Sun, 12 Jun 2022 02:00:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Bh+5cvdXh3nnzozHNpzUlua2s6tISLW0nqJPvBVheEw=; b=RDUvhVt18U9J3OuMRKOCldFZ/tV2MGuKPzdMcLjIrxSpqqnq3EPnIHWGXU+6Oj4Mb3 6bTxjUXe+5kYOwV/kL6w/Uz7Hj9SlihPEFvEIEHwyoV9mrw3YjFF2T8E6f8J4Y3U3y99 S+cOMnUr2QKpBEtWXolUFuV7ZwjBp4xwLsiPg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Bh+5cvdXh3nnzozHNpzUlua2s6tISLW0nqJPvBVheEw=; b=s4oNlmYmi3fCSxLCFlwtUp0Wpw83sLb4ZdYXgoOmzG4lSqOnKcc5TUn1WjFO+Fj1ie mmVFZXriXCruK1vy0zvFz6L9knVZ29pGGI8WMOixyxTVUTW/ptJEOcRTe+EO5Nul65Le kC4bjmqRUzUzWk4+YUvhN2JLhuCyd7N0jWkh4Wd7F/YU6116SlVyYPadvxV0MTSXhBBD fNkJfholmirkhmCDfmiprlUgvNEZnaQqNCtOUu7jsJ9g8AN4WRqwkpuIHyXjy0riVo60 ZQaiuQLurPMkBTPB2m4mIndBVWl4DvErUMmKP2IHHQ+mptvvk8kcmdY2HJ2n0vixWqyH mEKw== X-Gm-Message-State: AOAM533zoKMLhzgIHYxxd+W1ki/YY9KTt+mN9UunwErYRBGLi+BqtXPa yv9C3FIk6NMtb21cMiAskKi6/h8ztHkiSukDjwR939rFGIuXhkUbLrM3KOZ7O7S5VLaHaWksoNB lI8xyJWp2gwfGGQDS1T3+wKs+4xjs1kfOlfb4WghsezwMrJ5/3gL5pS1GIl6zaythH65AhibTb/ BA X-Google-Smtp-Source: ABdhPJzHwiWJn0Q5mqMHXp5JXd4bsJvT8DXD1tByMG5tzxXc8u3ebIVE9ktGcRLM0xx+cGfJp2Oycw== X-Received: by 2002:a05:600c:58a:b0:39c:80ed:68be with SMTP id o10-20020a05600c058a00b0039c80ed68bemr8304657wmd.150.1655024442341; Sun, 12 Jun 2022 02:00:42 -0700 (PDT) Received: from localhost.localdomain ([178.130.153.185]) by smtp.gmail.com with ESMTPSA id d34-20020a05600c4c2200b0039c5b4ab1b0sm4798603wmp.48.2022.06.12.02.00.40 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 12 Jun 2022 02:00:41 -0700 (PDT) From: Joe Damato To: linux-kernel@vger.kernel.org Cc: Joe Damato Subject: [RFC,net-next v2 4/8] net: Add MSG_NTCOPY sendmsg flag Date: Sun, 12 Jun 2022 01:57:53 -0700 Message-Id: <1655024280-23827-5-git-send-email-jdamato@fastly.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1655024280-23827-1-git-send-email-jdamato@fastly.com> References: <1655024280-23827-1-git-send-email-jdamato@fastly.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Add MSG_NTCOPY so that user applications can ask the kernel for a non-temporal copy when copying data into the kernel for TX. A simple helper is provided to set the iovec iterator copy type if MSG_NTCOPY is set. Signed-off-by: Joe Damato --- include/linux/socket.h | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/include/linux/socket.h b/include/linux/socket.h index 17311ad..98cb735 100644 --- a/include/linux/socket.h +++ b/include/linux/socket.h @@ -318,6 +318,7 @@ struct ucred { * plain text and require encryption */ =20 +#define MSG_NTCOPY 0x2000000 /* Use a non-temporal copy */ #define MSG_ZEROCOPY 0x4000000 /* Use user data in kernel path */ #define MSG_FASTOPEN 0x20000000 /* Send data in TCP SYN */ #define MSG_CMSG_CLOEXEC 0x40000000 /* Set close_on_exec for file @@ -378,6 +379,14 @@ struct ucred { extern int move_addr_to_kernel(void __user *uaddr, int ulen, struct sockad= dr_storage *kaddr); extern int put_cmsg(struct msghdr*, int level, int type, int len, void *da= ta); =20 +static inline void msg_set_iter_copy_type(struct msghdr *msg) +{ + if (msg->msg_flags & MSG_NTCOPY) + msg->msg_iter.iter_copy_type =3D ITER_NOCACHE_COPY; + else + msg->msg_iter.iter_copy_type =3D ITER_COPY; +} + struct timespec64; struct __kernel_timespec; struct old_timespec32; --=20 2.7.4 From nobody Wed Apr 29 08:07:43 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9A55AC433EF for ; Sun, 12 Jun 2022 09:01:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235473AbiFLJA7 (ORCPT ); Sun, 12 Jun 2022 05:00:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39866 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235303AbiFLJAr (ORCPT ); Sun, 12 Jun 2022 05:00:47 -0400 Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3A50149F3E for ; Sun, 12 Jun 2022 02:00:45 -0700 (PDT) Received: by mail-wm1-x330.google.com with SMTP id z17so1482730wmi.1 for ; Sun, 12 Jun 2022 02:00:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Ea5VDYyACTdLFfDR1OzYQ+USVPk0XrM3SL5Uf8rgmF0=; b=KR0M7uzWU0BPwWn1Brdk2z+8Fi7ZcEFz/0BqFDgszVZgE8lwHoAW6oSObc3l1b6vr4 BoC6dWRtE6EBumonrLPkkC91REo5S9yRXWhNwSvD4HcKvP5LKq/IS1A2cXQD+219YWhl ZvQrPdoNkDwgS4izTjiqJ0S57mfhBZYKU3hDk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Ea5VDYyACTdLFfDR1OzYQ+USVPk0XrM3SL5Uf8rgmF0=; b=CJKgXrYYQr6dv1Cq8WdnH7JPtdTQGRDgb/Yi4kus/KBoXDV3Sj/dslOYZoUomImbdL yLKqwh8gb001mMay0kaupKuF0IHhOOnkoQ7XyXXdvlK0l5CSwSHIj4DC4RQK1MZ+9kBf mJJs2LyszgzBESIe+ttckeh6gBLR9Zf+YTDdX2WQ37IR+rNhIgd6bAUtKt9jpX/rOfhB T/erDs8QbJhXmwvtTUl5SUur0LLtUoCFYXZmoLJIpjBi4JFus71bKnrces6YpbYd1GGW lEYlHYYsA62OUl7d/5pd4sp8YY6JSXfOlkLFtW8t1H+oEh9ormpBKalGfzhQMVIFb72k XaOQ== X-Gm-Message-State: AOAM532qaJ39R5R6+82VHeibxXzv1ut40kTw2r6zXV7jpeg5iTVSBAw/ tXtU7wDtLe1LJS6umT1weM25Cg== X-Google-Smtp-Source: ABdhPJyNlQK22KcaUyTUuErIsRmeLRfYWG48oRcA2vWnHqeaWVoWe/3tpFsgtVdSSbyWZCmkaqjVDg== X-Received: by 2002:a7b:c346:0:b0:397:626d:d2c4 with SMTP id l6-20020a7bc346000000b00397626dd2c4mr8629025wmj.172.1655024444792; Sun, 12 Jun 2022 02:00:44 -0700 (PDT) Received: from localhost.localdomain ([178.130.153.185]) by smtp.gmail.com with ESMTPSA id d34-20020a05600c4c2200b0039c5b4ab1b0sm4798603wmp.48.2022.06.12.02.00.42 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 12 Jun 2022 02:00:43 -0700 (PDT) From: Joe Damato To: "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , netdev@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Joe Damato Subject: [RFC,net-next v2 5/8] net: unix: Support MSG_NTCOPY Date: Sun, 12 Jun 2022 01:57:54 -0700 Message-Id: <1655024280-23827-6-git-send-email-jdamato@fastly.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1655024280-23827-1-git-send-email-jdamato@fastly.com> References: <1655024280-23827-1-git-send-email-jdamato@fastly.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Add support for MSG_NTCOPY to unix sockets. The helper function msg_set_iter_copy_type is used to set the copy flag on the iovec iterator correctly. Signed-off-by: Joe Damato --- net/unix/af_unix.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/net/unix/af_unix.c b/net/unix/af_unix.c index 3453e00..b8f522d 100644 --- a/net/unix/af_unix.c +++ b/net/unix/af_unix.c @@ -1905,6 +1905,8 @@ static int unix_dgram_sendmsg(struct socket *sock, st= ruct msghdr *msg, skb_put(skb, len - data_len); skb->data_len =3D data_len; skb->len =3D len; + + msg_set_iter_copy_type(msg); err =3D skb_copy_datagram_from_iter(skb, 0, &msg->msg_iter, len); if (err) goto out_free; @@ -2165,6 +2167,8 @@ static int unix_stream_sendmsg(struct socket *sock, s= truct msghdr *msg, skb_put(skb, size - data_len); skb->data_len =3D data_len; skb->len =3D size; + + msg_set_iter_copy_type(msg); err =3D skb_copy_datagram_from_iter(skb, 0, &msg->msg_iter, size); if (err) { kfree_skb(skb); --=20 2.7.4 From nobody Wed Apr 29 08:07:43 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4B05C433EF for ; Sun, 12 Jun 2022 09:01:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235501AbiFLJBC (ORCPT ); Sun, 12 Jun 2022 05:01:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40436 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235259AbiFLJAv (ORCPT ); Sun, 12 Jun 2022 05:00:51 -0400 Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 35A21517F8 for ; Sun, 12 Jun 2022 02:00:49 -0700 (PDT) Received: by mail-wm1-x32b.google.com with SMTP id l126-20020a1c2584000000b0039c1a10507fso1611652wml.1 for ; Sun, 12 Jun 2022 02:00:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=QEsnoS6fs7x7z63+N74TXGO62/r4Rll/JtLDyGKeftw=; b=RyVH1BNMS71kJucK5zElSTO/iNUJtJ1wzfB62MK2wkxOOJQg/HG4FYnSYIx5b28jtI c1Wk6gsCMqqP+0DlffzZlNALt2iyg5mxWghT7RpQ3Yotns8G4e/n7SJRzvMrjiCaxae/ PUMD95tfWDtLUEIwDO+4QJirah2khGY715i+g= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=QEsnoS6fs7x7z63+N74TXGO62/r4Rll/JtLDyGKeftw=; b=hwi7cQobWqNWP9l6IoP/vVZgioUgfJOhE48HpO2EV6SYwBnuMDixcUuNWh+A9hz3RW 9Gi2mCwzWVoR375r2ptizr5inmJQy36RGob9jnAgM97lmgVmXOYeN0GqAbuk1JOq4KKD qRZKttDKqUDTaRbv26xkHtbxkKp/Kq1Ry3E82n7vIpwCQw575TzTqcWOJbMITsYHJXjc gzvxt1hyzz3rvwT8n4ADLFggNkRFefjbd77PxbAsrGDsM116dKXYXc2qLtAmKadl2lp/ xj0Rq2esJDYn5t3ctmZtEaa1U1JxsneY3+YaPhp2e2MLVdmW8DPccFL3Z0BSdDkIYTj7 iBmQ== X-Gm-Message-State: AOAM533MxchnR5i7tXVQGfIjHY3Sl//lS10UGY7zM3FglgxFTNNAhsGY Z3zwMgLgsrp3I5IbXmLyLJLWVA== X-Google-Smtp-Source: ABdhPJwXZyVpsGAIc5rt4KdnvnyW5tPEH0k/SyiM2mkjkdzxtxUdKAXvWaJOWLmn6Fb+SsUMSvmc1g== X-Received: by 2002:a05:600c:4e94:b0:39c:81fd:6594 with SMTP id f20-20020a05600c4e9400b0039c81fd6594mr8030223wmq.198.1655024448441; Sun, 12 Jun 2022 02:00:48 -0700 (PDT) Received: from localhost.localdomain ([178.130.153.185]) by smtp.gmail.com with ESMTPSA id d34-20020a05600c4c2200b0039c5b4ab1b0sm4798603wmp.48.2022.06.12.02.00.45 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 12 Jun 2022 02:00:47 -0700 (PDT) From: Joe Damato To: "David S. Miller" , Hideaki YOSHIFUJI , David Ahern , Eric Dumazet , Jakub Kicinski , Paolo Abeni , netdev@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Joe Damato Subject: [RFC,net-next v2 6/8] net: ip: Support MSG_NTCOPY Date: Sun, 12 Jun 2022 01:57:55 -0700 Message-Id: <1655024280-23827-7-git-send-email-jdamato@fastly.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1655024280-23827-1-git-send-email-jdamato@fastly.com> References: <1655024280-23827-1-git-send-email-jdamato@fastly.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Support nontemporal copies of IP data when applications set the MSG_NTCOPY flag and checksumming is offloaded. ip_generic_getfrag is used by UDP and raw sockets, so this change effectively enables MSG_NTCOPY support for both. Signed-off-by: Joe Damato --- net/ipv4/ip_output.c | 1 + 1 file changed, 1 insertion(+) diff --git a/net/ipv4/ip_output.c b/net/ipv4/ip_output.c index 00b4bf2..75c8627 100644 --- a/net/ipv4/ip_output.c +++ b/net/ipv4/ip_output.c @@ -933,6 +933,7 @@ ip_generic_getfrag(void *from, char *to, int offset, in= t len, int odd, struct sk struct msghdr *msg =3D from; =20 if (skb->ip_summed =3D=3D CHECKSUM_PARTIAL) { + msg_set_iter_copy_type(msg); if (!copy_from_iter_full(to, len, &msg->msg_iter)) return -EFAULT; } else { --=20 2.7.4 From nobody Wed Apr 29 08:07:43 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2B16DC433EF for ; Sun, 12 Jun 2022 09:01:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235531AbiFLJBG (ORCPT ); Sun, 12 Jun 2022 05:01:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41116 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235423AbiFLJA7 (ORCPT ); Sun, 12 Jun 2022 05:00:59 -0400 Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A53D951E4F for ; Sun, 12 Jun 2022 02:00:52 -0700 (PDT) Received: by mail-wm1-x335.google.com with SMTP id m32-20020a05600c3b2000b0039756bb41f2so1602714wms.3 for ; Sun, 12 Jun 2022 02:00:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=ELhqEDOqgmC+TbLCt4NyDbvbHR56jry/w5HtXxvrUIM=; b=PaZYXKu8oFHY0LGzLxFo66YYimisvTv48Bj8XCIKfLpjdtwA5eikvThJJwxRW5WgYG m6/1fv09uv8Q9xhYBCq4o2+KLW3MutYdmX/1ljpZoq+2GlpOkiLdQGZWqnuj8+qoVTvB T45fXPVb9A6z8tzB5O+p0Sm1B0o5nToqWufhY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=ELhqEDOqgmC+TbLCt4NyDbvbHR56jry/w5HtXxvrUIM=; b=PqdJl+34xZa4j8qhab3ps/7MJZM2V0VYSaLKrgGvW4E0bz/dKPo1e6fSIcW3LXf+kI 5Vqmj38KhAePXtjV4c69xYf2B7VYhoVc5wAT+9tjtSAIhO2AK01dxOVU/MmbYwaNwyJ2 w46MAjZcPtgLB6l5FNwH7tZyJSvw+yVkio18caH7enRJlwwZYYvTzbwNcQiD/ZEKdFJw HOOznIJMvB2IGAp7qy85vx1XokuVSitbLG2KGgWhpb60fxJVpMy/rW+Jg1NUbLyx3V2a iA5E2Oc9HPx34zdQggk6Se/EeHPIsqRz2gGBp7AFB8gMN1al71BkIUORPqydwtUDOcQC uNuQ== X-Gm-Message-State: AOAM532jimV11xLGim3v6Xkd9VM2afiawN7fBsrCKGSGG9iyLd058UBl Afm2jrs5OXIWOz48z0ZY0PBGZQ== X-Google-Smtp-Source: ABdhPJzd9bkHYJC56LA9yvbOs3N/OrPbAmtYouTWcxhGPvju+dr0MxD3WT7YPW2C1hZk3nHzKsVgMA== X-Received: by 2002:a05:600c:4a28:b0:39c:4d97:37e2 with SMTP id c40-20020a05600c4a2800b0039c4d9737e2mr8439018wmp.31.1655024451149; Sun, 12 Jun 2022 02:00:51 -0700 (PDT) Received: from localhost.localdomain ([178.130.153.185]) by smtp.gmail.com with ESMTPSA id d34-20020a05600c4c2200b0039c5b4ab1b0sm4798603wmp.48.2022.06.12.02.00.49 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 12 Jun 2022 02:00:49 -0700 (PDT) From: Joe Damato To: "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , netdev@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Joe Damato Subject: [RFC,net-next v2 7/8] net: udplite: Support MSG_NTCOPY Date: Sun, 12 Jun 2022 01:57:56 -0700 Message-Id: <1655024280-23827-8-git-send-email-jdamato@fastly.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1655024280-23827-1-git-send-email-jdamato@fastly.com> References: <1655024280-23827-1-git-send-email-jdamato@fastly.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Support non-temporal copies of udp-lite data when applications set the MSG_NTCOPY. Signed-off-by: Joe Damato --- include/net/udplite.h | 1 + 1 file changed, 1 insertion(+) diff --git a/include/net/udplite.h b/include/net/udplite.h index a3c5311..000677c5 100644 --- a/include/net/udplite.h +++ b/include/net/udplite.h @@ -21,6 +21,7 @@ static __inline__ int udplite_getfrag(void *from, char *t= o, int offset, int len, int odd, struct sk_buff *skb) { struct msghdr *msg =3D from; + msg_set_iter_copy_type(msg); return copy_from_iter_full(to, len, &msg->msg_iter) ? 0 : -EFAULT; } =20 --=20 2.7.4 From nobody Wed Apr 29 08:07:43 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6E497C433EF for ; Sun, 12 Jun 2022 09:01:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235584AbiFLJBK (ORCPT ); Sun, 12 Jun 2022 05:01:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41074 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235287AbiFLJA7 (ORCPT ); Sun, 12 Jun 2022 05:00:59 -0400 Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5864E51E6E for ; Sun, 12 Jun 2022 02:00:55 -0700 (PDT) Received: by mail-wm1-x32b.google.com with SMTP id m125-20020a1ca383000000b0039c63fe5f64so1625334wme.0 for ; Sun, 12 Jun 2022 02:00:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=oZewD39xI/d3cA2lrYzrzbuReF/vvOH6OKqzVd89Vi0=; b=I7WHh5fScRUlNuotdc665LoLnMNVifFMFBlEzjUFcKf9gROoXJ845rIiW1Qk+PTxiR PtbBonb26DI70Kx2ARKOD7D1WUmUHI/gQp3TkURjBlfMVEc24tG55CXut6lZKgTXhGkZ xV/ndMmtPSnsvL4UM8x/2vxsNs0Yfbu6sQJzs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=oZewD39xI/d3cA2lrYzrzbuReF/vvOH6OKqzVd89Vi0=; b=DupWr0NLqD8BH9KR+QT53BFnfnPHe+KhIFS0pBopY/2ushpqa5cH0i38P2u4mbQ3K/ wzqR2ZdGhyM3Np9b5lv/wBnB5GEXLE7gu3+nejX318pBNmhCGouI2k1B1f52sViHVEX1 EVuhz85zPylbHoBS0ceP265pREAelPCq3h2goJiDH2rjpJySVo+yymVz1PnSZNT2sUue mp98ojtSU0sQ7xY14b0fOYSgxlmVy9TdEMSgR0FNajiAjfMt6NioXrec1jUBDa+BCQ8X shGOLy/8TKZ8ycyye9lLDI1xyw5dZR9002YXyA87ZHWiKNgsPbO7xeOjbboXVXIMJ+5Q QjvA== X-Gm-Message-State: AOAM5324rov4YJanotiBFr6YEZBvQ08uNhQREX7qMwzzMebEGY3u+PaU hHUMfxytml3ZKcfQ2ayYMDPmaQ== X-Google-Smtp-Source: ABdhPJyRc2I34omfc6W/l7b3YpL+JA18yp8NmiSPvr3XlLpY4Wwvv7Z0CdG4SFGFbc7ui5FdZrlNTQ== X-Received: by 2002:a05:600c:1ca0:b0:39c:4dbd:e9ed with SMTP id k32-20020a05600c1ca000b0039c4dbde9edmr8452040wms.40.1655024453936; Sun, 12 Jun 2022 02:00:53 -0700 (PDT) Received: from localhost.localdomain ([178.130.153.185]) by smtp.gmail.com with ESMTPSA id d34-20020a05600c4c2200b0039c5b4ab1b0sm4798603wmp.48.2022.06.12.02.00.51 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 12 Jun 2022 02:00:53 -0700 (PDT) From: Joe Damato To: "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Hideaki YOSHIFUJI , David Ahern , netdev@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Joe Damato Subject: [RFC,net-next v2 8/8] net: tcp: Support MSG_NTCOPY Date: Sun, 12 Jun 2022 01:57:57 -0700 Message-Id: <1655024280-23827-9-git-send-email-jdamato@fastly.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1655024280-23827-1-git-send-email-jdamato@fastly.com> References: <1655024280-23827-1-git-send-email-jdamato@fastly.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Support non-temporal copies in the TCP sendmsg path. Previously, the only way to enable non-temporal copies was to enable them for the entire interface (via ethtool). This change allows user programs to request non-temporal copies for specific sendmsg calls. Signed-off-by: Joe Damato --- include/net/sock.h | 2 +- net/ipv4/tcp.c | 2 ++ 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/include/net/sock.h b/include/net/sock.h index 0063e84..b666ecd 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -2200,7 +2200,7 @@ static inline int skb_do_copy_data_nocache(struct soc= k *sk, struct sk_buff *skb, if (!csum_and_copy_from_iter_full(to, copy, &csum, from)) return -EFAULT; skb->csum =3D csum_block_add(skb->csum, csum, offset); - } else if (sk->sk_route_caps & NETIF_F_NOCACHE_COPY) { + } else if (sk->sk_route_caps & NETIF_F_NOCACHE_COPY || iov_iter_copy_is_n= t(from)) { if (!copy_from_iter_full_nocache(to, copy, from)) return -EFAULT; } else if (!copy_from_iter_full(to, copy, from)) diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index 14ebb4e..5b36e00 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -1201,6 +1201,8 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr= *msg, size_t size) =20 flags =3D msg->msg_flags; =20 + msg_set_iter_copy_type(msg); + if (flags & MSG_ZEROCOPY && size && sock_flag(sk, SOCK_ZEROCOPY)) { skb =3D tcp_write_queue_tail(sk); uarg =3D msg_zerocopy_realloc(sk, size, skb_zcopy(skb)); --=20 2.7.4