From nobody Tue Apr 7 16:31:00 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5021ECAAA3 for ; Fri, 26 Aug 2022 15:08:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244204AbiHZPIv (ORCPT ); Fri, 26 Aug 2022 11:08:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33910 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243853AbiHZPIi (ORCPT ); Fri, 26 Aug 2022 11:08:38 -0400 Received: from mail-ed1-x549.google.com (mail-ed1-x549.google.com [IPv6:2a00:1450:4864:20::549]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5869CDC5EC for ; Fri, 26 Aug 2022 08:08:23 -0700 (PDT) Received: by mail-ed1-x549.google.com with SMTP id q18-20020a056402519200b0043dd2ff50feso1231236edd.9 for ; Fri, 26 Aug 2022 08:08:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc; bh=BPAgteciaEShNCu1utOx4VHGX1XRyq5ObvUAakfxq7k=; b=jCaizTYg1qMmJ26neRflpmyoGoKSSzwoQ3ghp5ED6sQ0QTlwBDX6eGH4Q1incOUpRf A9evOUHUN2DRNN3KIY+fweMjD/DZ4xlbZu00h6mBm12fvHomrnz+0lpoebslTIvGgTe+ gdtLap2UAeAYuweKYtWNFnZCFeo+7QmPjS4mEY0fqEBA+wTuqFT0Ebh7UvC69tA0VceO U+A1VATc2FKIdkK4YFsc/hZEovB4eynb2DvJ5fjLkaN1eoq9/dzW4ni3/SQJWfAyRBQ+ mWKfQjkLkiWrUYhGM1fBmA4YzoA+IjtGx7XKjxo7S8MYC3SRxlNpW94EGzvpucAhGMMX rdcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc; bh=BPAgteciaEShNCu1utOx4VHGX1XRyq5ObvUAakfxq7k=; b=8JiIRooJ+QlpFxQoKtZxXY6haL9qsgYhQMIWf+vEWOeTT7Mg4ZF5hhg14HSbVptQ88 cB2JzBR5b9WoSvpuKRkwP23h4K2fJxV1G9qVrGnWfgJPk7gj8GJ1CYdFggmN0lZLQ3s5 ArQNHrw8QtLgIKYXhOGcdm6jQIIz3FavwlDW6V8r8FMzCd9v65/silmYlzBDHo0HLMk6 QG3gAnHO58JdIzYYdZz9UzF03OkFjenciHTTiRSgXUf6WgXl7khiVcuhWIltjzUtGool GuNu4J1649hSvIdX/8cypEBf3Gs4RT2LLzRsptnoDN1I19UIMaxhIwoVA6pKnSEkxcDB MH8g== X-Gm-Message-State: ACgBeo0glxtbxZcrSRCerNL/nPzW54Ba8hgxuC95AiWAc5++cqSAXbnm 4nKNbuUbvl7COGLWUOD+MilByzzFO/4= X-Google-Smtp-Source: AA6agR7KbGTD+Yt7qgz7f9gG2VmjSlBnG9STKzG8jXRNjHiFbuQJEvlbRex0Fv49kcKUtGLpmFtNC/Gl7xY= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:5207:ac36:fdd3:502d]) (user=glider job=sendgmr) by 2002:a17:907:da0:b0:730:d0ba:7b13 with SMTP id go32-20020a1709070da000b00730d0ba7b13mr6038874ejc.332.1661526501538; Fri, 26 Aug 2022 08:08:21 -0700 (PDT) Date: Fri, 26 Aug 2022 17:07:26 +0200 In-Reply-To: <20220826150807.723137-1-glider@google.com> Mime-Version: 1.0 References: <20220826150807.723137-1-glider@google.com> X-Mailer: git-send-email 2.37.2.672.g94769d06f0-goog Message-ID: <20220826150807.723137-4-glider@google.com> Subject: [PATCH v5 03/44] instrumented.h: allow instrumenting both sides of copy_from_user() From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Introduce instrument_copy_from_user_before() and instrument_copy_from_user_after() hooks to be invoked before and after the call to copy_from_user(). KASAN and KCSAN will be only using instrument_copy_from_user_before(), but for KMSAN we'll need to insert code after copy_from_user(). Signed-off-by: Alexander Potapenko Reviewed-by: Marco Elver --- v4: -- fix _copy_from_user_key() in arch/s390/lib/uaccess.c (Reported-by: kernel test robot ) Link: https://linux-review.googlesource.com/id/I855034578f0b0f126734cbd734f= b4ae1d3a6af99 --- arch/s390/lib/uaccess.c | 3 ++- include/linux/instrumented.h | 21 +++++++++++++++++++-- include/linux/uaccess.h | 19 ++++++++++++++----- lib/iov_iter.c | 9 ++++++--- lib/usercopy.c | 3 ++- 5 files changed, 43 insertions(+), 12 deletions(-) diff --git a/arch/s390/lib/uaccess.c b/arch/s390/lib/uaccess.c index d7b3b193d1088..58033dfcb6d45 100644 --- a/arch/s390/lib/uaccess.c +++ b/arch/s390/lib/uaccess.c @@ -81,8 +81,9 @@ unsigned long _copy_from_user_key(void *to, const void __= user *from, =20 might_fault(); if (!should_fail_usercopy()) { - instrument_copy_from_user(to, from, n); + instrument_copy_from_user_before(to, from, n); res =3D raw_copy_from_user_key(to, from, n, key); + instrument_copy_from_user_after(to, from, n, res); } if (unlikely(res)) memset(to + (n - res), 0, res); diff --git a/include/linux/instrumented.h b/include/linux/instrumented.h index 42faebbaa202a..ee8f7d17d34f5 100644 --- a/include/linux/instrumented.h +++ b/include/linux/instrumented.h @@ -120,7 +120,7 @@ instrument_copy_to_user(void __user *to, const void *fr= om, unsigned long n) } =20 /** - * instrument_copy_from_user - instrument writes of copy_from_user + * instrument_copy_from_user_before - add instrumentation before copy_from= _user * * Instrument writes to kernel memory, that are due to copy_from_user (and * variants). The instrumentation should be inserted before the accesses. @@ -130,10 +130,27 @@ instrument_copy_to_user(void __user *to, const void *= from, unsigned long n) * @n number of bytes to copy */ static __always_inline void -instrument_copy_from_user(const void *to, const void __user *from, unsigne= d long n) +instrument_copy_from_user_before(const void *to, const void __user *from, = unsigned long n) { kasan_check_write(to, n); kcsan_check_write(to, n); } =20 +/** + * instrument_copy_from_user_after - add instrumentation after copy_from_u= ser + * + * Instrument writes to kernel memory, that are due to copy_from_user (and + * variants). The instrumentation should be inserted after the accesses. + * + * @to destination address + * @from source address + * @n number of bytes to copy + * @left number of bytes not copied (as returned by copy_from_user) + */ +static __always_inline void +instrument_copy_from_user_after(const void *to, const void __user *from, + unsigned long n, unsigned long left) +{ +} + #endif /* _LINUX_INSTRUMENTED_H */ diff --git a/include/linux/uaccess.h b/include/linux/uaccess.h index 47e5d374c7ebe..afb18f198843b 100644 --- a/include/linux/uaccess.h +++ b/include/linux/uaccess.h @@ -58,20 +58,28 @@ static __always_inline __must_check unsigned long __copy_from_user_inatomic(void *to, const void __user *from, unsigned long= n) { - instrument_copy_from_user(to, from, n); + unsigned long res; + + instrument_copy_from_user_before(to, from, n); check_object_size(to, n, false); - return raw_copy_from_user(to, from, n); + res =3D raw_copy_from_user(to, from, n); + instrument_copy_from_user_after(to, from, n, res); + return res; } =20 static __always_inline __must_check unsigned long __copy_from_user(void *to, const void __user *from, unsigned long n) { + unsigned long res; + might_fault(); + instrument_copy_from_user_before(to, from, n); if (should_fail_usercopy()) return n; - instrument_copy_from_user(to, from, n); check_object_size(to, n, false); - return raw_copy_from_user(to, from, n); + res =3D raw_copy_from_user(to, from, n); + instrument_copy_from_user_after(to, from, n, res); + return res; } =20 /** @@ -115,8 +123,9 @@ _copy_from_user(void *to, const void __user *from, unsi= gned long n) unsigned long res =3D n; might_fault(); if (!should_fail_usercopy() && likely(access_ok(from, n))) { - instrument_copy_from_user(to, from, n); + instrument_copy_from_user_before(to, from, n); res =3D raw_copy_from_user(to, from, n); + instrument_copy_from_user_after(to, from, n, res); } if (unlikely(res)) memset(to + (n - res), 0, res); diff --git a/lib/iov_iter.c b/lib/iov_iter.c index 4b7fce72e3e52..c3ca28ca68a65 100644 --- a/lib/iov_iter.c +++ b/lib/iov_iter.c @@ -174,13 +174,16 @@ static int copyout(void __user *to, const void *from,= size_t n) =20 static int copyin(void *to, const void __user *from, size_t n) { + size_t res =3D n; + if (should_fail_usercopy()) return n; if (access_ok(from, n)) { - instrument_copy_from_user(to, from, n); - n =3D raw_copy_from_user(to, from, n); + instrument_copy_from_user_before(to, from, n); + res =3D raw_copy_from_user(to, from, n); + instrument_copy_from_user_after(to, from, n, res); } - return n; + return res; } =20 static inline struct pipe_buffer *pipe_buf(const struct pipe_inode_info *p= ipe, diff --git a/lib/usercopy.c b/lib/usercopy.c index 7413dd300516e..1505a52f23a01 100644 --- a/lib/usercopy.c +++ b/lib/usercopy.c @@ -12,8 +12,9 @@ unsigned long _copy_from_user(void *to, const void __user= *from, unsigned long n unsigned long res =3D n; might_fault(); if (!should_fail_usercopy() && likely(access_ok(from, n))) { - instrument_copy_from_user(to, from, n); + instrument_copy_from_user_before(to, from, n); res =3D raw_copy_from_user(to, from, n); + instrument_copy_from_user_after(to, from, n, res); } if (unlikely(res)) memset(to + (n - res), 0, res); --=20 2.37.2.672.g94769d06f0-goog