From nobody Mon Apr 28 17:41:40 2025
Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com
[148.163.158.5])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(No client certificate requested)
by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5E0DB1F63F9;
Mon, 7 Apr 2025 15:12:03 +0000 (UTC)
Authentication-Results: smtp.subspace.kernel.org;
arc=none smtp.client-ip=148.163.158.5
ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116;
t=1744038724; cv=none;
b=KEHavxyrO/TVZ43H5EFKTQ8WJ6jC5qmeKZYH/9HxOxmB8WLNjDEUHIQArANUUisBh8sFT5ZgJZkd8fuc1XpWJNTsOZ2Ibnl5DeSVzSfI+2JGndaq1eKq+4HeZiqdhyHEpMHP1wJueiKoiH+8pQyOHWc5ZYZaE5tyOUKcv5enruM=
ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org;
s=arc-20240116; t=1744038724; c=relaxed/simple;
bh=hj/TDsisJCnOvRQax5hfaCc22qUCiJgMR2YNMLLyCWQ=;
h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References:
MIME-Version;
b=qtaoU0BWLF/D+t3p/TIiVcgtdnjKx40Pnsz1DWRhBPtF1Q/2ym49r4JypNYDGW6shNTqsNIxTUCbd3fNvNhdQ6EpNVpDdKbA32c26eNKABAOdNTUYtp4+vNZmk6V10I/+UxS3pImX8Avy7ojtHGVQvRoaGwrc5+zNBdue8A4Bs4=
ARC-Authentication-Results: i=1; smtp.subspace.kernel.org;
dmarc=pass (p=none dis=none) header.from=linux.ibm.com;
spf=pass smtp.mailfrom=linux.ibm.com;
dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com
header.b=IKe6NKPK; arc=none smtp.client-ip=148.163.158.5
Authentication-Results: smtp.subspace.kernel.org;
dmarc=pass (p=none dis=none) header.from=linux.ibm.com
Authentication-Results: smtp.subspace.kernel.org;
spf=pass smtp.mailfrom=linux.ibm.com
Authentication-Results: smtp.subspace.kernel.org;
dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com
header.b="IKe6NKPK"
Received: from pps.filterd (m0360072.ppops.net [127.0.0.1])
by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id
537En0ZD029644;
Mon, 7 Apr 2025 15:11:34 GMT
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc
:content-transfer-encoding:date:from:in-reply-to:message-id
:mime-version:references:subject:to; s=pp1; bh=JAdRtQpYpJ83Wan/q
ay54BErF461aFj9Lbn233V6/Vo=; b=IKe6NKPK0RQuaHMhk18ftPiOYYmCBCWNo
B9AMFWR/OSyAEWP+S886dEpJeibdEmZl36pABv5K+wup8TiyL8NG8ViK9od6oBRM
3hfWr3Nzyj+cpTRWraAtGGyrEizF/IwgeyvpxnuqLvo0iHgyxm7f3x2y4rrF4BuZ
8DVbuOci3sgrMA9QHMuCtrLrrnIMC0eF9Lq4xmJACPC8mwWkrruApDLDO3y+TF6K
EKoPhfmZsym89+7c65PcFL6w5iov3VKT/uGxaF9PJzGWmz4HwXohtM7m83O2AHTx
G4AzXPBlc8+L56yqeI1A9XrdpDtpRX+WW/G2StWfINFf09T/NbHTA==
Received: from pps.reinject (localhost [127.0.0.1])
by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 45uwswvt33-1
(version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
Mon, 07 Apr 2025 15:11:34 +0000 (GMT)
Received: from m0360072.ppops.net (m0360072.ppops.net [127.0.0.1])
by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 537FBXTb013844;
Mon, 7 Apr 2025 15:11:33 GMT
Received: from ppma13.dal12v.mail.ibm.com
(dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221])
by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 45uwswvt2v-1
(version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
Mon, 07 Apr 2025 15:11:33 +0000 (GMT)
Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1])
by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id
537F2iM6018432;
Mon, 7 Apr 2025 15:11:32 GMT
Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228])
by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 45uh2ke5ug-1
(version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
Mon, 07 Apr 2025 15:11:32 +0000
Received: from smtpav06.fra02v.mail.ibm.com (smtpav06.fra02v.mail.ibm.com
[10.20.54.105])
by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id
537FBVQW33751652
(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
Mon, 7 Apr 2025 15:11:31 GMT
Received: from smtpav06.fra02v.mail.ibm.com (unknown [127.0.0.1])
by IMSVA (Postfix) with ESMTP id E82452004B;
Mon, 7 Apr 2025 15:11:30 +0000 (GMT)
Received: from smtpav06.fra02v.mail.ibm.com (unknown [127.0.0.1])
by IMSVA (Postfix) with ESMTP id D3F6E20049;
Mon, 7 Apr 2025 15:11:30 +0000 (GMT)
Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9])
by smtpav06.fra02v.mail.ibm.com (Postfix) with ESMTPS;
Mon, 7 Apr 2025 15:11:30 +0000 (GMT)
Received: by tuxmaker.boeblingen.de.ibm.com (Postfix, from userid 55669)
id 7F795E10FC; Mon, 07 Apr 2025 17:11:30 +0200 (CEST)
From: Alexander Gordeev <agordeev@linux.ibm.com>
To: Andrew Morton <akpm@linux-foundation.org>,
Andrey Ryabinin <ryabinin.a.a@gmail.com>
Cc: Hugh Dickins <hughd@google.com>, Nicholas Piggin <npiggin@gmail.com>,
Guenter Roeck <linux@roeck-us.net>, Juergen Gross <jgross@suse.com>,
Jeremy Fitzhardinge <jeremy@goop.org>, linux-kernel@vger.kernel.org,
linux-mm@kvack.org, kasan-dev@googlegroups.com,
sparclinux@vger.kernel.org, xen-devel@lists.xenproject.org,
linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org
Subject: [PATCH v1 1/4] kasan: Avoid sleepable page allocation from atomic
context
Date: Mon, 7 Apr 2025 17:11:27 +0200
Message-ID:
<ad1b313b6e3e1a84d2df6f686680ad78ae99710c.1744037648.git.agordeev@linux.ibm.com>
X-Mailer: git-send-email 2.45.2
In-Reply-To: <cover.1744037648.git.agordeev@linux.ibm.com>
References: <cover.1744037648.git.agordeev@linux.ibm.com>
Precedence: bulk
X-Mailing-List: linux-kernel@vger.kernel.org
List-Id: <linux-kernel.vger.kernel.org>
List-Subscribe: <mailto:linux-kernel+subscribe@vger.kernel.org>
List-Unsubscribe: <mailto:linux-kernel+unsubscribe@vger.kernel.org>
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
X-TM-AS-GCONF: 00
X-Proofpoint-GUID: noh5DKX9Jwz7KQQbR0xgtrf0Cnrok7JA
X-Proofpoint-ORIG-GUID: B9e39pOuvk43sXxp1B5Vd8FFaIlzYizr
X-Proofpoint-Virus-Version: vendor=baseguard
engine=ICAP:2.0.293,Aquarius:18.0.1095,Hydra:6.0.680,FMLib:17.12.68.34
definitions=2025-04-07_04,2025-04-03_03,2024-11-22_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0
priorityscore=1501 mlxscore=0
bulkscore=0 mlxlogscore=999 spamscore=0 adultscore=0 clxscore=1011
phishscore=0 lowpriorityscore=0 impostorscore=0 suspectscore=0
malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1
engine=8.19.0-2502280000 definitions=main-2504070104
Content-Type: text/plain; charset="utf-8"
apply_to_page_range() enters lazy MMU mode and then invokes
kasan_populate_vmalloc_pte() callback on each page table walk
iteration. The lazy MMU mode may only be entered only under
protection of the page table lock. However, the callback can
go into sleep when trying to allocate a single page.
Change __get_free_page() allocation mode from GFP_KERNEL to
GFP_ATOMIC to avoid scheduling out while in atomic context.
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
Reviewed-by: Nicholas Piggin <npiggin@gmail.com>
---
mm/kasan/shadow.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c
index 88d1c9dcb507..edfa77959474 100644
--- a/mm/kasan/shadow.c
+++ b/mm/kasan/shadow.c
@@ -301,7 +301,7 @@ static int kasan_populate_vmalloc_pte(pte_t *ptep, unsi=
gned long addr,
if (likely(!pte_none(ptep_get(ptep))))
return 0;
=20
- page =3D __get_free_page(GFP_KERNEL);
+ page =3D __get_free_page(GFP_ATOMIC);
if (!page)
return -ENOMEM;
=20
--=20
2.45.2
From nobody Mon Apr 28 17:41:40 2025
Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com
[148.163.158.5])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(No client certificate requested)
by smtp.subspace.kernel.org (Postfix) with ESMTPS id 667AD1FF7DB;
Mon, 7 Apr 2025 15:12:17 +0000 (UTC)
Authentication-Results: smtp.subspace.kernel.org;
arc=none smtp.client-ip=148.163.158.5
ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116;
t=1744038739; cv=none;
b=n77CRFaClUGqSaZjpGxuG4znystdjSTO5wxEjEB1+e5e+JLx3dPtp9Bhhar+0LqCsIUndwOWff37ZNNPPzizo/PEcLUmZfNxdNySBROtDb8VJVMxUjpubUGL4pfuSYT15DFfSmmLH+a57Knoh/9geNcXnn2cWguVdO0frtOdta0=
ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org;
s=arc-20240116; t=1744038739; c=relaxed/simple;
bh=jFVBTHiCForYlFppPibSwc11ScH6PVzHLg5zWuXyKL4=;
h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References:
MIME-Version;
b=ay35Mn0e19grqnK5M3vb617Wp90fY6kkWmleFemz7n04P0tiIRMvyKLECPYU1s561ffJxs9PmPV6nnsM7/3fkw+R497H8omP1GUKg7oeOjv6bWe8FrxSPXVxJ8V1XLWgKUmc6RYo+yhzWSc4syLhseC7ROuZd8H3hUB8Nfu0Rwg=
ARC-Authentication-Results: i=1; smtp.subspace.kernel.org;
dmarc=pass (p=none dis=none) header.from=linux.ibm.com;
spf=pass smtp.mailfrom=linux.ibm.com;
dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com
header.b=WOoAcnFv; arc=none smtp.client-ip=148.163.158.5
Authentication-Results: smtp.subspace.kernel.org;
dmarc=pass (p=none dis=none) header.from=linux.ibm.com
Authentication-Results: smtp.subspace.kernel.org;
spf=pass smtp.mailfrom=linux.ibm.com
Authentication-Results: smtp.subspace.kernel.org;
dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com
header.b="WOoAcnFv"
Received: from pps.filterd (m0360072.ppops.net [127.0.0.1])
by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id
5378IrTc029651;
Mon, 7 Apr 2025 15:11:34 GMT
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc
:content-transfer-encoding:date:from:in-reply-to:message-id
:mime-version:references:subject:to; s=pp1; bh=OcNsdUXLajTS6Up4y
b87KBBXCRng/iRPfXkDho493IU=; b=WOoAcnFvpuNxNEiTrGC3QZIq5o+jYBCOb
LyLvBIz0pxghOpRjonhLcZeU5GpFEhaJUVrjW4lEjOfClydMsoxIlT1tdJfSYTrD
w4ni3gxfKYkXD7bD8soBquTEt6UrKrs12uA2MgZo7ShPARvXziKp6H/vG/OR0j4l
xZ01w6SQLYfrZApqIj7cCKLpACusgLuv8O/JsHOSLcLihXpk7kQfCZv5I8mWLAkI
jB3Jm3YdqkahCDJoZ3Dg5tjezoL+c8gFXxALeoX8dMZALeFXSQU3kyz0vIaTpzdk
0hozKrK7Ar8LywsJy5kov8c0Zzsq4KVBjV6jp2caOQi+tFMaWRfCQ==
Received: from pps.reinject (localhost [127.0.0.1])
by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 45uwswvt35-1
(version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
Mon, 07 Apr 2025 15:11:34 +0000 (GMT)
Received: from m0360072.ppops.net (m0360072.ppops.net [127.0.0.1])
by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 537F41Wv022508;
Mon, 7 Apr 2025 15:11:33 GMT
Received: from ppma22.wdc07v.mail.ibm.com
(5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92])
by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 45uwswvt30-1
(version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
Mon, 07 Apr 2025 15:11:33 +0000 (GMT)
Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1])
by ppma22.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id
537DrCNJ011062;
Mon, 7 Apr 2025 15:11:33 GMT
Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228])
by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 45uf7yege2-1
(version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
Mon, 07 Apr 2025 15:11:33 +0000
Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com
[10.20.54.103])
by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id
537FBVtw16122324
(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
Mon, 7 Apr 2025 15:11:31 GMT
Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1])
by IMSVA (Postfix) with ESMTP id 10E992004E;
Mon, 7 Apr 2025 15:11:31 +0000 (GMT)
Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1])
by IMSVA (Postfix) with ESMTP id EDA652004B;
Mon, 7 Apr 2025 15:11:30 +0000 (GMT)
Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9])
by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTPS;
Mon, 7 Apr 2025 15:11:30 +0000 (GMT)
Received: by tuxmaker.boeblingen.de.ibm.com (Postfix, from userid 55669)
id 823D8E15AF; Mon, 07 Apr 2025 17:11:30 +0200 (CEST)
From: Alexander Gordeev <agordeev@linux.ibm.com>
To: Andrew Morton <akpm@linux-foundation.org>,
Andrey Ryabinin <ryabinin.a.a@gmail.com>
Cc: Hugh Dickins <hughd@google.com>, Nicholas Piggin <npiggin@gmail.com>,
Guenter Roeck <linux@roeck-us.net>, Juergen Gross <jgross@suse.com>,
Jeremy Fitzhardinge <jeremy@goop.org>, linux-kernel@vger.kernel.org,
linux-mm@kvack.org, kasan-dev@googlegroups.com,
sparclinux@vger.kernel.org, xen-devel@lists.xenproject.org,
linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org
Subject: [PATCH v1 2/4] mm: Cleanup apply_to_pte_range() routine
Date: Mon, 7 Apr 2025 17:11:28 +0200
Message-ID:
<93102722541b1daf541fce9fb316a1a2614d8c86.1744037648.git.agordeev@linux.ibm.com>
X-Mailer: git-send-email 2.45.2
In-Reply-To: <cover.1744037648.git.agordeev@linux.ibm.com>
References: <cover.1744037648.git.agordeev@linux.ibm.com>
Precedence: bulk
X-Mailing-List: linux-kernel@vger.kernel.org
List-Id: <linux-kernel.vger.kernel.org>
List-Subscribe: <mailto:linux-kernel+subscribe@vger.kernel.org>
List-Unsubscribe: <mailto:linux-kernel+unsubscribe@vger.kernel.org>
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
X-TM-AS-GCONF: 00
X-Proofpoint-GUID: ZjXrqfzVuI4O6PuB71DztWSnTeaDAz2F
X-Proofpoint-ORIG-GUID: B8AxCnBSacob3HedwAwYgmrsIk7eP_GG
X-Proofpoint-Virus-Version: vendor=baseguard
engine=ICAP:2.0.293,Aquarius:18.0.1095,Hydra:6.0.680,FMLib:17.12.68.34
definitions=2025-04-07_04,2025-04-03_03,2024-11-22_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0
priorityscore=1501 mlxscore=0
bulkscore=0 mlxlogscore=912 spamscore=0 adultscore=0 clxscore=1011
phishscore=0 lowpriorityscore=0 impostorscore=0 suspectscore=0
malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1
engine=8.19.0-2502280000 definitions=main-2504070104
Content-Type: text/plain; charset="utf-8"
Reverse 'create' vs 'mm =3D=3D &init_mm' conditions and move
page table mask modification out of the atomic context.
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
Reviewed-by: Nicholas Piggin <npiggin@gmail.com>
---
mm/memory.c | 28 +++++++++++++++++-----------
1 file changed, 17 insertions(+), 11 deletions(-)
diff --git a/mm/memory.c b/mm/memory.c
index 2d8c265fc7d6..f0201c8ec1ce 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -2915,24 +2915,28 @@ static int apply_to_pte_range(struct mm_struct *mm,=
pmd_t *pmd,
pte_fn_t fn, void *data, bool create,
pgtbl_mod_mask *mask)
{
+ int err =3D create ? -ENOMEM : -EINVAL;
pte_t *pte, *mapped_pte;
- int err =3D 0;
spinlock_t *ptl;
=20
- if (create) {
- mapped_pte =3D pte =3D (mm =3D=3D &init_mm) ?
- pte_alloc_kernel_track(pmd, addr, mask) :
- pte_alloc_map_lock(mm, pmd, addr, &ptl);
+ if (mm =3D=3D &init_mm) {
+ if (create)
+ pte =3D pte_alloc_kernel_track(pmd, addr, mask);
+ else
+ pte =3D pte_offset_kernel(pmd, addr);
if (!pte)
- return -ENOMEM;
+ return err;
} else {
- mapped_pte =3D pte =3D (mm =3D=3D &init_mm) ?
- pte_offset_kernel(pmd, addr) :
- pte_offset_map_lock(mm, pmd, addr, &ptl);
+ if (create)
+ pte =3D pte_alloc_map_lock(mm, pmd, addr, &ptl);
+ else
+ pte =3D pte_offset_map_lock(mm, pmd, addr, &ptl);
if (!pte)
- return -EINVAL;
+ return err;
+ mapped_pte =3D pte;
}
=20
+ err =3D 0;
arch_enter_lazy_mmu_mode();
=20
if (fn) {
@@ -2944,12 +2948,14 @@ static int apply_to_pte_range(struct mm_struct *mm,=
pmd_t *pmd,
}
} while (addr +=3D PAGE_SIZE, addr !=3D end);
}
- *mask |=3D PGTBL_PTE_MODIFIED;
=20
arch_leave_lazy_mmu_mode();
=20
if (mm !=3D &init_mm)
pte_unmap_unlock(mapped_pte, ptl);
+
+ *mask |=3D PGTBL_PTE_MODIFIED;
+
return err;
}
=20
--=20
2.45.2
From nobody Mon Apr 28 17:41:40 2025
Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com
[148.163.158.5])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(No client certificate requested)
by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2FE621F8BCC;
Mon, 7 Apr 2025 15:12:03 +0000 (UTC)
Authentication-Results: smtp.subspace.kernel.org;
arc=none smtp.client-ip=148.163.158.5
ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116;
t=1744038727; cv=none;
b=BK5OdvjdBePu3f2+N+kli5bwNXiObvq1/ikmyBeOyFYQ9tUR1ddxMxXq7rCg9Spm4De4NDuQDDTZ4S4v4Rp3sZr7RI+n0GI1EkJc4kF664JIkmrHTgJ2oFo4x+Ms+8gnarZsmRu3+OkxraSukdIbFYamnpH3EqwWykUGwYI0at8=
ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org;
s=arc-20240116; t=1744038727; c=relaxed/simple;
bh=X1e0KRygZqXTJ1367GXDzLSgd+NQHkAh4/zoRyJSDiE=;
h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References:
MIME-Version;
b=jggydIA0s8XIqYThFnVBlVty/Gc5M6cDfqETq9VTPSlSYjuyu3OLp+ApyBIW6UhlZaHya9ZpbhM0yMPaNNM/CCEsUUpqpFhpIcrvwAYUp6uJhzbem9T9rE4SVYIUkB48w7OEWzremZt5TX3AGQfNtA66NCrJBRwcH0s+VCBw/NI=
ARC-Authentication-Results: i=1; smtp.subspace.kernel.org;
dmarc=pass (p=none dis=none) header.from=linux.ibm.com;
spf=pass smtp.mailfrom=linux.ibm.com;
dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com
header.b=WCNS4jwc; arc=none smtp.client-ip=148.163.158.5
Authentication-Results: smtp.subspace.kernel.org;
dmarc=pass (p=none dis=none) header.from=linux.ibm.com
Authentication-Results: smtp.subspace.kernel.org;
spf=pass smtp.mailfrom=linux.ibm.com
Authentication-Results: smtp.subspace.kernel.org;
dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com
header.b="WCNS4jwc"
Received: from pps.filterd (m0356516.ppops.net [127.0.0.1])
by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id
5378eCst023081;
Mon, 7 Apr 2025 15:11:36 GMT
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc
:content-transfer-encoding:date:from:in-reply-to:message-id
:mime-version:references:subject:to; s=pp1; bh=ngcqzYVpe7xkxjw37
TRZ0RrIQbWfDyVW1WAk2hgrGY8=; b=WCNS4jwcRyfhgA/oGV/C7ufKUl5usySJY
JBkrPpB0Kj0xSv0oPCKUqsc+vBFB8dZYewnwf47Zz4FOayaQxY3jaFKQUDjdWPJt
ugxuydo8+0yZUmA+ohR403/w5zwKv2CqsgJbjNB2KabEV+JDdelbBoH8nco6FKOT
R5XE4tzipG+fY4VoyLYwQ+3uN9XzGnan0j6xdebHv5sDZ2LfDdbtnEUN2Il+RJjB
cmqGM+BCXcv4bJsoArQ7K3cEPEa9kiS4jW2g+IRmJx8IwWMG+l59zFFBzIXRlwbd
208F8LABLveUmxIFtMSBQJf8LY6ozCLBM9y7Hx9SlNJ6+MvGB6fBA==
Received: from pps.reinject (localhost [127.0.0.1])
by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 45v0spm8qy-1
(version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
Mon, 07 Apr 2025 15:11:36 +0000 (GMT)
Received: from m0356516.ppops.net (m0356516.ppops.net [127.0.0.1])
by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 537FBZII019687;
Mon, 7 Apr 2025 15:11:35 GMT
Received: from ppma13.dal12v.mail.ibm.com
(dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221])
by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 45v0spm8qw-1
(version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
Mon, 07 Apr 2025 15:11:35 +0000 (GMT)
Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1])
by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id
537Ehxeh017825;
Mon, 7 Apr 2025 15:11:34 GMT
Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228])
by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 45uh2ke5uh-1
(version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
Mon, 07 Apr 2025 15:11:32 +0000
Received: from smtpav06.fra02v.mail.ibm.com (smtpav06.fra02v.mail.ibm.com
[10.20.54.105])
by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id
537FBVTb14156090
(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
Mon, 7 Apr 2025 15:11:31 GMT
Received: from smtpav06.fra02v.mail.ibm.com (unknown [127.0.0.1])
by IMSVA (Postfix) with ESMTP id 0B32320049;
Mon, 7 Apr 2025 15:11:31 +0000 (GMT)
Received: from smtpav06.fra02v.mail.ibm.com (unknown [127.0.0.1])
by IMSVA (Postfix) with ESMTP id E98012004D;
Mon, 7 Apr 2025 15:11:30 +0000 (GMT)
Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9])
by smtpav06.fra02v.mail.ibm.com (Postfix) with ESMTPS;
Mon, 7 Apr 2025 15:11:30 +0000 (GMT)
Received: by tuxmaker.boeblingen.de.ibm.com (Postfix, from userid 55669)
id 85284E1613; Mon, 07 Apr 2025 17:11:30 +0200 (CEST)
From: Alexander Gordeev <agordeev@linux.ibm.com>
To: Andrew Morton <akpm@linux-foundation.org>,
Andrey Ryabinin <ryabinin.a.a@gmail.com>
Cc: Hugh Dickins <hughd@google.com>, Nicholas Piggin <npiggin@gmail.com>,
Guenter Roeck <linux@roeck-us.net>, Juergen Gross <jgross@suse.com>,
Jeremy Fitzhardinge <jeremy@goop.org>, linux-kernel@vger.kernel.org,
linux-mm@kvack.org, kasan-dev@googlegroups.com,
sparclinux@vger.kernel.org, xen-devel@lists.xenproject.org,
linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org
Subject: [PATCH v1 3/4] mm: Protect kernel pgtables in apply_to_pte_range()
Date: Mon, 7 Apr 2025 17:11:29 +0200
Message-ID:
<11dbe3ac88130dbd2b8554f9369cd93fe138c655.1744037648.git.agordeev@linux.ibm.com>
X-Mailer: git-send-email 2.45.2
In-Reply-To: <cover.1744037648.git.agordeev@linux.ibm.com>
References: <cover.1744037648.git.agordeev@linux.ibm.com>
Precedence: bulk
X-Mailing-List: linux-kernel@vger.kernel.org
List-Id: <linux-kernel.vger.kernel.org>
List-Subscribe: <mailto:linux-kernel+subscribe@vger.kernel.org>
List-Unsubscribe: <mailto:linux-kernel+unsubscribe@vger.kernel.org>
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
X-TM-AS-GCONF: 00
X-Proofpoint-GUID: Gsml-lqe4-oFygMCgc_o4kWGTfpeXFVg
X-Proofpoint-ORIG-GUID: 9HS87Mcesg4PbHflITRoaij8qwSTG9RL
X-Proofpoint-Virus-Version: vendor=baseguard
engine=ICAP:2.0.293,Aquarius:18.0.1095,Hydra:6.0.680,FMLib:17.12.68.34
definitions=2025-04-07_04,2025-04-03_03,2024-11-22_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0
malwarescore=0 clxscore=1011
bulkscore=0 impostorscore=0 suspectscore=0 lowpriorityscore=0 mlxscore=0
adultscore=0 phishscore=0 priorityscore=1501 spamscore=0 mlxlogscore=828
classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2502280000
definitions=main-2504070104
Content-Type: text/plain; charset="utf-8"
The lazy MMU mode can only be entered and left under the protection
of the page table locks for all page tables which may be modified.
Yet, when it comes to kernel mappings apply_to_pte_range() does not
take any locks. That does not conform arch_enter|leave_lazy_mmu_mode()
semantics and could potentially lead to re-schedulling a process while
in lazy MMU mode or racing on a kernel page table updates.
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
Reviewed-by: Nicholas Piggin <npiggin@gmail.com>
---
mm/kasan/shadow.c | 7 ++-----
mm/memory.c | 5 ++++-
2 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c
index edfa77959474..6531a7aa8562 100644
--- a/mm/kasan/shadow.c
+++ b/mm/kasan/shadow.c
@@ -308,14 +308,14 @@ static int kasan_populate_vmalloc_pte(pte_t *ptep, un=
signed long addr,
__memset((void *)page, KASAN_VMALLOC_INVALID, PAGE_SIZE);
pte =3D pfn_pte(PFN_DOWN(__pa(page)), PAGE_KERNEL);
=20
- spin_lock(&init_mm.page_table_lock);
if (likely(pte_none(ptep_get(ptep)))) {
set_pte_at(&init_mm, addr, ptep, pte);
page =3D 0;
}
- spin_unlock(&init_mm.page_table_lock);
+
if (page)
free_page(page);
+
return 0;
}
=20
@@ -401,13 +401,10 @@ static int kasan_depopulate_vmalloc_pte(pte_t *ptep, =
unsigned long addr,
=20
page =3D (unsigned long)__va(pte_pfn(ptep_get(ptep)) << PAGE_SHIFT);
=20
- spin_lock(&init_mm.page_table_lock);
-
if (likely(!pte_none(ptep_get(ptep)))) {
pte_clear(&init_mm, addr, ptep);
free_page(page);
}
- spin_unlock(&init_mm.page_table_lock);
=20
return 0;
}
diff --git a/mm/memory.c b/mm/memory.c
index f0201c8ec1ce..1f3727104e99 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -2926,6 +2926,7 @@ static int apply_to_pte_range(struct mm_struct *mm, p=
md_t *pmd,
pte =3D pte_offset_kernel(pmd, addr);
if (!pte)
return err;
+ spin_lock(&init_mm.page_table_lock);
} else {
if (create)
pte =3D pte_alloc_map_lock(mm, pmd, addr, &ptl);
@@ -2951,7 +2952,9 @@ static int apply_to_pte_range(struct mm_struct *mm, p=
md_t *pmd,
=20
arch_leave_lazy_mmu_mode();
=20
- if (mm !=3D &init_mm)
+ if (mm =3D=3D &init_mm)
+ spin_unlock(&init_mm.page_table_lock);
+ else
pte_unmap_unlock(mapped_pte, ptl);
=20
*mask |=3D PGTBL_PTE_MODIFIED;
--=20
2.45.2
From nobody Mon Apr 28 17:41:40 2025
Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com
[148.163.156.1])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(No client certificate requested)
by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4D0931F8BCD;
Mon, 7 Apr 2025 15:12:03 +0000 (UTC)
Authentication-Results: smtp.subspace.kernel.org;
arc=none smtp.client-ip=148.163.156.1
ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116;
t=1744038725; cv=none;
b=Mj/PugCmOi/NSMpC/Xx++T43B9z8p4iZUhiQl8NP6YEBVZxrnywfktOinMFJHoUMy9DUkVaQk/kXyaRWx5DeOjrY2LhFEN776Ujcb0fdRHDIFZanZOWO8KKIpkHEA2vynGGJrxNljlh2TKLkHjSg4avpxslrPKzlYlnmIbvvhgw=
ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org;
s=arc-20240116; t=1744038725; c=relaxed/simple;
bh=L5M52uoR+VNHr6Aqz1at4XGsAPpzStQXAW03Nn5h+w0=;
h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References:
MIME-Version;
b=WsvRSG0809c5PFJABKT2CVvWic2BrXNQnRut+jwISW23rjiE58COC8pOZx04aELau1oQoaQ8ad01V9LVJkW6c2V86ZxA+/+krzYKqF2n7XB61fClLJ+DASnTTH0d3gRuCAFzVhjmPb8983dvaBjtseB/t3AczJztQZ+X+uUySwg=
ARC-Authentication-Results: i=1; smtp.subspace.kernel.org;
dmarc=pass (p=none dis=none) header.from=linux.ibm.com;
spf=pass smtp.mailfrom=linux.ibm.com;
dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com
header.b=GG9/qRMT; arc=none smtp.client-ip=148.163.156.1
Authentication-Results: smtp.subspace.kernel.org;
dmarc=pass (p=none dis=none) header.from=linux.ibm.com
Authentication-Results: smtp.subspace.kernel.org;
spf=pass smtp.mailfrom=linux.ibm.com
Authentication-Results: smtp.subspace.kernel.org;
dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com
header.b="GG9/qRMT"
Received: from pps.filterd (m0356517.ppops.net [127.0.0.1])
by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id
537E3FIb025560;
Mon, 7 Apr 2025 15:11:35 GMT
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc
:content-transfer-encoding:date:from:in-reply-to:message-id
:mime-version:references:subject:to; s=pp1; bh=pLqNtHwxdz+KjzF0o
4al3IuUdgKeg437/opuwF8rOgY=; b=GG9/qRMTr3c234BdNbl8qcd6JICn1dr6T
h+1CwgnJosxDoTGQHDJLnBC+r8sdfje311yGnSrC48KNz6kVG3ETEbXxkIldtS4P
DeWcQ/wSyDFqcrEo/l5mNQ2K0Lp2iKaRYkSmtxXTFAGqSvnaN8aWIz5IKTOTOKFq
+JDtlQ3sQxmiVDXWN/y5otPtf5J6aL25Ojv2dz/9vWnkrf3ALjS2QQAfJwUkVTgb
dIt52+/rxwX4BWgeopjfh5rB3DIp5dxMR6YbMdiedI2eDNzrYRJVARzz4+Y3g9I/
CxasV3YuMtzXVUxrwLXF7t0esmIiJJgRgZro+hqrnkdilacrPiCpg==
Received: from pps.reinject (localhost [127.0.0.1])
by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 45vg4q8cmm-1
(version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
Mon, 07 Apr 2025 15:11:34 +0000 (GMT)
Received: from m0356517.ppops.net (m0356517.ppops.net [127.0.0.1])
by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 537EoQ3W019216;
Mon, 7 Apr 2025 15:11:34 GMT
Received: from ppma21.wdc07v.mail.ibm.com
(5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91])
by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 45vg4q8cmf-1
(version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
Mon, 07 Apr 2025 15:11:34 +0000 (GMT)
Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1])
by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id
537EKIMr013925;
Mon, 7 Apr 2025 15:11:33 GMT
Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228])
by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 45ufunecfm-1
(version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
Mon, 07 Apr 2025 15:11:32 +0000
Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com
[10.20.54.103])
by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id
537FBVTI19595766
(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
Mon, 7 Apr 2025 15:11:31 GMT
Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1])
by IMSVA (Postfix) with ESMTP id 03DF82004D;
Mon, 7 Apr 2025 15:11:31 +0000 (GMT)
Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1])
by IMSVA (Postfix) with ESMTP id E2DFB20043;
Mon, 7 Apr 2025 15:11:30 +0000 (GMT)
Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9])
by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTPS;
Mon, 7 Apr 2025 15:11:30 +0000 (GMT)
Received: by tuxmaker.boeblingen.de.ibm.com (Postfix, from userid 55669)
id 88272E175F; Mon, 07 Apr 2025 17:11:30 +0200 (CEST)
From: Alexander Gordeev <agordeev@linux.ibm.com>
To: Andrew Morton <akpm@linux-foundation.org>,
Andrey Ryabinin <ryabinin.a.a@gmail.com>
Cc: Hugh Dickins <hughd@google.com>, Nicholas Piggin <npiggin@gmail.com>,
Guenter Roeck <linux@roeck-us.net>, Juergen Gross <jgross@suse.com>,
Jeremy Fitzhardinge <jeremy@goop.org>, linux-kernel@vger.kernel.org,
linux-mm@kvack.org, kasan-dev@googlegroups.com,
sparclinux@vger.kernel.org, xen-devel@lists.xenproject.org,
linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org
Subject: [PATCH v1 4/4] mm: Allow detection of wrong
arch_enter_lazy_mmu_mode() context
Date: Mon, 7 Apr 2025 17:11:30 +0200
Message-ID:
<5204eaec309f454efcb5a799c9e0ed9da1dff971.1744037648.git.agordeev@linux.ibm.com>
X-Mailer: git-send-email 2.45.2
In-Reply-To: <cover.1744037648.git.agordeev@linux.ibm.com>
References: <cover.1744037648.git.agordeev@linux.ibm.com>
Precedence: bulk
X-Mailing-List: linux-kernel@vger.kernel.org
List-Id: <linux-kernel.vger.kernel.org>
List-Subscribe: <mailto:linux-kernel+subscribe@vger.kernel.org>
List-Unsubscribe: <mailto:linux-kernel+unsubscribe@vger.kernel.org>
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
X-TM-AS-GCONF: 00
X-Proofpoint-ORIG-GUID: wG_AoPwbLBLy8-Y75yYVC-cpQvofPbGp
X-Proofpoint-GUID: Yijp_AWT6tzwOtSLybGJbVPGYVge3z23
X-Proofpoint-Virus-Version: vendor=baseguard
engine=ICAP:2.0.293,Aquarius:18.0.1095,Hydra:6.0.680,FMLib:17.12.68.34
definitions=2025-04-07_04,2025-04-03_03,2024-11-22_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0
adultscore=0 impostorscore=0
mlxscore=0 priorityscore=1501 phishscore=0 mlxlogscore=935 malwarescore=0
clxscore=1015 suspectscore=0 spamscore=0 bulkscore=0 lowpriorityscore=0
classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2502280000
definitions=main-2504070104
Content-Type: text/plain; charset="utf-8"
The lazy MMU batching may be only be entered and left under the
protection of the page table locks for all page tables which may
be modified. Yet, there were cases arch_enter_lazy_mmu_mode()
was called without the locks taken, e.g. commit b9ef323ea168
("powerpc/64s: Disable preemption in hash lazy mmu mode").
Make default arch_enter|leave|flush_lazy_mmu_mode() callbacks
complain at least in case the preemption is enabled to detect
wrong contexts.
Most platforms do not implement the callbacks, so to aovid a
performance impact allow the complaint when CONFIG_DEBUG_VM
option is enabled only.
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
Reviewed-by: Nicholas Piggin <npiggin@gmail.com>
---
include/linux/pgtable.h | 15 ++++++++++++---
1 file changed, 12 insertions(+), 3 deletions(-)
diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h
index e2b705c14945..959590bb66da 100644
--- a/include/linux/pgtable.h
+++ b/include/linux/pgtable.h
@@ -232,9 +232,18 @@ static inline int pmd_dirty(pmd_t pmd)
* and the mode cannot be used in interrupt context.
*/
#ifndef __HAVE_ARCH_ENTER_LAZY_MMU_MODE
-#define arch_enter_lazy_mmu_mode() do {} while (0)
-#define arch_leave_lazy_mmu_mode() do {} while (0)
-#define arch_flush_lazy_mmu_mode() do {} while (0)
+static inline void arch_enter_lazy_mmu_mode(void)
+{
+ VM_WARN_ON(preemptible());
+}
+static inline void arch_leave_lazy_mmu_mode(void)
+{
+ VM_WARN_ON(preemptible());
+}
+static inline void arch_flush_lazy_mmu_mode(void)
+{
+ VM_WARN_ON(preemptible());
+}
#endif
=20
#ifndef pte_batch_hint
--=20
2.45.2