From nobody Fri Nov 29 14:46:21 2024 Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2763D17588 for ; Thu, 19 Sep 2024 02:56:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726714592; cv=none; b=N6M/Y/bxMjbNEivHx57w5adtfQBOwASV2F65ILLBdNNEpe1IKAkn7MZ+l+QVM8ydqtkLaQbwmycDq7xAvL1CNZXYB5P61h4loLnxkSVJ9MPHjUIexHBchHE3y0QeyhasfBwhQf0fPeaG3U2+V5YQw+w/LbMIBZApemrDvQ/9f28= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726714592; c=relaxed/simple; bh=ENvgyjReStAE41tUUhhU7m+p1u0Q16yYcDmxWcLik8o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kE/7rVRvRMLrx6ZZLS0Aw27t6tXWE/hLFIF5eK/yzOiwKDi5xOimlXeg/vY12JPeLTu8y5+3QxmpYnYWfRH554TAJYAfKYe/MJXyNsP6XZBTGaQc0JYGfEP/ICTRomf/Ln8EJ3qrkW5U9lpJLJbnuTXv7fTt8ozV7lsDuzOr80c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=X1GfgXYb; arc=none smtp.client-ip=209.85.214.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="X1GfgXYb" Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-207115e3056so3701385ad.2 for ; Wed, 18 Sep 2024 19:56:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726714590; x=1727319390; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=L3z8aqN2PapGEchw6ngaiDWNeD417V3WkhQBwYXWFqM=; b=X1GfgXYbTWT+C+eL4PjaUDge1yvjGbwJCnnhBYS1NuTwQQMnkVw+afU8fkq05kdRK7 LV/vxuBODHqSeJtd7MBy0EjsOEsBeeAOjXq67cYLg2OkzWJP1Ag5dcOnf/NTBDOXDDr2 5bi8D9ihGbAOHWdG/rOeNRilb9j0jN4vcMqDFfhPlmUzDHDj190wAQTrtlJ6LZ2l/8id fJ+9tQ+K9EG6m/pmA5P2I5a22XlIizEz4Xl+pFVoz50XEVpY7V5zfZwKFjzZ5vciBTUG fWwJp5wxbr7yAi4iTfaZOCAA8j/TBxEkNAJN7fI/ZTIkXy2F2pfR11aLbEkVrrytfM8s HZxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726714590; x=1727319390; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=L3z8aqN2PapGEchw6ngaiDWNeD417V3WkhQBwYXWFqM=; b=t609lF/pzxiJ0EvGzbYg6W0RD/78tjAAaTZ3dPwThrmmlr99yCJm9LNryKD0D0waSd GIndRsc8d2VB2HgsJrW0SMCEsnsqwH80D51iAbwXgVNkDe7eOYPf9P0EdvZY1kWqukZG DYvvyU7IIPCSgYAgFYTugAD5nZg7ykW0r5KX9jC8X8Uq32Vb8DBiX+hh/89sB1tXqguh NawydcQOvokH5Yym5ZQynevUnqYACKitZEk6dJqdAAY+9nj3Y+Uat0wgXzeaNPLfAcay vhN4R+O/bRKLMvtIlDKNiZYgZ3gnvdhK3kTnnMlvLoffUsq8EPb+bkojARoMWEpdS4Je QNxw== X-Forwarded-Encrypted: i=1; AJvYcCVw7GVSaddxD7/0o83oX43I1OS0gJ2yuaCfYFPjxeRKnu093z+jU7AlHU7Ri92uZWfUooBnE6cV9eF7HfE=@vger.kernel.org X-Gm-Message-State: AOJu0YxKWlvb6AtVFaNTkqM545znnxzA7J0Cuy43jq+jgWIDQpsYYs1R sBA51O+ypCO1aA0O9emqo8w0A43LKNvFQZWVEGElopkUBN1mCrtc X-Google-Smtp-Source: AGHT+IG3E3x5sXQMsxEWayCG7th6zZkwGaojJWrlU5aw9OEqWkqNzOuRCe7304FOOtHaaUGVabeHyA== X-Received: by 2002:a17:902:da84:b0:206:c911:9d75 with SMTP id d9443c01a7336-2076e3b2654mr336649595ad.20.1726714590396; Wed, 18 Sep 2024 19:56:30 -0700 (PDT) Received: from dw-tp.. ([171.76.85.129]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-207946d2823sm71389105ad.148.2024.09.18.19.56.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Sep 2024 19:56:29 -0700 (PDT) From: "Ritesh Harjani (IBM)" To: linuxppc-dev@lists.ozlabs.org Cc: Michael Ellerman , Nicholas Piggin , Madhavan Srinivasan , Christophe Leroy , Hari Bathini , "Aneesh Kumar K . V" , Donet Tom , Pavithra Prakash , Nirjhar Roy , LKML , kasan-dev@googlegroups.com, "Ritesh Harjani (IBM)" , Alexander Potapenko , linux-mm@kvack.org Subject: [RFC v2 01/13] mm/kfence: Add a new kunit test test_use_after_free_read_nofault() Date: Thu, 19 Sep 2024 08:25:59 +0530 Message-ID: X-Mailer: git-send-email 2.46.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Nirjhar Roy Faults from copy_from_kernel_nofault() needs to be handled by fixup table and should not be handled by kfence. Otherwise while reading /proc/kcore which uses copy_from_kernel_nofault(), kfence can generate false negatives. This can happen when /proc/kcore ends up reading an unmapped address from kfence pool. Let's add a testcase to cover this case. Co-developed-by: Ritesh Harjani (IBM) Signed-off-by: Ritesh Harjani (IBM) Signed-off-by: Nirjhar Roy Cc: kasan-dev@googlegroups.com Cc: Alexander Potapenko Cc: linux-mm@kvack.org --- mm/kfence/kfence_test.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/mm/kfence/kfence_test.c b/mm/kfence/kfence_test.c index 00fd17285285..f65fb182466d 100644 --- a/mm/kfence/kfence_test.c +++ b/mm/kfence/kfence_test.c @@ -383,6 +383,22 @@ static void test_use_after_free_read(struct kunit *tes= t) KUNIT_EXPECT_TRUE(test, report_matches(&expect)); } +static void test_use_after_free_read_nofault(struct kunit *test) +{ + const size_t size =3D 32; + char *addr; + char dst; + int ret; + + setup_test_cache(test, size, 0, NULL); + addr =3D test_alloc(test, size, GFP_KERNEL, ALLOCATE_ANY); + test_free(addr); + /* Use after free with *_nofault() */ + ret =3D copy_from_kernel_nofault(&dst, addr, 1); + KUNIT_EXPECT_EQ(test, ret, -EFAULT); + KUNIT_EXPECT_FALSE(test, report_available()); +} + static void test_double_free(struct kunit *test) { const size_t size =3D 32; @@ -780,6 +796,7 @@ static struct kunit_case kfence_test_cases[] =3D { KFENCE_KUNIT_CASE(test_out_of_bounds_read), KFENCE_KUNIT_CASE(test_out_of_bounds_write), KFENCE_KUNIT_CASE(test_use_after_free_read), + KFENCE_KUNIT_CASE(test_use_after_free_read_nofault), KFENCE_KUNIT_CASE(test_double_free), KFENCE_KUNIT_CASE(test_invalid_addr_free), KFENCE_KUNIT_CASE(test_corruption), -- 2.46.0 From nobody Fri Nov 29 14:46:21 2024 Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 378771B28A for ; Thu, 19 Sep 2024 02:56:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726714597; cv=none; b=fcBt/W3aNNXBJUPqVHV4vDi95i33aYAMkSPGLj22Ou8RGlDA2kMgnENpWlfhN87NlJe2ooFbXnRg2bgzkAg/6RmLUSU33p5zG21aVpV3bCZoz19i6yplRO74ImbPaDYIbznwZpmK68IooDUfCbscCFZzHQMW5RnCMTscIbqUUCQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726714597; c=relaxed/simple; bh=m+dUswfxUsZy9prkRABXDXGfNap5/jDlWlqOFp9ZXy0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=EYWF5qtAimV/sPCngjdu3dXMhSzps3e1+3RUywhRIlHQOhL2MpZs3ud6kMuo+j6bnSRXEilibKfHqNAzK2Q+OvEc5RLTFeqLqt+UzsZxtylpEHCjr0OokUdrorWIeFoktb6hCb45eZIIGtRupYk/BztrD9YQX3cgk43M+p9Q4G0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=PfONtK4Q; arc=none smtp.client-ip=209.85.214.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="PfONtK4Q" Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-205722ba00cso3623795ad.0 for ; Wed, 18 Sep 2024 19:56:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726714595; x=1727319395; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2IOurK7ENej5lHHiP4d73lr8yQG82cdevQPBO00m0fo=; b=PfONtK4Qu3z7GUQqBkH0XkOFI406KheOFa3ONbqDc34OEKIQOmBN47kBT3H/thXH2p gCy+HGUQXz8R/XzsT7palKxPoJjLn1iwCGMQM9BwHNu4/Gzgkyaeuv5JSs4zdCep5/D1 qr1Q8DMQVw+xTscT3MWHx2FPQw6hk+JE3m6uXVTYMZLtcr3+8zl4RT+27TvcxJWOsuqq UpLQfius9TSsXkPvzrzpnKcqBRPA9Q43k96/wzGqQ8CmvthisKe7mTQWOdzSoqlw9CrY DTsAKxKBe5pKYHfcl636ujoiITSbXieoCedLrOw6MYsSOKj7tLRqTJd6tJOTd9/Y3Cvn vICQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726714595; x=1727319395; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2IOurK7ENej5lHHiP4d73lr8yQG82cdevQPBO00m0fo=; b=Hc0CnvdPbR0AkRQwR777EUP6VfxeH+/4In+gTK69cbYDah9Fhxl9/IWVGrDV/BHFmh chWJfaoBFerA32NSQ4IlpLyjy4ar6JOFUpKymC4z+XJ+bTf2OrOWtwoJWT1lc5exmgka GWL9c0SxyiHH7XkIUWe3jbmnKzLqSfpour8KEmSyPxoB3tX3ABE4k223+n4s4hAYK0To buTUWlV/757YrEZQzh6rNem5IpH/Zuw4SXrLfG21joXQzKTxQEXaenzwjq/DvEGnUYmy hV67ESGPOoJpC47i7SoF7abWmRdpMtigYj36D6H+jx72U1s2BUFJG0qvsbFX56vPaTo3 8KAg== X-Forwarded-Encrypted: i=1; AJvYcCUWj3/bK9EVXokTVDKoez+bBLJoRwoA80kbccsNmfaX26eQoL0OSI12HgchRsyD1XarLyzMRLH8hV7WrVs=@vger.kernel.org X-Gm-Message-State: AOJu0YzIKv4xCUdOjBuqHV9DfyPbczeCfRw0BCzSOJ94rPkOajiX4+5h RUnLUQXTT45fM1vH54zQMT0lGqACx4DSF0+RJFZ3wNnAGg3yNYA06pBT5w== X-Google-Smtp-Source: AGHT+IHfTwM1lnJEl3rUUm339wjWE2P+DCcagT96OnRmH6xTvh9VgnS8vtwxnlLDNUUy7qVft2NHpg== X-Received: by 2002:a17:902:dad0:b0:1fd:5eab:8c76 with SMTP id d9443c01a7336-2076e462c42mr369019805ad.41.1726714595450; Wed, 18 Sep 2024 19:56:35 -0700 (PDT) Received: from dw-tp.. ([171.76.85.129]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-207946d2823sm71389105ad.148.2024.09.18.19.56.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Sep 2024 19:56:34 -0700 (PDT) From: "Ritesh Harjani (IBM)" To: linuxppc-dev@lists.ozlabs.org Cc: Michael Ellerman , Nicholas Piggin , Madhavan Srinivasan , Christophe Leroy , Hari Bathini , "Aneesh Kumar K . V" , Donet Tom , Pavithra Prakash , Nirjhar Roy , LKML , kasan-dev@googlegroups.com, "Ritesh Harjani (IBM)" , Disha Goel Subject: [RFC v2 02/13] powerpc: mm: Fix kfence page fault reporting Date: Thu, 19 Sep 2024 08:26:00 +0530 Message-ID: <87095ffca1e3b932c495942defc598907bf955f6.1726571179.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" copy_from_kernel_nofault() can be called when doing read of /proc/kcore. /proc/kcore can have some unmapped kfence objects which when read via copy_from_kernel_nofault() can cause page faults. Since *_nofault() functions define their own fixup table for handling fault, use that instead of asking kfence to handle such faults. Hence we search the exception tables for the nip which generated the fault. If there is an entry then we let the fixup table handler handle the page fault by returning an error from within ___do_page_fault(). This can be easily triggered if someone tries to do dd from /proc/kcore. dd if=3D/proc/kcore of=3D/dev/null bs=3D1M =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D BUG: KFENCE: invalid read in copy_from_kernel_nofault+0xb0/0x1c8 Invalid read at 0x000000004f749d2e: copy_from_kernel_nofault+0xb0/0x1c8 0xc0000000057f7950 read_kcore_iter+0x41c/0x9ac proc_reg_read_iter+0xe4/0x16c vfs_read+0x2e4/0x3b0 ksys_read+0x88/0x154 system_call_exception+0x124/0x340 system_call_common+0x160/0x2c4 BUG: KFENCE: use-after-free read in copy_from_kernel_nofault+0xb0/0x1c8 Use-after-free read at 0x000000008fbb08ad (in kfence-#0): copy_from_kernel_nofault+0xb0/0x1c8 0xc0000000057f7950 read_kcore_iter+0x41c/0x9ac proc_reg_read_iter+0xe4/0x16c vfs_read+0x2e4/0x3b0 ksys_read+0x88/0x154 system_call_exception+0x124/0x340 system_call_common+0x160/0x2c4 Guessing the fix should go back to when we first got kfence on PPC32. Fixes: 90cbac0e995d ("powerpc: Enable KFENCE for PPC32") Reported-by: Disha Goel Signed-off-by: Ritesh Harjani (IBM) --- arch/powerpc/mm/fault.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/arch/powerpc/mm/fault.c b/arch/powerpc/mm/fault.c index 81c77ddce2e3..fa825198f29f 100644 --- a/arch/powerpc/mm/fault.c +++ b/arch/powerpc/mm/fault.c @@ -439,9 +439,17 @@ static int ___do_page_fault(struct pt_regs *regs, unsi= gned long address, /* * The kernel should never take an execute fault nor should it * take a page fault to a kernel address or a page fault to a user - * address outside of dedicated places + * address outside of dedicated places. + * + * Rather than kfence reporting false negatives, let the fixup table + * handler handle the page fault by returning SIGSEGV, if the fault + * has come from functions like copy_from_kernel_nofault(). */ if (unlikely(!is_user && bad_kernel_fault(regs, error_code, address, is_w= rite))) { + + if (search_exception_tables(instruction_pointer(regs))) + return SIGSEGV; + if (kfence_handle_page_fault(address, is_write, regs)) return 0; =20 --=20 2.46.0 From nobody Fri Nov 29 14:46:21 2024 Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0B8061C68F for ; Thu, 19 Sep 2024 02:56:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726714602; cv=none; b=vAtP+PQicNHIv1kxrh2Fb/CYyVBShf6bCMZQtKuhueXgI28siNDVWEwQ21NyfUvv0ut4Vu8sOXYf2Ta9Vil0mpS8lo9pOdkDzWC7PObg8X1cL00Wn0PjSvFAvtiKe3ru6pSdAWZ9Br1/Ka38tvnAXvVVELKMOiZbxafvrxHxHik= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726714602; c=relaxed/simple; bh=A/alsdJ4ayr16CiiF9B4T3/cT2Sxc3m5R8nEMlHIXkY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hMtKQBKUCZS+/d7zGanM7nUVCaRO2oFOPtgcy6NXCO9Dwlb0sukpHNnqIlKsI37egVKYWb4A88qikcAJOOTCKgIgv+eJneHupVOl31lcl+S5UovYn++JWY24+2yCRvcBpUdjvdy6SMR4L+bvJuxzErTetExCo6QaoUTg5zPukK0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=dVCeC1xx; arc=none smtp.client-ip=209.85.214.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="dVCeC1xx" Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-2057835395aso4873205ad.3 for ; Wed, 18 Sep 2024 19:56:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726714600; x=1727319400; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9gdBU6NTSnQ45ASWgUOhpCdQmMAIjthCFPvKHCG5k1I=; b=dVCeC1xx0kY3uo67c04Xqr+IU+/F+q+d0A+kvJNhMZRYIVuJhVCgFfLCq6gE8MQMTi A6KqVk8nSFB4ulEMU8NW8PnP27d5ww/dhtb4OuQQ2F4i6RAd9HrOmU/r2d0KEJrlL+2S vCtqIqsYsYk4xTDd0ofmYdsYNsnnQBHA7wynS2b7q0AacIEtOx3NO6Lf5+jfwR5eG0bf czT7BkynX/ecgHV+T0rFdsIxILGiJ8lt2uiQTMBWSYM8mNoIdOzWb5yYb+pB3iGr27T/ tS7YNFU4s3drsalN+/M6QQJ0dUdLT3S8b2TBQdxHQAdSWJI3cLffogBaPuDeRdSzf1ye CfeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726714600; x=1727319400; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9gdBU6NTSnQ45ASWgUOhpCdQmMAIjthCFPvKHCG5k1I=; b=LbfmZl7Lpj6nq+4XREs6XULQZx4JMRw54wNpM/KbNwweqM+O0UAVWNUuG4jRVW5rGX SS6JQUPXvMhD5j/bEvbH9oXZgxNyloEqidHwlIji2056hJfmLkpeI5+JPVRDigKfk6sZ v1fv2Rg0OAlWbJ9eQc78xtuCuUl4zLJ6qXXxb30jYKThSGYMFSvbUpWKquTJK02efIt2 RE9ONouxfveeRCOM2fc+83Lm+C8Hj7+vV7yuR9JvoVzS49GQWzuQFbJuMQ6+S2V967qb XIaurZEIR6dN2tL/AH2Xd8SYGnmkD9LABF1hSPisUq9qjC7ZzKaRwwa7qgUhCvdfhrbD E+hQ== X-Forwarded-Encrypted: i=1; AJvYcCXV+Ub+iVUD0v0D8I4BHMLIDoil/+XGONoHcbpCIPhnMJMgQm70VH0o2sPWtp0x91KQAchBgPAwFRSfOaE=@vger.kernel.org X-Gm-Message-State: AOJu0YyedqXGQi1m+1XV7/gk5zeTnYifXmg1ZCozXmn6GNxY99bSntcf mR1pptlwOqtOq+lL6WdcBhZRZ9DJdTE8Cz58SfoZFYpNbIxYbF6w X-Google-Smtp-Source: AGHT+IEErXbCyxVJ/WoGn8BSiRcCs9SEMeQSmif53htHavGlyC9/1FoEX9bAXEfU4wjU0v/bb5Bxuw== X-Received: by 2002:a17:902:e812:b0:206:c75a:29d9 with SMTP id d9443c01a7336-2076e3eb0cbmr391823725ad.42.1726714600089; Wed, 18 Sep 2024 19:56:40 -0700 (PDT) Received: from dw-tp.. ([171.76.85.129]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-207946d2823sm71389105ad.148.2024.09.18.19.56.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Sep 2024 19:56:39 -0700 (PDT) From: "Ritesh Harjani (IBM)" To: linuxppc-dev@lists.ozlabs.org Cc: Michael Ellerman , Nicholas Piggin , Madhavan Srinivasan , Christophe Leroy , Hari Bathini , "Aneesh Kumar K . V" , Donet Tom , Pavithra Prakash , Nirjhar Roy , LKML , kasan-dev@googlegroups.com, "Ritesh Harjani (IBM)" Subject: [RFC v2 03/13] book3s64/hash: Remove kfence support temporarily Date: Thu, 19 Sep 2024 08:26:01 +0530 Message-ID: <5f6809f3881d5929eedc33deac4847bf41a063b9.1726571179.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Kfence on book3s Hash on pseries is anyways broken. It fails to boot due to RMA size limitation. That is because, kfence with Hash uses debug_pagealloc infrastructure. debug_pagealloc allocates linear map for entire dram size instead of just kfence relevant objects. This means for 16TB of DRAM it will require (16TB >> PAGE_SHIFT) which is 256MB which is half of RMA region on P8. crash kernel reserves 256MB and we also need 2048 * 16KB * 3 for emergency stack and some more for paca allocations. That means there is not enough memory for reserving the full linear map in the RMA region, if the DRAM size is too big (>=3D16TB) (The issue is seen above 8TB with crash kernel 256 MB reservation). Now Kfence does not require linear memory map for entire DRAM. It only needs for kfence objects. So this patch temporarily removes the kfence functionality since debug_pagealloc code needs some refactoring. We will bring in kfence on Hash support in later patches. Signed-off-by: Ritesh Harjani (IBM) --- arch/powerpc/include/asm/kfence.h | 5 +++++ arch/powerpc/mm/book3s64/hash_utils.c | 16 +++++++++++----- 2 files changed, 16 insertions(+), 5 deletions(-) diff --git a/arch/powerpc/include/asm/kfence.h b/arch/powerpc/include/asm/k= fence.h index fab124ada1c7..f3a9476a71b3 100644 --- a/arch/powerpc/include/asm/kfence.h +++ b/arch/powerpc/include/asm/kfence.h @@ -10,6 +10,7 @@ =20 #include #include +#include =20 #ifdef CONFIG_PPC64_ELF_ABI_V1 #define ARCH_FUNC_PREFIX "." @@ -25,6 +26,10 @@ static inline void disable_kfence(void) =20 static inline bool arch_kfence_init_pool(void) { +#ifdef CONFIG_PPC64 + if (!radix_enabled()) + return false; +#endif return !kfence_disabled; } #endif diff --git a/arch/powerpc/mm/book3s64/hash_utils.c b/arch/powerpc/mm/book3s= 64/hash_utils.c index e1eadd03f133..296bb74dbf40 100644 --- a/arch/powerpc/mm/book3s64/hash_utils.c +++ b/arch/powerpc/mm/book3s64/hash_utils.c @@ -431,7 +431,7 @@ int htab_bolt_mapping(unsigned long vstart, unsigned lo= ng vend, break; =20 cond_resched(); - if (debug_pagealloc_enabled_or_kfence() && + if (debug_pagealloc_enabled() && (paddr >> PAGE_SHIFT) < linear_map_hash_count) linear_map_hash_slots[paddr >> PAGE_SHIFT] =3D ret | 0x80; } @@ -814,7 +814,7 @@ static void __init htab_init_page_sizes(void) bool aligned =3D true; init_hpte_page_sizes(); =20 - if (!debug_pagealloc_enabled_or_kfence()) { + if (!debug_pagealloc_enabled()) { /* * Pick a size for the linear mapping. Currently, we only * support 16M, 1M and 4K which is the default @@ -1134,7 +1134,7 @@ static void __init htab_initialize(void) =20 prot =3D pgprot_val(PAGE_KERNEL); =20 - if (debug_pagealloc_enabled_or_kfence()) { + if (debug_pagealloc_enabled()) { linear_map_hash_count =3D memblock_end_of_DRAM() >> PAGE_SHIFT; linear_map_hash_slots =3D memblock_alloc_try_nid( linear_map_hash_count, 1, MEMBLOCK_LOW_LIMIT, @@ -2120,7 +2120,7 @@ void hpt_do_stress(unsigned long ea, unsigned long hp= te_group) } } =20 -#if defined(CONFIG_DEBUG_PAGEALLOC) || defined(CONFIG_KFENCE) +#ifdef CONFIG_DEBUG_PAGEALLOC static DEFINE_RAW_SPINLOCK(linear_map_hash_lock); =20 static void kernel_map_linear_page(unsigned long vaddr, unsigned long lmi) @@ -2194,7 +2194,13 @@ int hash__kernel_map_pages(struct page *page, int nu= mpages, int enable) local_irq_restore(flags); return 0; } -#endif /* CONFIG_DEBUG_PAGEALLOC || CONFIG_KFENCE */ +#else /* CONFIG_DEBUG_PAGEALLOC */ +int hash__kernel_map_pages(struct page *page, int numpages, + int enable) +{ + return 0; +} +#endif /* CONFIG_DEBUG_PAGEALLOC */ =20 void hash__setup_initial_memory_limit(phys_addr_t first_memblock_base, phys_addr_t first_memblock_size) --=20 2.46.0 From nobody Fri Nov 29 14:46:21 2024 Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EB7E41CD2C for ; Thu, 19 Sep 2024 02:56:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726714607; cv=none; b=Vq6edVUUeDTYha9xZLAqjqpfzuxFBVBjKsX4dQ6MG6aToIFZQuwem2DnQ3HRDs8dnha2zOFp5sVvDBeC1TFR6ykN/W2Z0ruJvJ/4F0pjlvb/uZoFSOSSRYJzT8hyRVDe/Or1ptb0JIj3sbRo01SbIxbw9x+9HyLnT846Pd19YUc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726714607; c=relaxed/simple; bh=FsG1hRAQUYRA/vEnlI+7JEFrFUI5bmUaSjg6v6DcwoU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JEiy+pCTqcQtjZwy2Xfj1+1jyw0jkM94Hr+f3ymdlXQfncTTa8NYhjUePDxwxnzsBPjFCyl1q4jr4/h3YTR2qRACOCBapOJv2bn1P7kDfB/taLtJQrgJsKB/AdxE087uhnPzRsweTmN2bOiH2aR5KJJQ1CRZ6Ke0r+mQM0Sy68E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=f/E+EXOK; arc=none smtp.client-ip=209.85.214.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="f/E+EXOK" Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-20688fbaeafso4765125ad.0 for ; Wed, 18 Sep 2024 19:56:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726714605; x=1727319405; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=axWsqNzIpvlhLR7DqrnxUdvbHKQsISEkZ4mTeZAqu9w=; b=f/E+EXOKmuJvfkC26qQrLP3yn3K3DQQLvL4Zhxh/sS1zylYG5D1DSeMtmS4SEVsOMk AwoekJYliLtIVE2XVrbNRvftwRibXlwmH/HX/MQZixqg0iugMOuSvFbgjSkWbyQdultJ GTTLB89ezzy4RKTqhhjrVC0VgIk43qU4kIg7XmgANm4hk8MQxaR13/u6/WPNdL1MX2qp hW4JyKl1/G+Y2N6QLuhjOjVoTbUPUVFhzaNCbXMFeI9jrzdynkaV6zlqGyipB15fzLwZ fMO0xnaY+ob4k3p5KRW8Sv0AlSmIIOGnAKhq1lFaqxWMRtQdSYP1VJW60qUTuSmyZT3M X94A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726714605; x=1727319405; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=axWsqNzIpvlhLR7DqrnxUdvbHKQsISEkZ4mTeZAqu9w=; b=Fh77DJcN/FDDNCjHyucXprd/CiOtogXsWxcRSbnek8O3j+3hv35knTppZ7NwlL+Y1M zCRTWzC7M+Hh+IwBUMzyQv9fcI0O1Gvt3mSoyHZPgXVnSWWaYGLvgAHUIM0aK7pE+gdd pNao6uxp8USvFKkITH41p/+wXlXTPYEqo0Uq4ePYziQtGTupuHFQh+hoKOw7miCDbrPg HMdJn6K2kxL91AZh73fQVxhA7sbPOfx1M0Le034xmZfpnvw3bTlizUy3JjqeA1r+GDtB HttVu8MQg78eA+taQmt3U9/DAVCARXmhJJV1/e5Ui5pBrp22mZy9FWB0fBJf3TEcdCxZ hCFg== X-Forwarded-Encrypted: i=1; AJvYcCU9FWaiHssgMKIHjPsTeON5akjUAjFiQ/pIVxW0BJan3kpHdJaCJWAhn3WT3pftoXiyvL6O75ASAxKmcHw=@vger.kernel.org X-Gm-Message-State: AOJu0YwlRsMMDI6zlI26CW0SAVXO2Vbz3atjKBm1tO4HNaXLhMrCAWPg RCsCpjv14lbgp9r9u/Q4EmMZPcJCx89f5H5XdTEKx//xwU5mmPGx X-Google-Smtp-Source: AGHT+IHA1Q2JlW37EA2VKdDDVikQbbi1MnJIFVGu950hUG+dA+q41WmFWBJe/jOOl1HWU88lYqBIzQ== X-Received: by 2002:a17:902:d4d2:b0:206:f065:f45d with SMTP id d9443c01a7336-2078252bd1emr238945975ad.31.1726714605103; Wed, 18 Sep 2024 19:56:45 -0700 (PDT) Received: from dw-tp.. ([171.76.85.129]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-207946d2823sm71389105ad.148.2024.09.18.19.56.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Sep 2024 19:56:44 -0700 (PDT) From: "Ritesh Harjani (IBM)" To: linuxppc-dev@lists.ozlabs.org Cc: Michael Ellerman , Nicholas Piggin , Madhavan Srinivasan , Christophe Leroy , Hari Bathini , "Aneesh Kumar K . V" , Donet Tom , Pavithra Prakash , Nirjhar Roy , LKML , kasan-dev@googlegroups.com, "Ritesh Harjani (IBM)" Subject: [RFC v2 04/13] book3s64/hash: Refactor kernel linear map related calls Date: Thu, 19 Sep 2024 08:26:02 +0530 Message-ID: X-Mailer: git-send-email 2.46.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This just brings all linear map related handling at one place instead of having those functions scattered in hash_utils file. Makes it easy for review. No functionality changes in this patch. Signed-off-by: Ritesh Harjani (IBM) --- arch/powerpc/mm/book3s64/hash_utils.c | 164 +++++++++++++------------- 1 file changed, 82 insertions(+), 82 deletions(-) diff --git a/arch/powerpc/mm/book3s64/hash_utils.c b/arch/powerpc/mm/book3s= 64/hash_utils.c index 296bb74dbf40..82151fff9648 100644 --- a/arch/powerpc/mm/book3s64/hash_utils.c +++ b/arch/powerpc/mm/book3s64/hash_utils.c @@ -273,6 +273,88 @@ void hash__tlbiel_all(unsigned int action) WARN(1, "%s called on pre-POWER7 CPU\n", __func__); } =20 +#ifdef CONFIG_DEBUG_PAGEALLOC +static DEFINE_RAW_SPINLOCK(linear_map_hash_lock); + +static void kernel_map_linear_page(unsigned long vaddr, unsigned long lmi) +{ + unsigned long hash; + unsigned long vsid =3D get_kernel_vsid(vaddr, mmu_kernel_ssize); + unsigned long vpn =3D hpt_vpn(vaddr, vsid, mmu_kernel_ssize); + unsigned long mode =3D htab_convert_pte_flags(pgprot_val(PAGE_KERNEL), HP= TE_USE_KERNEL_KEY); + long ret; + + hash =3D hpt_hash(vpn, PAGE_SHIFT, mmu_kernel_ssize); + + /* Don't create HPTE entries for bad address */ + if (!vsid) + return; + + if (linear_map_hash_slots[lmi] & 0x80) + return; + + ret =3D hpte_insert_repeating(hash, vpn, __pa(vaddr), mode, + HPTE_V_BOLTED, + mmu_linear_psize, mmu_kernel_ssize); + + BUG_ON (ret < 0); + raw_spin_lock(&linear_map_hash_lock); + BUG_ON(linear_map_hash_slots[lmi] & 0x80); + linear_map_hash_slots[lmi] =3D ret | 0x80; + raw_spin_unlock(&linear_map_hash_lock); +} + +static void kernel_unmap_linear_page(unsigned long vaddr, unsigned long lm= i) +{ + unsigned long hash, hidx, slot; + unsigned long vsid =3D get_kernel_vsid(vaddr, mmu_kernel_ssize); + unsigned long vpn =3D hpt_vpn(vaddr, vsid, mmu_kernel_ssize); + + hash =3D hpt_hash(vpn, PAGE_SHIFT, mmu_kernel_ssize); + raw_spin_lock(&linear_map_hash_lock); + if (!(linear_map_hash_slots[lmi] & 0x80)) { + raw_spin_unlock(&linear_map_hash_lock); + return; + } + hidx =3D linear_map_hash_slots[lmi] & 0x7f; + linear_map_hash_slots[lmi] =3D 0; + raw_spin_unlock(&linear_map_hash_lock); + if (hidx & _PTEIDX_SECONDARY) + hash =3D ~hash; + slot =3D (hash & htab_hash_mask) * HPTES_PER_GROUP; + slot +=3D hidx & _PTEIDX_GROUP_IX; + mmu_hash_ops.hpte_invalidate(slot, vpn, mmu_linear_psize, + mmu_linear_psize, + mmu_kernel_ssize, 0); +} + +int hash__kernel_map_pages(struct page *page, int numpages, int enable) +{ + unsigned long flags, vaddr, lmi; + int i; + + local_irq_save(flags); + for (i =3D 0; i < numpages; i++, page++) { + vaddr =3D (unsigned long)page_address(page); + lmi =3D __pa(vaddr) >> PAGE_SHIFT; + if (lmi >=3D linear_map_hash_count) + continue; + if (enable) + kernel_map_linear_page(vaddr, lmi); + else + kernel_unmap_linear_page(vaddr, lmi); + } + local_irq_restore(flags); + return 0; +} +#else /* CONFIG_DEBUG_PAGEALLOC */ +int hash__kernel_map_pages(struct page *page, int numpages, + int enable) +{ + return 0; +} +#endif /* CONFIG_DEBUG_PAGEALLOC */ + /* * 'R' and 'C' update notes: * - Under pHyp or KVM, the updatepp path will not set C, thus it *will* @@ -2120,88 +2202,6 @@ void hpt_do_stress(unsigned long ea, unsigned long h= pte_group) } } =20 -#ifdef CONFIG_DEBUG_PAGEALLOC -static DEFINE_RAW_SPINLOCK(linear_map_hash_lock); - -static void kernel_map_linear_page(unsigned long vaddr, unsigned long lmi) -{ - unsigned long hash; - unsigned long vsid =3D get_kernel_vsid(vaddr, mmu_kernel_ssize); - unsigned long vpn =3D hpt_vpn(vaddr, vsid, mmu_kernel_ssize); - unsigned long mode =3D htab_convert_pte_flags(pgprot_val(PAGE_KERNEL), HP= TE_USE_KERNEL_KEY); - long ret; - - hash =3D hpt_hash(vpn, PAGE_SHIFT, mmu_kernel_ssize); - - /* Don't create HPTE entries for bad address */ - if (!vsid) - return; - - if (linear_map_hash_slots[lmi] & 0x80) - return; - - ret =3D hpte_insert_repeating(hash, vpn, __pa(vaddr), mode, - HPTE_V_BOLTED, - mmu_linear_psize, mmu_kernel_ssize); - - BUG_ON (ret < 0); - raw_spin_lock(&linear_map_hash_lock); - BUG_ON(linear_map_hash_slots[lmi] & 0x80); - linear_map_hash_slots[lmi] =3D ret | 0x80; - raw_spin_unlock(&linear_map_hash_lock); -} - -static void kernel_unmap_linear_page(unsigned long vaddr, unsigned long lm= i) -{ - unsigned long hash, hidx, slot; - unsigned long vsid =3D get_kernel_vsid(vaddr, mmu_kernel_ssize); - unsigned long vpn =3D hpt_vpn(vaddr, vsid, mmu_kernel_ssize); - - hash =3D hpt_hash(vpn, PAGE_SHIFT, mmu_kernel_ssize); - raw_spin_lock(&linear_map_hash_lock); - if (!(linear_map_hash_slots[lmi] & 0x80)) { - raw_spin_unlock(&linear_map_hash_lock); - return; - } - hidx =3D linear_map_hash_slots[lmi] & 0x7f; - linear_map_hash_slots[lmi] =3D 0; - raw_spin_unlock(&linear_map_hash_lock); - if (hidx & _PTEIDX_SECONDARY) - hash =3D ~hash; - slot =3D (hash & htab_hash_mask) * HPTES_PER_GROUP; - slot +=3D hidx & _PTEIDX_GROUP_IX; - mmu_hash_ops.hpte_invalidate(slot, vpn, mmu_linear_psize, - mmu_linear_psize, - mmu_kernel_ssize, 0); -} - -int hash__kernel_map_pages(struct page *page, int numpages, int enable) -{ - unsigned long flags, vaddr, lmi; - int i; - - local_irq_save(flags); - for (i =3D 0; i < numpages; i++, page++) { - vaddr =3D (unsigned long)page_address(page); - lmi =3D __pa(vaddr) >> PAGE_SHIFT; - if (lmi >=3D linear_map_hash_count) - continue; - if (enable) - kernel_map_linear_page(vaddr, lmi); - else - kernel_unmap_linear_page(vaddr, lmi); - } - local_irq_restore(flags); - return 0; -} -#else /* CONFIG_DEBUG_PAGEALLOC */ -int hash__kernel_map_pages(struct page *page, int numpages, - int enable) -{ - return 0; -} -#endif /* CONFIG_DEBUG_PAGEALLOC */ - void hash__setup_initial_memory_limit(phys_addr_t first_memblock_base, phys_addr_t first_memblock_size) { --=20 2.46.0 From nobody Fri Nov 29 14:46:21 2024 Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C932E1F942 for ; Thu, 19 Sep 2024 02:56:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726714612; cv=none; b=kzEwBq5OfC4G0Nj826Hjhia51iU7c1fsKjfvcD0M+P6BID+RdN+ZGUwZKN3yawH7zixniikLy6IBcRlCGubVZjwJF1wK5mar9A5Kf56yMI0HGdT3EFXHipslEvQsrFdvQmLV88Rcpr7gd0/+qOkEgpEy9LMTHW6LbnKNkqq6yrU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726714612; c=relaxed/simple; bh=lGDU6+XsEZA63vePePBnGkCzitISc6raOI8xarwqjFo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=VpD5WUJaj6sckhWXroESKmZ+aAsRYwPFTXUXF5TV19gWW+1+0n/f3CqW/LCQk0iYTzmqUb4EpsTuR/2kI9MssntPe2ws3UOFbIWGITKzcbyuWIwa/YHwmdZKPD0afbxkH0ufMRP6l6UegUOQWxYDNC42YGiqNAz8WyGQ1wKv0FU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=A90dXAbS; arc=none smtp.client-ip=209.85.214.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="A90dXAbS" Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-20543fdb7acso3502405ad.1 for ; Wed, 18 Sep 2024 19:56:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726714610; x=1727319410; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=U+q0qzhvgy5Htf+jNoj0jpwh62QEw37SjQucvD3DcJc=; b=A90dXAbSzjxDkMBL/p08VdbHN2dcuXq0SxCoCJS38J//j0QZ2oY3M97PEv2jNfSL5d 02p2cajm94lcs6laej2avDxrQjXnB+5k3b4tkhK0c7FWTjcJHElLphk/RUGk2qNLYbCV bLSoqeCFhjN5ti8ITakB3x/+U56K8QDulSW8cWZQuOD8zHmBFCfNdcNyTggMOshNEm4n OSxYZqebyRERUBPJzzvJ4at/kfVDU4KDlimit2+yiu0jGUfaCmbR6BxX0mQw3DINiqw8 nalt0vyDpUYISMOQxBMmfA8WZLPtefod1gEZ2E/JpeIYxi1H9H0RtkY5ZLYYuunjm6qn Efrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726714610; x=1727319410; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=U+q0qzhvgy5Htf+jNoj0jpwh62QEw37SjQucvD3DcJc=; b=PoTr2vW73EGrQWMzwnVr6RcOBNwmu9NOOijPRN7cYLnRAuFDEMakUA0H+57yfIJwDR 95mHRhifE8ATAvj2T6psn4wTDxOyJseU5Y99QrQJAr01cCf/v3uOOvqzTQorfXUyjoTe aHaTVHDdrKk9fHzjnaUVEGoT8bQ9rRtQ666n46wXi/d79Y2v0836s9zC/ilDXatTyXvb 3opCu/PG/SI59LRg7FaxeUfDFz/afEc9eWjTehUdCEPeeXhjCo8b221P4O9KK24c9gkk BGtEsqzjm28izZx6uINDmGFtoVCdzKsAZXU1e9uX/FSu9SzgXa7ino+YiujhczL7A9le h+wg== X-Forwarded-Encrypted: i=1; AJvYcCVUkOIqCb355XxtHobYbz9HMYWGq4JP2rWPrKisIEV5EpXPWPV+36z98Ghp09cmJPThJOjrz/+iW0WHwOY=@vger.kernel.org X-Gm-Message-State: AOJu0YyGOUWlDvcspSzMoAoJIIcbtiv8sCTqVZNaxOG32kFyD7436xX3 t0zXvvdzkm44ccECrvNQpywUEOe7tKxW3iWALG1qVjjRqTVJ5Lnp X-Google-Smtp-Source: AGHT+IHDCEhyE6aDKQMLRWn1dVpCIu8ywaRfsfF6B1LtKOZnRl3x2/5HVbN1RjELYtzDCtK/03wddQ== X-Received: by 2002:a17:903:2452:b0:200:869c:95e3 with SMTP id d9443c01a7336-20781b42cfcmr332110025ad.4.1726714610017; Wed, 18 Sep 2024 19:56:50 -0700 (PDT) Received: from dw-tp.. ([171.76.85.129]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-207946d2823sm71389105ad.148.2024.09.18.19.56.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Sep 2024 19:56:49 -0700 (PDT) From: "Ritesh Harjani (IBM)" To: linuxppc-dev@lists.ozlabs.org Cc: Michael Ellerman , Nicholas Piggin , Madhavan Srinivasan , Christophe Leroy , Hari Bathini , "Aneesh Kumar K . V" , Donet Tom , Pavithra Prakash , Nirjhar Roy , LKML , kasan-dev@googlegroups.com, "Ritesh Harjani (IBM)" Subject: [RFC v2 05/13] book3s64/hash: Add hash_debug_pagealloc_add_slot() function Date: Thu, 19 Sep 2024 08:26:03 +0530 Message-ID: X-Mailer: git-send-email 2.46.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This adds hash_debug_pagealloc_add_slot() function instead of open coding that in htab_bolt_mapping(). This is required since we will be separating kfence functionality to not depend upon debug_pagealloc. No functionality change in this patch. Signed-off-by: Ritesh Harjani (IBM) --- arch/powerpc/mm/book3s64/hash_utils.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/arch/powerpc/mm/book3s64/hash_utils.c b/arch/powerpc/mm/book3s= 64/hash_utils.c index 82151fff9648..6e3860224351 100644 --- a/arch/powerpc/mm/book3s64/hash_utils.c +++ b/arch/powerpc/mm/book3s64/hash_utils.c @@ -328,6 +328,14 @@ static void kernel_unmap_linear_page(unsigned long vad= dr, unsigned long lmi) mmu_kernel_ssize, 0); } =20 +static inline void hash_debug_pagealloc_add_slot(phys_addr_t paddr, int sl= ot) +{ + if (!debug_pagealloc_enabled()) + return; + if ((paddr >> PAGE_SHIFT) < linear_map_hash_count) + linear_map_hash_slots[paddr >> PAGE_SHIFT] =3D slot | 0x80; +} + int hash__kernel_map_pages(struct page *page, int numpages, int enable) { unsigned long flags, vaddr, lmi; @@ -353,6 +361,7 @@ int hash__kernel_map_pages(struct page *page, int numpa= ges, { return 0; } +static inline void hash_debug_pagealloc_add_slot(phys_addr_t paddr, int sl= ot) {} #endif /* CONFIG_DEBUG_PAGEALLOC */ =20 /* @@ -513,9 +522,7 @@ int htab_bolt_mapping(unsigned long vstart, unsigned lo= ng vend, break; =20 cond_resched(); - if (debug_pagealloc_enabled() && - (paddr >> PAGE_SHIFT) < linear_map_hash_count) - linear_map_hash_slots[paddr >> PAGE_SHIFT] =3D ret | 0x80; + hash_debug_pagealloc_add_slot(paddr, ret); } return ret < 0 ? ret : 0; } --=20 2.46.0 From nobody Fri Nov 29 14:46:21 2024 Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 789A8210FF for ; Thu, 19 Sep 2024 02:56:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726714616; cv=none; b=Gs4+xFAgcyYqTHmzUt+cpcUoJkqzUlHYV74IwR0+THnFMCcjcAh+ATxA6yKo6h14FO9CUyhrLecv1dT25Vr75D4jdKzjrYdaCL23Zqpy5XnUaE2FJrCLhgxodZXKbVFFmurq+oUECiMmgpbd6SEdmraNP7xAI46BNJNr4LCBRjI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726714616; c=relaxed/simple; bh=ll46Y5ohND+TUmqRmSlDtnD60Fmr6r5zMc07Tyy0M5k=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=LUWgzejn6qJth8Op5ntpUoMQk9hvIM8L3A31FG8PtfUQh2mRfU4B1YGQ4Td6nwDTOC+OemStbiL8jlk79nuJgdZfquQbnv/yMGgFXsRDWh8M+wqYwN8wJOmhSm7iu0I+nYe/SA1fmCvvRtUyUnnt85myrI8q1z0ZyVtx1lYR/zg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=KQgwLw4R; arc=none smtp.client-ip=209.85.214.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="KQgwLw4R" Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-2068bee21d8so4131005ad.2 for ; Wed, 18 Sep 2024 19:56:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726714615; x=1727319415; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/MHsxmP9bN1bxx4msR28qZ2la9zK0wSY8EDrce3K+zQ=; b=KQgwLw4Rx2irx/shbP0Ng1/6rkR2AUHbGx2qGWjqqci5ACLwb7LsXQJapqHR+Ki6dV P3iZEbfPc6dCDe7KaIb0HICbVxExA3jEb5qlXT2GQWeQW2vUeLjxSjvHVwl/HfTVpz5E dsXLv4qguBpYcaAlnLCzxAbuTppjmEUzAwRpQbOvXrI24WtLsjBe6IhzGjSQN2+6L+NI Xlriw0JVecBrrv55+H9UYxEMLsM3Rh5nrJcRcTTRZ1VKIXfrZdAsJRPM2pCKqdU4e7wS GiNwbFBZhqOGcZwDJuWBI6JqrVoIDsTUPTZ3qtTC84yI/cK3rKrIbjc7j8VztKAASUOh vu7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726714615; x=1727319415; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/MHsxmP9bN1bxx4msR28qZ2la9zK0wSY8EDrce3K+zQ=; b=OfLItC9SkChnXTpid1ZmwSxXNkd1Zd8Agy8Z8iByrxSwF0+Rm9d8Bl7DXsuDdbc08E 3OPhl+Anx4isHHiE2J/cAecF5+oN0tQYvfK0Fc/q+b5ypl0G7EaVu/6LzGMZir6kJyZn ZpvTBHJNHYhi54C/5l5qePSsJ2MpHh5shk8rzgZGQLNBQfPNO8yKNchsE1Pnzrf8+7VT PenrBlHiPINi+E0v2NxA6Wa6EWkfjcriGpnRhQeiC5wUQ+l6vxzpxTbEnHYbA25k5Hwm /ksdlzPz184TyzpxZpxo021uADV7al5oK6mY+CxWh9mhd+Te3dwF3lL4YW5qUbLUYSHK EEHA== X-Forwarded-Encrypted: i=1; AJvYcCWFRGuWCRbm80y5PE5nE4iKHmDA2CNRgMaWS8clb+hvzewpP6KP+PdOpRH3TL4CfRNn0MRm+gx4hFIuFS0=@vger.kernel.org X-Gm-Message-State: AOJu0Yx4V5SJByG9UY+TBdx0PoMIJmvRXuZpwTc9qOC1i4VYJrPF7AwH zCrZPHwxVcRRXU6rys/V/8zmIevxMYEqHd3UT55qKrwMo5vXE6LH X-Google-Smtp-Source: AGHT+IEGxeiVxZh5o9vYhwB8mBVBHLv/pSg64zA2+3t1GDmylunq7BBEStCgtPZymo2nVWUT4NL8Bw== X-Received: by 2002:a17:902:f70c:b0:206:8acc:8871 with SMTP id d9443c01a7336-2076e39c331mr311700805ad.31.1726714614667; Wed, 18 Sep 2024 19:56:54 -0700 (PDT) Received: from dw-tp.. ([171.76.85.129]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-207946d2823sm71389105ad.148.2024.09.18.19.56.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Sep 2024 19:56:54 -0700 (PDT) From: "Ritesh Harjani (IBM)" To: linuxppc-dev@lists.ozlabs.org Cc: Michael Ellerman , Nicholas Piggin , Madhavan Srinivasan , Christophe Leroy , Hari Bathini , "Aneesh Kumar K . V" , Donet Tom , Pavithra Prakash , Nirjhar Roy , LKML , kasan-dev@googlegroups.com, "Ritesh Harjani (IBM)" Subject: [RFC v2 06/13] book3s64/hash: Add hash_debug_pagealloc_alloc_slots() function Date: Thu, 19 Sep 2024 08:26:04 +0530 Message-ID: <47af6bef68ce0a82da4694174f004d11519e8757.1726571179.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This adds hash_debug_pagealloc_alloc_slots() function instead of open coding that in htab_initialize(). This is required since we will be separating the kfence functionality to not depend upon debug_pagealloc. Now that everything required for debug_pagealloc is under a #ifdef config. Bring in linear_map_hash_slots and linear_map_hash_count variables under the same config too. Signed-off-by: Ritesh Harjani (IBM) --- arch/powerpc/mm/book3s64/hash_utils.c | 29 ++++++++++++++++----------- 1 file changed, 17 insertions(+), 12 deletions(-) diff --git a/arch/powerpc/mm/book3s64/hash_utils.c b/arch/powerpc/mm/book3s= 64/hash_utils.c index 6e3860224351..030c120d1399 100644 --- a/arch/powerpc/mm/book3s64/hash_utils.c +++ b/arch/powerpc/mm/book3s64/hash_utils.c @@ -123,8 +123,6 @@ EXPORT_SYMBOL_GPL(mmu_slb_size); #ifdef CONFIG_PPC_64K_PAGES int mmu_ci_restrictions; #endif -static u8 *linear_map_hash_slots; -static unsigned long linear_map_hash_count; struct mmu_hash_ops mmu_hash_ops __ro_after_init; EXPORT_SYMBOL(mmu_hash_ops); =20 @@ -274,6 +272,8 @@ void hash__tlbiel_all(unsigned int action) } =20 #ifdef CONFIG_DEBUG_PAGEALLOC +static u8 *linear_map_hash_slots; +static unsigned long linear_map_hash_count; static DEFINE_RAW_SPINLOCK(linear_map_hash_lock); =20 static void kernel_map_linear_page(unsigned long vaddr, unsigned long lmi) @@ -328,6 +328,19 @@ static void kernel_unmap_linear_page(unsigned long vad= dr, unsigned long lmi) mmu_kernel_ssize, 0); } =20 +static inline void hash_debug_pagealloc_alloc_slots(void) +{ + if (!debug_pagealloc_enabled()) + return; + linear_map_hash_count =3D memblock_end_of_DRAM() >> PAGE_SHIFT; + linear_map_hash_slots =3D memblock_alloc_try_nid( + linear_map_hash_count, 1, MEMBLOCK_LOW_LIMIT, + ppc64_rma_size, NUMA_NO_NODE); + if (!linear_map_hash_slots) + panic("%s: Failed to allocate %lu bytes max_addr=3D%pa\n", + __func__, linear_map_hash_count, &ppc64_rma_size); +} + static inline void hash_debug_pagealloc_add_slot(phys_addr_t paddr, int sl= ot) { if (!debug_pagealloc_enabled()) @@ -361,6 +374,7 @@ int hash__kernel_map_pages(struct page *page, int numpa= ges, { return 0; } +static inline void hash_debug_pagealloc_alloc_slots(void) {} static inline void hash_debug_pagealloc_add_slot(phys_addr_t paddr, int sl= ot) {} #endif /* CONFIG_DEBUG_PAGEALLOC */ =20 @@ -1223,16 +1237,7 @@ static void __init htab_initialize(void) =20 prot =3D pgprot_val(PAGE_KERNEL); =20 - if (debug_pagealloc_enabled()) { - linear_map_hash_count =3D memblock_end_of_DRAM() >> PAGE_SHIFT; - linear_map_hash_slots =3D memblock_alloc_try_nid( - linear_map_hash_count, 1, MEMBLOCK_LOW_LIMIT, - ppc64_rma_size, NUMA_NO_NODE); - if (!linear_map_hash_slots) - panic("%s: Failed to allocate %lu bytes max_addr=3D%pa\n", - __func__, linear_map_hash_count, &ppc64_rma_size); - } - + hash_debug_pagealloc_alloc_slots(); /* create bolted the linear mapping in the hash table */ for_each_mem_range(i, &base, &end) { size =3D end - base; --=20 2.46.0 From nobody Fri Nov 29 14:46:21 2024 Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5F0D9225A8 for ; Thu, 19 Sep 2024 02:57:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726714621; cv=none; b=ZYnNsi4UcipVlFDmNbPcd+ZGFh18F+Nv61cYpK7oWoE1x/ue05giLYvqYdy1VKPRCqlOSUykNUKA0tn27QQvkpLVX+6+T8xBH9NwZsljTwPJVnN4KYSh0vUfIDoKnNSCzgSn9v6xKyhPTJwQlinGu0oTdpchk5Kfdj6XXJI1VNA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726714621; c=relaxed/simple; bh=v1P4WAxWuvUCZPOP2tMFqMbCJcwChW39Ab67XAl34ag=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Q+4LadwEacGzeFSd2PHjeOR9jnI5vDxvP1BLWKNZvjfnxFgXj9qV//ChsZMKvZyaLfIPwkelB6JH+mhFvI359KlIlqgvQjTXeu2tvt/h8xWwGfKQQkBzIrniR//cxs3qP+l/6DH3RMJ2XOGXDdMVwL+iMDcWFxyDC7KeMSmlsmw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Qh6HqwvB; arc=none smtp.client-ip=209.85.214.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Qh6HqwvB" Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-2053a0bd0a6so4300265ad.3 for ; Wed, 18 Sep 2024 19:57:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726714619; x=1727319419; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=oTnklaF1aF0bTWeWldi91Sj9bwNgHQZw6gVAVQdUYd0=; b=Qh6HqwvB7nppCsdGIvvXgSX3WaSOwATmHkNKdEkcYviCyNS5J8/0v09IS5HH1Xin86 4Y16U7Ad3SXwbX05OWPKdpEKduLsFksBdD0j0fMuRICtVfr9r37tydUpSydH6JnB8JR7 Tf8tCHCBOyOTBKnR6VYa3UHIPA9xXnSg9lGu0/38tc5x5mb4BxOhcE576yXx0wUr1gY8 ly9BcLUb/GTrwkASVAvRgGKE9g6lUltcKDuWFI9u+fJJwJ1hRRa9LaI0w4h/2M1CsRRg a+ZnIy3wuxSuYmXCABz/ztm1ikdHS5X/wDE9BEMp3P/Mx8lOVF4s9JHF+kH8vP4faSTv EGOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726714620; x=1727319420; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oTnklaF1aF0bTWeWldi91Sj9bwNgHQZw6gVAVQdUYd0=; b=fFBJYiOi3ke6J65eGShn+i4ujJcka7lgQhTslGIMGFpALYvKFBHM0tWru9kVyTh4Me FQfD9fi7EMBLmxFUKTBA4QJr2rSK0ZAiXDWnBIXIEvynJ0kpVNl8FpTn4jM/KbH3o5vw 5IHhQSdec4WLvmxiKaJ9XiOF++ZLxZ5diiYzpIMKtmps6z0hceD8BYfNPBGz8DDKVSVW MNw6tix4unQn9u+1omwanGdEl3o3plmbZRyU0vF9jVuF24GdDq6bNxw6d+FFVAMIsC6C oOLFJyWL1aTjIUJ5dk9lsZnPxcb+/kIBqiKiw+CDRGSWPtQXOXH6wJskIWov5bRnb7wc aQjA== X-Forwarded-Encrypted: i=1; AJvYcCUeMNcpfM9aw0flWeMYdcXTjYhl95tNNJlSTsGfs+kyyUFbvXe2x6pESk3dQnpqzsKx0WZEfkWMV462m0M=@vger.kernel.org X-Gm-Message-State: AOJu0YwWjMyf6vJTYdTuCI0raqjb7wqkeCvE/wyo7/lvxxPM3akiZrIQ N4leBD+A+b6VPoCfzhRBNKOmo8z9inLqXNHfB2hsbkZuyA6ckysG X-Google-Smtp-Source: AGHT+IFXJApA5WgaqQlHFHHFqKDy2nOzFXTEZu7K9vwx9KzKHpaYLGI4OiWl2mQQHdfyZu/PNzppfQ== X-Received: by 2002:a17:903:2445:b0:205:7574:3b79 with SMTP id d9443c01a7336-2076e3622b7mr412645275ad.25.1726714619622; Wed, 18 Sep 2024 19:56:59 -0700 (PDT) Received: from dw-tp.. ([171.76.85.129]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-207946d2823sm71389105ad.148.2024.09.18.19.56.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Sep 2024 19:56:59 -0700 (PDT) From: "Ritesh Harjani (IBM)" To: linuxppc-dev@lists.ozlabs.org Cc: Michael Ellerman , Nicholas Piggin , Madhavan Srinivasan , Christophe Leroy , Hari Bathini , "Aneesh Kumar K . V" , Donet Tom , Pavithra Prakash , Nirjhar Roy , LKML , kasan-dev@googlegroups.com, "Ritesh Harjani (IBM)" Subject: [RFC v2 07/13] book3s64/hash: Refactor hash__kernel_map_pages() function Date: Thu, 19 Sep 2024 08:26:05 +0530 Message-ID: <0ced93c215459479ae70dd9bbe00daa595f9aff0.1726571179.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This refactors hash__kernel_map_pages() function to call hash_debug_pagealloc_map_pages(). This will come useful when we will add kfence support. No functionality changes in this patch. Signed-off-by: Ritesh Harjani (IBM) --- arch/powerpc/mm/book3s64/hash_utils.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/arch/powerpc/mm/book3s64/hash_utils.c b/arch/powerpc/mm/book3s= 64/hash_utils.c index 030c120d1399..da9b089c8e8b 100644 --- a/arch/powerpc/mm/book3s64/hash_utils.c +++ b/arch/powerpc/mm/book3s64/hash_utils.c @@ -349,7 +349,8 @@ static inline void hash_debug_pagealloc_add_slot(phys_a= ddr_t paddr, int slot) linear_map_hash_slots[paddr >> PAGE_SHIFT] =3D slot | 0x80; } =20 -int hash__kernel_map_pages(struct page *page, int numpages, int enable) +static int hash_debug_pagealloc_map_pages(struct page *page, int numpages, + int enable) { unsigned long flags, vaddr, lmi; int i; @@ -368,6 +369,12 @@ int hash__kernel_map_pages(struct page *page, int nump= ages, int enable) local_irq_restore(flags); return 0; } + +int hash__kernel_map_pages(struct page *page, int numpages, int enable) +{ + return hash_debug_pagealloc_map_pages(page, numpages, enable); +} + #else /* CONFIG_DEBUG_PAGEALLOC */ int hash__kernel_map_pages(struct page *page, int numpages, int enable) --=20 2.46.0 From nobody Fri Nov 29 14:46:21 2024 Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 67F6D2746C for ; Thu, 19 Sep 2024 02:57:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726714626; cv=none; b=nP6gxWyN7lSM9Hkq8KFiE1MtAA889+TodljFkbyYrt2bCn1MZJSapL7R/3i0IFYta81lPjJoU0PGtobGWx3NzBK91y1WYRyUG15Ld6Yb1NkhM583OFLShf7+eFkMdvLPBHsmBvglYN3WhFxUlf1qetm3Do3H/t8KtLyXdJSCsCg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726714626; c=relaxed/simple; bh=R968dba8OE7cCpOttgpeILQbdzMZ+6PWqhvC1JS2Eaw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=N/ucsEzbdUS1ZK8gcdP7Q8BfAEvppYWwuqyNNLzIvBRZnYQHnwjrzbXaXwRzrjW7nqEtjVcNcWSIsLzylvEVujWqDKQZTsR3s6PDr1yIaer64xVb7YsCbzhvfISOryrbcq8GMdJsjOvTFHnifELwYetjrqxWSY/5PI4SiGoCDQ8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=br5qtc9g; arc=none smtp.client-ip=209.85.214.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="br5qtc9g" Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-2057835395aso4875385ad.3 for ; Wed, 18 Sep 2024 19:57:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726714624; x=1727319424; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ww6ZbzhcNbPuRWG0GzwKnAEreEL780mKVa0cmRg0+f0=; b=br5qtc9glKdvd1LpRpBgryge7vQoBf4X54SmlekMMj1jWwdP2NyKxVSkc2BZ3If6kv Z8X0No8MApOmmbt9cQu1yvk8WBrDNqSLbaTIMAL/Oe8NhDXCzte04J3NRM2uoNOnogZe vHucmb7x0pNisU6i3uQCPtDDU3wLaeZAxcq/hnkB4XlAg+7cXdspkEE683Khl7vMunVp CQQEA7ObuXyMBNYFtjZwCsdDsHvBCAoLxSHIsAK3AbLK7mIpqGWOt9Na0OFXsIbLZ+nc 7NXwMtGQx9tzKA6VTOQ1v65wemjUAT5WpG/X9e+bLWXnTana/gKGwQHCJF5iY/Zc3tyG 1qOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726714625; x=1727319425; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ww6ZbzhcNbPuRWG0GzwKnAEreEL780mKVa0cmRg0+f0=; b=OKYIMv3aX3uSchnbFVblhpXpyLdZ75kkjiZdhKrgw3NughqI0e9FQxngcztXPMLebq MHjXjgrt+YVAOW5R/6xJN0QpFNlwvMA5j1wbbkt3QaXLdLVspzkKw9RmPFdwnF/7R584 atojdpmppGvZmkTwvEnXnbTPAE/YW0j0S8NyPGx4Av1woVYJlM1kMiUVAaz2pQhLIOCJ z4AnqbPulvcyGV91lWIiuuGIOvJNPDhJjYSn0RINrAZMMRKTMgsbaikiZoHPuxEes9IR Xhu/wU9eAzSsog73yJUQSITngXY3aGiKnTlqUbCYujZdMIUawfZBYN0K1Gh6IhBA3k8E cH4Q== X-Forwarded-Encrypted: i=1; AJvYcCXqmycqKt7Jl/lpi0bUJMpDst8lhA63uxfr7QES4qKLb30p6dtMj9Qbh7CANRNfHSHb2Gv89HS7Gmy7mlY=@vger.kernel.org X-Gm-Message-State: AOJu0YxskdjmHC98t9wUqctdGtq731f9cOwAOflrMVy3ndiEhPzpGdPA Hm01N7QIyau2URlHd9QBuP2Kb+FF1xLjmKdkS4w3++BGoeUuiG2a X-Google-Smtp-Source: AGHT+IEGqIOz4HsQttBODX4offSohXpn6nX19SV5HvK4k8qFlVoDJsalsJ6Y9uSYNEbB3JibzB2xxg== X-Received: by 2002:a17:902:f689:b0:206:96bf:b0d6 with SMTP id d9443c01a7336-2076e41703amr412516255ad.51.1726714624534; Wed, 18 Sep 2024 19:57:04 -0700 (PDT) Received: from dw-tp.. ([171.76.85.129]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-207946d2823sm71389105ad.148.2024.09.18.19.56.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Sep 2024 19:57:03 -0700 (PDT) From: "Ritesh Harjani (IBM)" To: linuxppc-dev@lists.ozlabs.org Cc: Michael Ellerman , Nicholas Piggin , Madhavan Srinivasan , Christophe Leroy , Hari Bathini , "Aneesh Kumar K . V" , Donet Tom , Pavithra Prakash , Nirjhar Roy , LKML , kasan-dev@googlegroups.com, "Ritesh Harjani (IBM)" Subject: [RFC v2 08/13] book3s64/hash: Make kernel_map_linear_page() generic Date: Thu, 19 Sep 2024 08:26:06 +0530 Message-ID: <8d06f263a903b5867fb23c319c4ddf7db7b7a431.1726571179.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Currently kernel_map_linear_page() function assumes to be working on linear_map_hash_slots array. But since in later patches we need a separate linear map array for kfence, hence make kernel_map_linear_page() take a linear map array and lock in it's function argument. This is needed to separate out kfence from debug_pagealloc infrastructure. Signed-off-by: Ritesh Harjani (IBM) --- arch/powerpc/mm/book3s64/hash_utils.c | 47 ++++++++++++++------------- 1 file changed, 25 insertions(+), 22 deletions(-) diff --git a/arch/powerpc/mm/book3s64/hash_utils.c b/arch/powerpc/mm/book3s= 64/hash_utils.c index da9b089c8e8b..cc2eaa97982c 100644 --- a/arch/powerpc/mm/book3s64/hash_utils.c +++ b/arch/powerpc/mm/book3s64/hash_utils.c @@ -272,11 +272,8 @@ void hash__tlbiel_all(unsigned int action) } =20 #ifdef CONFIG_DEBUG_PAGEALLOC -static u8 *linear_map_hash_slots; -static unsigned long linear_map_hash_count; -static DEFINE_RAW_SPINLOCK(linear_map_hash_lock); - -static void kernel_map_linear_page(unsigned long vaddr, unsigned long lmi) +static void kernel_map_linear_page(unsigned long vaddr, unsigned long idx, + u8 *slots, raw_spinlock_t *lock) { unsigned long hash; unsigned long vsid =3D get_kernel_vsid(vaddr, mmu_kernel_ssize); @@ -290,7 +287,7 @@ static void kernel_map_linear_page(unsigned long vaddr,= unsigned long lmi) if (!vsid) return; =20 - if (linear_map_hash_slots[lmi] & 0x80) + if (slots[idx] & 0x80) return; =20 ret =3D hpte_insert_repeating(hash, vpn, __pa(vaddr), mode, @@ -298,36 +295,40 @@ static void kernel_map_linear_page(unsigned long vadd= r, unsigned long lmi) mmu_linear_psize, mmu_kernel_ssize); =20 BUG_ON (ret < 0); - raw_spin_lock(&linear_map_hash_lock); - BUG_ON(linear_map_hash_slots[lmi] & 0x80); - linear_map_hash_slots[lmi] =3D ret | 0x80; - raw_spin_unlock(&linear_map_hash_lock); + raw_spin_lock(lock); + BUG_ON(slots[idx] & 0x80); + slots[idx] =3D ret | 0x80; + raw_spin_unlock(lock); } =20 -static void kernel_unmap_linear_page(unsigned long vaddr, unsigned long lm= i) +static void kernel_unmap_linear_page(unsigned long vaddr, unsigned long id= x, + u8 *slots, raw_spinlock_t *lock) { - unsigned long hash, hidx, slot; + unsigned long hash, hslot, slot; unsigned long vsid =3D get_kernel_vsid(vaddr, mmu_kernel_ssize); unsigned long vpn =3D hpt_vpn(vaddr, vsid, mmu_kernel_ssize); =20 hash =3D hpt_hash(vpn, PAGE_SHIFT, mmu_kernel_ssize); - raw_spin_lock(&linear_map_hash_lock); - if (!(linear_map_hash_slots[lmi] & 0x80)) { - raw_spin_unlock(&linear_map_hash_lock); + raw_spin_lock(lock); + if (!(slots[idx] & 0x80)) { + raw_spin_unlock(lock); return; } - hidx =3D linear_map_hash_slots[lmi] & 0x7f; - linear_map_hash_slots[lmi] =3D 0; - raw_spin_unlock(&linear_map_hash_lock); - if (hidx & _PTEIDX_SECONDARY) + hslot =3D slots[idx] & 0x7f; + slots[idx] =3D 0; + raw_spin_unlock(lock); + if (hslot & _PTEIDX_SECONDARY) hash =3D ~hash; slot =3D (hash & htab_hash_mask) * HPTES_PER_GROUP; - slot +=3D hidx & _PTEIDX_GROUP_IX; + slot +=3D hslot & _PTEIDX_GROUP_IX; mmu_hash_ops.hpte_invalidate(slot, vpn, mmu_linear_psize, mmu_linear_psize, mmu_kernel_ssize, 0); } =20 +static u8 *linear_map_hash_slots; +static unsigned long linear_map_hash_count; +static DEFINE_RAW_SPINLOCK(linear_map_hash_lock); static inline void hash_debug_pagealloc_alloc_slots(void) { if (!debug_pagealloc_enabled()) @@ -362,9 +363,11 @@ static int hash_debug_pagealloc_map_pages(struct page = *page, int numpages, if (lmi >=3D linear_map_hash_count) continue; if (enable) - kernel_map_linear_page(vaddr, lmi); + kernel_map_linear_page(vaddr, lmi, + linear_map_hash_slots, &linear_map_hash_lock); else - kernel_unmap_linear_page(vaddr, lmi); + kernel_unmap_linear_page(vaddr, lmi, + linear_map_hash_slots, &linear_map_hash_lock); } local_irq_restore(flags); return 0; --=20 2.46.0 From nobody Fri Nov 29 14:46:21 2024 Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 38FFB2AF0E for ; Thu, 19 Sep 2024 02:57:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726714631; cv=none; b=LJfOALcjPaXLRTAJmpaMSC5WMJY5GRkHDrs+OtSQXmKGlhHipgUL1ix6HuckkaarI2qGrRdLC0HqHagTD55ruIPdbT7RpcnKlCHFziRI3YHRkhRAAYQDxvljw800rAp2+qtVKNACsp/ayqbxV5AVgqSCH563pZPFb89nVDdZqvc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726714631; c=relaxed/simple; bh=iNTLscZVjBDcYCPDjAxQPKLBjQZrCDHjaFhmiiWrPrk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=PDsF6KYlUvGKWfFQyH8bcjGO/G20JTCQOBGQU7+NjW7JH5RDPXyTyPOL5UrS70PeOEFAdghCUSX5DgQMIPX7bYZBfhm3EgwZfopPsKYrKVhpWbnyDkG+SDgwFXt7iVDAdxUAH9foGrwfiVpbR1moHYHASevQ//NhkIkme3/yuqs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=JHQlPkio; arc=none smtp.client-ip=209.85.214.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="JHQlPkio" Received: by mail-pl1-f178.google.com with SMTP id d9443c01a7336-1fee6435a34so4605645ad.0 for ; Wed, 18 Sep 2024 19:57:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726714629; x=1727319429; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jpwPGGqxMjTTALmPRltkI2obnVC0TIxSM0BTVcrrVEU=; b=JHQlPkioXnXQ3JMACqlWzgvoxzUdJSCPeMRb+KRRAqNFoa5nnRXo8he+cWdmK18VkR 6cc6KnU1ctug1kOpr64UFl2vjNGbu0ang5Rlrx97zTl4bw2uwbzyiYbfXtkU7ifFpjnk YAG1Y1PF/8Xt2qGvy3v854UYbjUDTrJsgwZFDhJZpShds9EBuAipRPehxjSrmJbsqrlV 5r3MTZ6dUSBVDNthQOPGa9tij45rglsgyfKw7IT9/C5ieWxx9I9eMtGNrpuQVLQ1vqXs lLAFqg0YBez9fIOJysDYvwGVkvIxXJsA/vsbyse9WPNj9fFqSD0OP7jd33kltcnCX0n3 8e3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726714629; x=1727319429; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jpwPGGqxMjTTALmPRltkI2obnVC0TIxSM0BTVcrrVEU=; b=Pf5HPJG4k/IAzfEuODEWMdor2+T+YzvaCChsImhKHyYiPHNgCK3sujHZVCjzmON+9d +qHbNEgDPyl9bArmzBAZvm0txsFWHWWSLNjdqxG7VUOuDKzqT+42k+428fXw/dOcY7Ow wOvM0//uJQm0XXvDrUahygpl+HNx8H9DMChkbMOwHtnqHMjpz+TvJrVngZxMjKDuzIO2 CBagx+ZNWUON15U09BCxWzXc+gb5+4j33/GwyVP25Az7z4BnEkMOcyAVkxFinZURk/Qa k/kMQS9rrd26Fdbv13VW38vYBc37+gYRQ4UQQHymp9Ci/FUO66AeyZ/CBVFPb9mxbE3v hEjQ== X-Forwarded-Encrypted: i=1; AJvYcCUjEB71D4OqD+x5WxIMFIB1LpWtphDrRrfQ4o+WlbUYOlVUXMovTmZ08uEf5xhEHxfBD/4ZzSTVil7JGgo=@vger.kernel.org X-Gm-Message-State: AOJu0Yxlx15YxvTGzBwqxgr/lvlAwnPUvJst9uT5fEYXxfISUYNIJNCo NuVQ1MiYzYdDqhkQbCQmMrtZhTx2lbkzNOLLBHYEMYdCqAFLo/vcMG/d8w== X-Google-Smtp-Source: AGHT+IEYcIlRMeaSd/gWkrGWTytjNF8qHos36tFlvKj7R9gv/PIoeETu5a0NdQvLU3Umw0/gtUoAEg== X-Received: by 2002:a17:903:2b0d:b0:205:9220:aa37 with SMTP id d9443c01a7336-2076e35f0a0mr358517625ad.22.1726714629566; Wed, 18 Sep 2024 19:57:09 -0700 (PDT) Received: from dw-tp.. ([171.76.85.129]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-207946d2823sm71389105ad.148.2024.09.18.19.57.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Sep 2024 19:57:09 -0700 (PDT) From: "Ritesh Harjani (IBM)" To: linuxppc-dev@lists.ozlabs.org Cc: Michael Ellerman , Nicholas Piggin , Madhavan Srinivasan , Christophe Leroy , Hari Bathini , "Aneesh Kumar K . V" , Donet Tom , Pavithra Prakash , Nirjhar Roy , LKML , kasan-dev@googlegroups.com, "Ritesh Harjani (IBM)" Subject: [RFC v2 09/13] book3s64/hash: Disable debug_pagealloc if it requires more memory Date: Thu, 19 Sep 2024 08:26:07 +0530 Message-ID: <8fa0c82332bfaa4c4766372ca0573021dbd8a85e.1726571179.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Make size of the linear map to be allocated in RMA region to be of ppc64_rma_size / 4. If debug_pagealloc requires more memory than that then do not allocate any memory and disable debug_pagealloc. Signed-off-by: Ritesh Harjani (IBM) --- arch/powerpc/mm/book3s64/hash_utils.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/arch/powerpc/mm/book3s64/hash_utils.c b/arch/powerpc/mm/book3s= 64/hash_utils.c index cc2eaa97982c..cffbb6499ac4 100644 --- a/arch/powerpc/mm/book3s64/hash_utils.c +++ b/arch/powerpc/mm/book3s64/hash_utils.c @@ -331,9 +331,19 @@ static unsigned long linear_map_hash_count; static DEFINE_RAW_SPINLOCK(linear_map_hash_lock); static inline void hash_debug_pagealloc_alloc_slots(void) { + unsigned long max_hash_count =3D ppc64_rma_size / 4; + if (!debug_pagealloc_enabled()) return; linear_map_hash_count =3D memblock_end_of_DRAM() >> PAGE_SHIFT; + if (unlikely(linear_map_hash_count > max_hash_count)) { + pr_info("linear map size (%llu) greater than 4 times RMA region (%llu). = Disabling debug_pagealloc\n", + ((u64)linear_map_hash_count << PAGE_SHIFT), + ppc64_rma_size); + linear_map_hash_count =3D 0; + return; + } + linear_map_hash_slots =3D memblock_alloc_try_nid( linear_map_hash_count, 1, MEMBLOCK_LOW_LIMIT, ppc64_rma_size, NUMA_NO_NODE); @@ -344,7 +354,7 @@ static inline void hash_debug_pagealloc_alloc_slots(voi= d) =20 static inline void hash_debug_pagealloc_add_slot(phys_addr_t paddr, int sl= ot) { - if (!debug_pagealloc_enabled()) + if (!debug_pagealloc_enabled() || !linear_map_hash_count) return; if ((paddr >> PAGE_SHIFT) < linear_map_hash_count) linear_map_hash_slots[paddr >> PAGE_SHIFT] =3D slot | 0x80; @@ -356,6 +366,9 @@ static int hash_debug_pagealloc_map_pages(struct page *= page, int numpages, unsigned long flags, vaddr, lmi; int i; =20 + if (!debug_pagealloc_enabled() || !linear_map_hash_count) + return 0; + local_irq_save(flags); for (i =3D 0; i < numpages; i++, page++) { vaddr =3D (unsigned long)page_address(page); --=20 2.46.0 From nobody Fri Nov 29 14:46:21 2024 Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 520C82BB05 for ; Thu, 19 Sep 2024 02:57:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726714636; cv=none; b=kv5VgJ1EvRynlFz50Xf/UrqTeYpgcyPh1tO43PUt/xp6WFGyXkPzQxbdUwrbkse/tEVVph7D/JE5hvy00SWtsqVjAzP3kTn/Bhs6shmR3XvAY0IuZRHPTSuqQQOs/mnJaNwiKDyGcZdlPZXKoe2KlMuUbcFNO3tF56ahznTcikE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726714636; c=relaxed/simple; bh=2xzcSmJ90OeB9AEpfOWapA2ZrBMBcKUdC05kW071KNs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fplJZv4oLXmctY6+tylnp7e9jJ908tRxs47hZVxBXrXI/2CDbfiXokbZZVhDrqEknzXPc4l+Ebdmyq+i/C2L3AnNQaG/YBzFD5Wm9/d9c8QfHCBozhVGfhdnN1UAwHyZS1l+xVdeIxcZK2Rrjk2w3o+rFV3S7RByqcbSY+vxZJM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=aR4/Mipm; arc=none smtp.client-ip=209.85.214.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="aR4/Mipm" Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-20543fdb7acso3504365ad.1 for ; Wed, 18 Sep 2024 19:57:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726714634; x=1727319434; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=OpMDq1IEGMMPwRojJ8i4AsUukx9XqidcnS2hx9BDF5Q=; b=aR4/MipmBDJkIaaYdLodYL7RDJpokc8SHXM+BkWcGFl3cSxpTh23fg8PVdaz+pWUJV dtEnPgNFumTJnCmT4fDCTWUwFCwbTtZ1OPycozdv2A+rbwfoBBezui58bEXNdawLwCoB 9w3qKlPqONbm79yV0YV1qNarRDOqWFsrPHI4G/64OK8dW9dB3uSZmKIHSRSJ/xhn4IIv g8nUftUpgXQt8ya7/bJznHfCn2Hd1mHjRnbBT/+oPRxIr1UPRN0giuvy4Ivl5lWwhynz zx38e2EdpvtjfVIHkf6zmX6VK00xvIi2Gwr+DIXySvj6wIbb8rlkYhdfXjrjjFIPs5id LzOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726714634; x=1727319434; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OpMDq1IEGMMPwRojJ8i4AsUukx9XqidcnS2hx9BDF5Q=; b=W1RXYP3eBKo/gQ7FTZHpUYFE4E8LlGPYyN23L5e6DeexK/jEeBInXxfB4wMI9btr+T kfQP/X2jOKyz5mtpvLott1k1IjR7jLX5j5MITrA9FYFuc2gaumJ081za3JCXy3Lv3UfI Ow0Pvo1bF+Tx8Y1UcFEfUNlmJFN/XtVRHKPwMGpg4XLY4ChD9SwaVe27SaFkeKZAmM4z XL6CnCwx+HqMXiMNFqigLPK03sMa1WHVHLjV6KOMcRkcaNtMiVKrro01MVpGDXqQ8M9D np7FZf1FHTyQ67KSDExBE1H5YTJTYxF7Z8oablgGRn+x1EN/3M6/S0fsNCe+slA2gK8M Gn9A== X-Forwarded-Encrypted: i=1; AJvYcCUKysX2QMQi43cQdSy5wdftOw+9lwAYnWkghKGY8fZlAL1Frdd0aznJg2++HqLMOww07N1Zz2IEupJ2IcI=@vger.kernel.org X-Gm-Message-State: AOJu0Yygp2t0paq+wFL+TWmW6BJBqlDLO6Mo67rM/sOm12htaXAj30oo xkce6vSv/fvXbcHwg1PlkWX88kAcldwbQqi1RK0jn9CM5bCjggA8 X-Google-Smtp-Source: AGHT+IF4quvbfkZ0rJHqO+7BogiiWf+cQnNI1id44Yv+kHDK3MIlFucEVogC7B736x0cPugHmuek8Q== X-Received: by 2002:a17:902:e80a:b0:202:190e:2adb with SMTP id d9443c01a7336-2078253791bmr316942145ad.36.1726714633762; Wed, 18 Sep 2024 19:57:13 -0700 (PDT) Received: from dw-tp.. ([171.76.85.129]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-207946d2823sm71389105ad.148.2024.09.18.19.57.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Sep 2024 19:57:13 -0700 (PDT) From: "Ritesh Harjani (IBM)" To: linuxppc-dev@lists.ozlabs.org Cc: Michael Ellerman , Nicholas Piggin , Madhavan Srinivasan , Christophe Leroy , Hari Bathini , "Aneesh Kumar K . V" , Donet Tom , Pavithra Prakash , Nirjhar Roy , LKML , kasan-dev@googlegroups.com, "Ritesh Harjani (IBM)" Subject: [RFC v2 10/13] book3s64/hash: Add kfence functionality Date: Thu, 19 Sep 2024 08:26:08 +0530 Message-ID: <449e751d8c64538076079a8fcb19749260817e62.1726571179.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that linear map functionality of debug_pagealloc is made generic, enable kfence to use this generic infrastructure. 1. Define kfence related linear map variables. - u8 *linear_map_kf_hash_slots; - unsigned long linear_map_kf_hash_count; - DEFINE_RAW_SPINLOCK(linear_map_kf_hash_lock); 2. The linear map size allocated in RMA region is quite small (KFENCE_POOL_SIZE >> PAGE_SHIFT) which is 512 bytes by default. 3. kfence pool memory is reserved using memblock_phys_alloc() which has can come from anywhere. (default 255 objects =3D> ((1+255) * 2) << PAGE_SHIFT =3D 32MB) 4. The hash slot information for kfence memory gets added in linear map in hash_linear_map_add_slot() (which also adds for debug_pagealloc). Reported-by: Pavithra Prakash Signed-off-by: Ritesh Harjani (IBM) --- arch/powerpc/include/asm/kfence.h | 5 - arch/powerpc/mm/book3s64/hash_utils.c | 162 +++++++++++++++++++++++--- 2 files changed, 149 insertions(+), 18 deletions(-) diff --git a/arch/powerpc/include/asm/kfence.h b/arch/powerpc/include/asm/k= fence.h index f3a9476a71b3..fab124ada1c7 100644 --- a/arch/powerpc/include/asm/kfence.h +++ b/arch/powerpc/include/asm/kfence.h @@ -10,7 +10,6 @@ =20 #include #include -#include =20 #ifdef CONFIG_PPC64_ELF_ABI_V1 #define ARCH_FUNC_PREFIX "." @@ -26,10 +25,6 @@ static inline void disable_kfence(void) =20 static inline bool arch_kfence_init_pool(void) { -#ifdef CONFIG_PPC64 - if (!radix_enabled()) - return false; -#endif return !kfence_disabled; } #endif diff --git a/arch/powerpc/mm/book3s64/hash_utils.c b/arch/powerpc/mm/book3s= 64/hash_utils.c index cffbb6499ac4..53e6f3a524eb 100644 --- a/arch/powerpc/mm/book3s64/hash_utils.c +++ b/arch/powerpc/mm/book3s64/hash_utils.c @@ -40,6 +40,7 @@ #include #include #include +#include =20 #include #include @@ -66,6 +67,7 @@ #include #include #include +#include =20 #include =20 @@ -271,7 +273,7 @@ void hash__tlbiel_all(unsigned int action) WARN(1, "%s called on pre-POWER7 CPU\n", __func__); } =20 -#ifdef CONFIG_DEBUG_PAGEALLOC +#if defined(CONFIG_DEBUG_PAGEALLOC) || defined(CONFIG_KFENCE) static void kernel_map_linear_page(unsigned long vaddr, unsigned long idx, u8 *slots, raw_spinlock_t *lock) { @@ -325,11 +327,13 @@ static void kernel_unmap_linear_page(unsigned long va= ddr, unsigned long idx, mmu_linear_psize, mmu_kernel_ssize, 0); } +#endif =20 +#ifdef CONFIG_DEBUG_PAGEALLOC static u8 *linear_map_hash_slots; static unsigned long linear_map_hash_count; static DEFINE_RAW_SPINLOCK(linear_map_hash_lock); -static inline void hash_debug_pagealloc_alloc_slots(void) +static void hash_debug_pagealloc_alloc_slots(void) { unsigned long max_hash_count =3D ppc64_rma_size / 4; =20 @@ -352,7 +356,8 @@ static inline void hash_debug_pagealloc_alloc_slots(voi= d) __func__, linear_map_hash_count, &ppc64_rma_size); } =20 -static inline void hash_debug_pagealloc_add_slot(phys_addr_t paddr, int sl= ot) +static inline void hash_debug_pagealloc_add_slot(phys_addr_t paddr, + int slot) { if (!debug_pagealloc_enabled() || !linear_map_hash_count) return; @@ -386,20 +391,148 @@ static int hash_debug_pagealloc_map_pages(struct pag= e *page, int numpages, return 0; } =20 -int hash__kernel_map_pages(struct page *page, int numpages, int enable) +#else /* CONFIG_DEBUG_PAGEALLOC */ +static inline void hash_debug_pagealloc_alloc_slots(void) {} +static inline void hash_debug_pagealloc_add_slot(phys_addr_t paddr, int sl= ot) {} +static int __maybe_unused +hash_debug_pagealloc_map_pages(struct page *page, int numpages, int enable) { - return hash_debug_pagealloc_map_pages(page, numpages, enable); + return 0; } +#endif /* CONFIG_DEBUG_PAGEALLOC */ =20 -#else /* CONFIG_DEBUG_PAGEALLOC */ -int hash__kernel_map_pages(struct page *page, int numpages, - int enable) +#ifdef CONFIG_KFENCE +static u8 *linear_map_kf_hash_slots; +static unsigned long linear_map_kf_hash_count; +static DEFINE_RAW_SPINLOCK(linear_map_kf_hash_lock); + +static phys_addr_t kfence_pool; + +static inline void hash_kfence_alloc_pool(void) +{ + + // allocate linear map for kfence within RMA region + linear_map_kf_hash_count =3D KFENCE_POOL_SIZE >> PAGE_SHIFT; + linear_map_kf_hash_slots =3D memblock_alloc_try_nid( + linear_map_kf_hash_count, 1, + MEMBLOCK_LOW_LIMIT, ppc64_rma_size, + NUMA_NO_NODE); + if (!linear_map_kf_hash_slots) { + pr_err("%s: memblock for linear map (%lu) failed\n", __func__, + linear_map_kf_hash_count); + goto err; + } + + // allocate kfence pool early + kfence_pool =3D memblock_phys_alloc_range(KFENCE_POOL_SIZE, PAGE_SIZE, + MEMBLOCK_LOW_LIMIT, MEMBLOCK_ALLOC_ANYWHERE); + if (!kfence_pool) { + pr_err("%s: memblock for kfence pool (%lu) failed\n", __func__, + KFENCE_POOL_SIZE); + memblock_free(linear_map_kf_hash_slots, + linear_map_kf_hash_count); + linear_map_kf_hash_count =3D 0; + goto err; + } + memblock_mark_nomap(kfence_pool, KFENCE_POOL_SIZE); + + return; +err: + pr_info("Disabling kfence\n"); + disable_kfence(); +} + +static inline void hash_kfence_map_pool(void) +{ + unsigned long kfence_pool_start, kfence_pool_end; + unsigned long prot =3D pgprot_val(PAGE_KERNEL); + + if (!kfence_pool) + return; + + kfence_pool_start =3D (unsigned long) __va(kfence_pool); + kfence_pool_end =3D kfence_pool_start + KFENCE_POOL_SIZE; + __kfence_pool =3D (char *) kfence_pool_start; + BUG_ON(htab_bolt_mapping(kfence_pool_start, kfence_pool_end, + kfence_pool, prot, mmu_linear_psize, + mmu_kernel_ssize)); + memblock_clear_nomap(kfence_pool, KFENCE_POOL_SIZE); +} + +static inline void hash_kfence_add_slot(phys_addr_t paddr, int slot) { + unsigned long vaddr =3D (unsigned long) __va(paddr); + unsigned long lmi =3D (vaddr - (unsigned long)__kfence_pool) + >> PAGE_SHIFT; + + if (!kfence_pool) + return; + BUG_ON(!is_kfence_address((void *)vaddr)); + BUG_ON(lmi >=3D linear_map_kf_hash_count); + linear_map_kf_hash_slots[lmi] =3D slot | 0x80; +} + +static int hash_kfence_map_pages(struct page *page, int numpages, int enab= le) +{ + unsigned long flags, vaddr, lmi; + int i; + + WARN_ON_ONCE(!linear_map_kf_hash_count); + local_irq_save(flags); + for (i =3D 0; i < numpages; i++, page++) { + vaddr =3D (unsigned long)page_address(page); + lmi =3D (vaddr - (unsigned long)__kfence_pool) >> PAGE_SHIFT; + + /* Ideally this should never happen */ + if (lmi >=3D linear_map_kf_hash_count) { + WARN_ON_ONCE(1); + continue; + } + + if (enable) + kernel_map_linear_page(vaddr, lmi, + linear_map_kf_hash_slots, + &linear_map_kf_hash_lock); + else + kernel_unmap_linear_page(vaddr, lmi, + linear_map_kf_hash_slots, + &linear_map_kf_hash_lock); + } + local_irq_restore(flags); return 0; } -static inline void hash_debug_pagealloc_alloc_slots(void) {} -static inline void hash_debug_pagealloc_add_slot(phys_addr_t paddr, int sl= ot) {} -#endif /* CONFIG_DEBUG_PAGEALLOC */ +#else +static inline void hash_kfence_alloc_pool(void) {} +static inline void hash_kfence_map_pool(void) {} +static inline void hash_kfence_add_slot(phys_addr_t paddr, int slot) {} +static int __maybe_unused +hash_kfence_map_pages(struct page *page, int numpages, int enable) +{ + return 0; +} +#endif + +#if defined(CONFIG_DEBUG_PAGEALLOC) || defined(CONFIG_KFENCE) +int hash__kernel_map_pages(struct page *page, int numpages, int enable) +{ + void *vaddr =3D page_address(page); + + if (is_kfence_address(vaddr)) + return hash_kfence_map_pages(page, numpages, enable); + else + return hash_debug_pagealloc_map_pages(page, numpages, enable); +} + +static void hash_linear_map_add_slot(phys_addr_t paddr, int slot) +{ + if (is_kfence_address(__va(paddr))) + hash_kfence_add_slot(paddr, slot); + else + hash_debug_pagealloc_add_slot(paddr, slot); +} +#else +static void hash_linear_map_add_slot(phys_addr_t paddr, int slot) {} +#endif =20 /* * 'R' and 'C' update notes: @@ -559,7 +692,8 @@ int htab_bolt_mapping(unsigned long vstart, unsigned lo= ng vend, break; =20 cond_resched(); - hash_debug_pagealloc_add_slot(paddr, ret); + // add slot info in debug_pagealloc / kfence linear map + hash_linear_map_add_slot(paddr, ret); } return ret < 0 ? ret : 0; } @@ -940,7 +1074,7 @@ static void __init htab_init_page_sizes(void) bool aligned =3D true; init_hpte_page_sizes(); =20 - if (!debug_pagealloc_enabled()) { + if (!debug_pagealloc_enabled_or_kfence()) { /* * Pick a size for the linear mapping. Currently, we only * support 16M, 1M and 4K which is the default @@ -1261,6 +1395,7 @@ static void __init htab_initialize(void) prot =3D pgprot_val(PAGE_KERNEL); =20 hash_debug_pagealloc_alloc_slots(); + hash_kfence_alloc_pool(); /* create bolted the linear mapping in the hash table */ for_each_mem_range(i, &base, &end) { size =3D end - base; @@ -1277,6 +1412,7 @@ static void __init htab_initialize(void) BUG_ON(htab_bolt_mapping(base, base + size, __pa(base), prot, mmu_linear_psize, mmu_kernel_ssize)); } + hash_kfence_map_pool(); memblock_set_current_limit(MEMBLOCK_ALLOC_ANYWHERE); =20 /* --=20 2.46.0 From nobody Fri Nov 29 14:46:21 2024 Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8A1A02D611 for ; Thu, 19 Sep 2024 02:57:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726714640; cv=none; b=ZxFAEAkb94AQu7d6KPrT2y2bsLFhHkuQ4FQf5VVXyxi4BAijhXgBYT6CxfD3PP6fl/RH2FRwjcwPKcPBFUhRnpjhDvtQg0OwVYhI3oxnesYiWiTrXL6q0+ASjhrBGyrS+K2CeBNMZ2iaCckRdnD5ksXnOeaO92p9KnJ4Uy41PJE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726714640; c=relaxed/simple; bh=IrvocGPAGOkqMhnkbOHlfXSKlsgjDUpOdoDHwplLjvE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=B0OGiaUh+StDudk6KqoElKs9RGJ77tyig7TY+yCvmtK+skipwB0QDm9z2Lz3OY6sxjuUK5eY9CqvnlNp6HO2/AB5JjnCZFzM1+bXzltTqEJ8Sp5YlMqOiQdHnoL1iyi+u1U5kD53mJVySjVJQkqol0jLKrS0JoemIWbUk1Hyt4s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=DHpPGvpa; arc=none smtp.client-ip=209.85.214.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="DHpPGvpa" Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-208ccde82e6so3158935ad.0 for ; Wed, 18 Sep 2024 19:57:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726714638; x=1727319438; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AV6BxL5gMqFPtklTutfX47z8PwIuU3wFqvoyRitrLCw=; b=DHpPGvpaMRK5let+p6SuU2zIRyhiP6z2olKV68met5ghc3PQ11dKd54g5nogXUQMLO /vTObbFihh9fLmyM1kKK1+oSBNRoveYyCYK6Xo8fdG2AK2w5UL6oLVLLr+EN8uC3P9xp AKxbUwWQWM7n/Qc1SPdiQHnYKpyFFrFRr9erBXZNoG8an4b49OABh9cGBFzuoAlyQzJy NwebU+yh62knaPqJEXOiYrjXnbkwO/NY5n9aW5ornU7rCOM2leJqHfzp13CBIZf6g0ml FqnmuJAQbiBILmIEpbU/Uqyt5DT4Fx3yzH+HFKLOnpX1lOidhYUapxKXowgQWB9rLD8m apVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726714638; x=1727319438; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AV6BxL5gMqFPtklTutfX47z8PwIuU3wFqvoyRitrLCw=; b=buNfMNWS2847GCXKb5ltosxyGJq5upS2j81VcgZtAqHpQcwnEavkW34K2Af+Asnc37 HirsooSRU0XyZgIajW2El+JsUQfdJfB/bN/roDhCC13LtdnNnoAP7Lvu9dclfMKncibn m/buABrvV4fEBmQi8mokm3OpFaLc4wl+6DsGDRsnhXrl/vkCCFGEOhxqC0CLZ95ibywU ZKcqQYNlwzqLwnB92egZvfL0tOG/hdLrPLGumzocHr7FvMX8Jxv1F831MiGP7FJ567bG C6EcxIXg33z2oNYsFjBNAm3EAadyXrtqpuwVKc0NKu47JS6V6AADXCH+4cPVO1btnmHm u+ZA== X-Forwarded-Encrypted: i=1; AJvYcCXHE8D3+/x9mMGEUYJ7hXeHSUqkKpKyuJ6qiVe0t00tE4ZuvsxDGQ/KQmWeb86HQAs6xy/2qiywkY6b9S4=@vger.kernel.org X-Gm-Message-State: AOJu0Yz39Y66SVIgHWO1rN4CMkm3clB0v1MiRLAmdcNpnslwgIknDmKy I+gcEssLcxzCQ3a/Q7ul5mQI727l0Zoc3bKB1WfKecjvLsSAlhSD X-Google-Smtp-Source: AGHT+IEcNBq8l5FK/5juI+RUhRkCIsvdP0Nu+zueXcpdJJCPT6m3f88Yiku2lxjEo/N6lofe8UkElQ== X-Received: by 2002:a17:902:e545:b0:205:7db3:fdd1 with SMTP id d9443c01a7336-2076e3b7821mr382291845ad.36.1726714637877; Wed, 18 Sep 2024 19:57:17 -0700 (PDT) Received: from dw-tp.. ([171.76.85.129]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-207946d2823sm71389105ad.148.2024.09.18.19.57.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Sep 2024 19:57:17 -0700 (PDT) From: "Ritesh Harjani (IBM)" To: linuxppc-dev@lists.ozlabs.org Cc: Michael Ellerman , Nicholas Piggin , Madhavan Srinivasan , Christophe Leroy , Hari Bathini , "Aneesh Kumar K . V" , Donet Tom , Pavithra Prakash , Nirjhar Roy , LKML , kasan-dev@googlegroups.com, "Ritesh Harjani (IBM)" Subject: [RFC v2 11/13] book3s64/radix: Refactoring common kfence related functions Date: Thu, 19 Sep 2024 08:26:09 +0530 Message-ID: <0711340e9050001020c284154064b3a4cf781045.1726571179.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Both radix and hash on book3s requires to detect if kfence early init is enabled or not. Hash needs to disable kfence if early init is not enabled because with kfence the linear map is mapped using PAGE_SIZE rather than 16M mapping. We don't support multiple page sizes for slb entry used for kernel linear map in book3s64. This patch refactors out the common functions required to detect kfence early init is enabled or not. Signed-off-by: Ritesh Harjani (IBM) --- arch/powerpc/include/asm/kfence.h | 8 ++++++-- arch/powerpc/mm/book3s64/pgtable.c | 13 +++++++++++++ arch/powerpc/mm/book3s64/radix_pgtable.c | 12 ------------ arch/powerpc/mm/init-common.c | 1 + 4 files changed, 20 insertions(+), 14 deletions(-) diff --git a/arch/powerpc/include/asm/kfence.h b/arch/powerpc/include/asm/k= fence.h index fab124ada1c7..1f7cab58ab2c 100644 --- a/arch/powerpc/include/asm/kfence.h +++ b/arch/powerpc/include/asm/kfence.h @@ -15,7 +15,7 @@ #define ARCH_FUNC_PREFIX "." #endif =20 -#ifdef CONFIG_KFENCE +extern bool kfence_early_init; extern bool kfence_disabled; =20 static inline void disable_kfence(void) @@ -27,7 +27,11 @@ static inline bool arch_kfence_init_pool(void) { return !kfence_disabled; } -#endif + +static inline bool kfence_early_init_enabled(void) +{ + return IS_ENABLED(CONFIG_KFENCE) && kfence_early_init; +} =20 #ifdef CONFIG_PPC64 static inline bool kfence_protect_page(unsigned long addr, bool protect) diff --git a/arch/powerpc/mm/book3s64/pgtable.c b/arch/powerpc/mm/book3s64/= pgtable.c index f4d8d3c40e5c..1563a8c28feb 100644 --- a/arch/powerpc/mm/book3s64/pgtable.c +++ b/arch/powerpc/mm/book3s64/pgtable.c @@ -37,6 +37,19 @@ EXPORT_SYMBOL(__pmd_frag_nr); unsigned long __pmd_frag_size_shift; EXPORT_SYMBOL(__pmd_frag_size_shift); =20 +#ifdef CONFIG_KFENCE +extern bool kfence_early_init; +static int __init parse_kfence_early_init(char *arg) +{ + int val; + + if (get_option(&arg, &val)) + kfence_early_init =3D !!val; + return 0; +} +early_param("kfence.sample_interval", parse_kfence_early_init); +#endif + #ifdef CONFIG_TRANSPARENT_HUGEPAGE /* * This is called when relaxing access to a hugepage. It's also called in = the page diff --git a/arch/powerpc/mm/book3s64/radix_pgtable.c b/arch/powerpc/mm/boo= k3s64/radix_pgtable.c index b0d927009af8..311e2112d782 100644 --- a/arch/powerpc/mm/book3s64/radix_pgtable.c +++ b/arch/powerpc/mm/book3s64/radix_pgtable.c @@ -363,18 +363,6 @@ static int __meminit create_physical_mapping(unsigned = long start, } =20 #ifdef CONFIG_KFENCE -static bool __ro_after_init kfence_early_init =3D !!CONFIG_KFENCE_SAMPLE_I= NTERVAL; - -static int __init parse_kfence_early_init(char *arg) -{ - int val; - - if (get_option(&arg, &val)) - kfence_early_init =3D !!val; - return 0; -} -early_param("kfence.sample_interval", parse_kfence_early_init); - static inline phys_addr_t alloc_kfence_pool(void) { phys_addr_t kfence_pool; diff --git a/arch/powerpc/mm/init-common.c b/arch/powerpc/mm/init-common.c index 9b4a675eb8f8..85875820b113 100644 --- a/arch/powerpc/mm/init-common.c +++ b/arch/powerpc/mm/init-common.c @@ -33,6 +33,7 @@ bool disable_kuep =3D !IS_ENABLED(CONFIG_PPC_KUEP); bool disable_kuap =3D !IS_ENABLED(CONFIG_PPC_KUAP); #ifdef CONFIG_KFENCE bool __ro_after_init kfence_disabled; +bool __ro_after_init kfence_early_init =3D !!CONFIG_KFENCE_SAMPLE_INTERVAL; #endif =20 static int __init parse_nosmep(char *p) --=20 2.46.0 From nobody Fri Nov 29 14:46:21 2024 Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DF5E838DDB for ; Thu, 19 Sep 2024 02:57:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726714644; cv=none; b=GksuiSoCDicn6FSb9h8Ft8H8w8PtcmNBusEb+ddFlLKtYg7PMfwinc/Cpgwl7GYfsX0rId59fegYAa/LNAodC8AMhsx1GU4LgsDidDjZ6zDAV+mz8V2WCwV4flZjYf/NlUgV6xIpbd1V9eE/3zvZ0S0TtWAzCnKWwuwHyFwBiRk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726714644; c=relaxed/simple; bh=rKAkwnrKqFe/pXWHE++1cg7Hf02Exo4/eTPzv1Uabug=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=NRZFMsv8oL1MF1I+eYgBnxQnwQIzFL4vF8Oh/UMMK+FHyP/d7KQslVbEB3KlUJWJQL8mHtc0klOtB+v6xF72YWPhxRjxaEqVM2FNlaJ0RxHhw11VpkvtJroT8OxINjuU7ZJAh7LyKYFidtOeycgA1YyKNhqfyFpgvcIW4Ap2ffo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=e2twLYMU; arc=none smtp.client-ip=209.85.214.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="e2twLYMU" Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-2053616fa36so5561715ad.0 for ; Wed, 18 Sep 2024 19:57:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726714642; x=1727319442; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=t5daj1tikBz1GmvjDxbc2gCsvgSNJoz/zYutj+6dtSk=; b=e2twLYMUXf1t5YMRV5DzxKQDZD9eWUGus+pN/5VjMvLJpPbY1JU3n06vOBcMEam4DB fShmrf5w4DgdbkFLZpl0GkqI4bCNP+TTZ+wJAMxTVpsNDBDDw83XQRqPuhCyHkMyIMgq RIAeuI88bAMxHkZmdVjSDPiQEbAszy9QSDc/0g4eC6pfM/jXevto+xRgdFcXxPUA4lQx YSu1f5npyJYv5nRQCdso2m7OH3yaK1h6w8SM3d5iCu4LMZD0cz2LCgdag+oHGQbsEyjL Ta7f3DyYKWRSAEDF35o1TPmT0WPjRxPVhjGuPv8uLb3MmCeuRxjxHYERGovD9Z8VO2RG d3Zg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726714642; x=1727319442; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=t5daj1tikBz1GmvjDxbc2gCsvgSNJoz/zYutj+6dtSk=; b=vAiC7bp6yNEz/Cd5+ftQ7wlU7ODDe+9jvw+93gW7B7F4oUKDarCCFIfmitLoKIcHhO +RQHABC6OfT7DzYlyCxzN3UwSFDmjVd5YN9L5GEoktbwcqMmNB2u2g316ZD+4c/nMEQg bJOA1/wZ7MPbZ8VXIq4WiNzBzcx9YuKaMaVyc8DrIosq12/lDraShXwYn+PAEJSxlEYY 8zW5Cs1zmO5kKEfMuAfQm+P6HvOqjPGydx/U1ckX79eZVXcs232OZcsyFVvrurIaJQUD dfowaNwSHMXTf/8i1Te/XibHQ3CGdt/FGQMved1YjnZ04kX9Z383/4npFHiaZmuA+8CX 4JOA== X-Forwarded-Encrypted: i=1; AJvYcCUietOpeGAdG2lCoWEQTgjxfgkByZIJSVnSZxIS1f9zl04oTPKBNFkw9z7FrVpQMzkIvZpmylWWsXp3Z7Y=@vger.kernel.org X-Gm-Message-State: AOJu0Yy3VuWUsWfRG+4aAhdBxIccMt4lqsB/pVTRdofJSdKAWclSOnuN /sJ1zwCFUdp6PtpV/TGHR6P0WedZU5h0DLUl+SAjf8mY9JsnAz+Mo6amCw== X-Google-Smtp-Source: AGHT+IE3VeS4AWbgmq58tHFaKEJ3AWgWdI2umfqoY9F5wsipy1rAhwBkftW22XvGMU7l7yJyDOWfeA== X-Received: by 2002:a17:902:ea0b:b0:206:8c4a:7b73 with SMTP id d9443c01a7336-2076e414901mr370597715ad.50.1726714642162; Wed, 18 Sep 2024 19:57:22 -0700 (PDT) Received: from dw-tp.. ([171.76.85.129]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-207946d2823sm71389105ad.148.2024.09.18.19.57.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Sep 2024 19:57:21 -0700 (PDT) From: "Ritesh Harjani (IBM)" To: linuxppc-dev@lists.ozlabs.org Cc: Michael Ellerman , Nicholas Piggin , Madhavan Srinivasan , Christophe Leroy , Hari Bathini , "Aneesh Kumar K . V" , Donet Tom , Pavithra Prakash , Nirjhar Roy , LKML , kasan-dev@googlegroups.com, "Ritesh Harjani (IBM)" Subject: [RFC v2 12/13] book3s64/hash: Disable kfence if not early init Date: Thu, 19 Sep 2024 08:26:10 +0530 Message-ID: <43e00322c775645a251c4526484d5bc61c62850d.1726571179.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Enable kfence on book3s64 hash only when early init is enabled. This is because, kfence could cause the kernel linear map to be mapped at PAGE_SIZE level instead of 16M (which I guess we don't want). Also currently there is no way to - 1. Make multiple page size entries for the SLB used for kernel linear map. 2. No easy way of getting the hash slot details after the page table mapping for kernel linear setup. So even if kfence allocate the pool in late init, we won't be able to get the hash slot details in kfence linear map. Thus this patch disables kfence on hash if kfence early init is not enabled. Signed-off-by: Ritesh Harjani (IBM) --- arch/powerpc/mm/book3s64/hash_utils.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/powerpc/mm/book3s64/hash_utils.c b/arch/powerpc/mm/book3s= 64/hash_utils.c index 53e6f3a524eb..b6da25719e37 100644 --- a/arch/powerpc/mm/book3s64/hash_utils.c +++ b/arch/powerpc/mm/book3s64/hash_utils.c @@ -410,6 +410,8 @@ static phys_addr_t kfence_pool; =20 static inline void hash_kfence_alloc_pool(void) { + if (!kfence_early_init_enabled()) + goto err; =20 // allocate linear map for kfence within RMA region linear_map_kf_hash_count =3D KFENCE_POOL_SIZE >> PAGE_SHIFT; @@ -1074,7 +1076,7 @@ static void __init htab_init_page_sizes(void) bool aligned =3D true; init_hpte_page_sizes(); =20 - if (!debug_pagealloc_enabled_or_kfence()) { + if (!debug_pagealloc_enabled() && !kfence_early_init_enabled()) { /* * Pick a size for the linear mapping. Currently, we only * support 16M, 1M and 4K which is the default --=20 2.46.0 From nobody Fri Nov 29 14:46:21 2024 Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1DB4541746 for ; Thu, 19 Sep 2024 02:57:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726714648; cv=none; b=gX/HGIOW9bxvlow49oHUKbe1TQQCVeUeSY1m7ZDxn+e73GR8224d2Fpv8jRvsRi1Jf5rcRnIWR/20jL+y6FOMr+b1v3plUwaWwKhalUy1OIIjrdZ+8YwWFsSkV1s5tSEJiMCuH33Nqp6udvnZqIjGLWgjKZAiH/0yNw15KZsZjg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726714648; c=relaxed/simple; bh=7O0Fesye+x8XRmHWMdz8FpcQFaEIlQURErbev6IpoRI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BW5Qvx4/t0zaZ0WBaSHRAJc17XswYdW4FTv7PyfvIyEUdyrsPL6q84yuKRkKbEvdNl9vCAElh9Qq4h5i15pg51N+nbscFoP3ZeushkkypuGabKlt23wcAW0Sj34cqFR3EnnNpyYcpvCwFlHeKNgdm/NshkCGrVLVTiwmTI3vsTY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=AC84Ppyx; arc=none smtp.client-ip=209.85.214.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="AC84Ppyx" Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-206bd1c6ccdso4157385ad.3 for ; Wed, 18 Sep 2024 19:57:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726714646; x=1727319446; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rG0rlJ7RpNnGy840avhtCY+OIP6FbV4qxEoUkbOoVC0=; b=AC84Ppyxhw9If8DjXcoKQQosyP54fVWwcqB7lDFxdbLw3aL7GLa/tUZvunR43LsllI a9WkX7/OefDA9aRWtVLdENz2BHPcu9KwxrJl33hUOlxqn1+T9XtII82MwAs17lqVxgyL fgvkB5QxwVo0fjyyeAkZjM3DyJx/YVjbmubbnaba+AEx+Uq4uuE41gOOgjcIXfOs3vit 7ji2xUxcLIvKMISVINTVJjf5paiiHYTkhPJs0qaBp2VTKat1gRRszPYn8w6qcDrzfxUs kNz8cSy2vSYK1jjF6A5CjjnngUFpCNRGNcC1nwBkdzGxh5LppgO4/BGfmXkifIgLfK4e M6AQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726714646; x=1727319446; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rG0rlJ7RpNnGy840avhtCY+OIP6FbV4qxEoUkbOoVC0=; b=khEv/dYE2DdzyCZbkdt8hXH0F0HekWD30OSatnEE1uLydjRsJDl8PdS/h4b2s58ycX Fpusi/x7OIrfpd1DrxDERUWsR9N+LXVU4CpQq42oQ5pb/NARrsrakD42LKRBuZosaYfW ghu54FxuZwjnaJBPzfcZybArMerPL/LW8fbZhWjE2ji2hOZ39XBXu8jNnnUPhJvkriTx TcF56SzjCQkw19K+loqsLfgGtwY+3PRk3//ec3c75TiwUXJWroDGnXSrEj9By5VZ29tC yLqGFwmQFxXh54sWep66pHbxvbx8D023Ib13iBo0pTqf5CecswiLQ2a7vbLOjbVIJaZw oHkg== X-Forwarded-Encrypted: i=1; AJvYcCU8r5EnbDyXwFLs4f/L1bsG9ofv1e4uYdrVJ/XLdRkjYSdN0RxuKWUnCtX1EpCJ3++Rn4Z8rqOi0gqn288=@vger.kernel.org X-Gm-Message-State: AOJu0YxKQUWK2LeYtxLwEolsnAi+wdGEfbARMi5BXVuQQVkiguC8Zi23 FlFKoiHAKbAxDkLK/cfVHt9Hl60BDPCqfS10Ep9m5ilq3kYBkjI/ X-Google-Smtp-Source: AGHT+IE34eW0PHWVkirCwr75sAQsAEQKWo1G+ukba10mNkf/9dp+AGD30+GUy1tuone82B+yBz345Q== X-Received: by 2002:a17:903:32c9:b0:206:ae39:9f4 with SMTP id d9443c01a7336-20781d5f6fcmr301143135ad.20.1726714646433; Wed, 18 Sep 2024 19:57:26 -0700 (PDT) Received: from dw-tp.. ([171.76.85.129]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-207946d2823sm71389105ad.148.2024.09.18.19.57.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Sep 2024 19:57:25 -0700 (PDT) From: "Ritesh Harjani (IBM)" To: linuxppc-dev@lists.ozlabs.org Cc: Michael Ellerman , Nicholas Piggin , Madhavan Srinivasan , Christophe Leroy , Hari Bathini , "Aneesh Kumar K . V" , Donet Tom , Pavithra Prakash , Nirjhar Roy , LKML , kasan-dev@googlegroups.com, "Ritesh Harjani (IBM)" Subject: [RFC v2 13/13] book3s64/hash: Early detect debug_pagealloc size requirement Date: Thu, 19 Sep 2024 08:26:11 +0530 Message-ID: <616bf94910b0c77323ea9ccb86571f78ebfd421b.1726571179.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add hash_supports_debug_pagealloc() helper to detect whether debug_pagealloc can be supported on hash or not. This checks for both, whether debug_pagealloc config is enabled and the linear map should fit within rma_size/4 region size. This can then be used early during htab_init_page_sizes() to decide linear map pagesize if hash supports either debug_pagealloc or kfence. Signed-off-by: Ritesh Harjani (IBM) --- arch/powerpc/mm/book3s64/hash_utils.c | 25 +++++++++++++------------ 1 file changed, 13 insertions(+), 12 deletions(-) diff --git a/arch/powerpc/mm/book3s64/hash_utils.c b/arch/powerpc/mm/book3s= 64/hash_utils.c index b6da25719e37..3ffc98b3deb1 100644 --- a/arch/powerpc/mm/book3s64/hash_utils.c +++ b/arch/powerpc/mm/book3s64/hash_utils.c @@ -329,25 +329,26 @@ static void kernel_unmap_linear_page(unsigned long va= ddr, unsigned long idx, } #endif +static inline bool hash_supports_debug_pagealloc(void) +{ + unsigned long max_hash_count =3D ppc64_rma_size / 4; + unsigned long linear_map_count =3D memblock_end_of_DRAM() >> PAGE_SHIFT; + + if (!debug_pagealloc_enabled() || linear_map_count > max_hash_count) + return false; + return true; +} + #ifdef CONFIG_DEBUG_PAGEALLOC static u8 *linear_map_hash_slots; static unsigned long linear_map_hash_count; static DEFINE_RAW_SPINLOCK(linear_map_hash_lock); static void hash_debug_pagealloc_alloc_slots(void) { - unsigned long max_hash_count =3D ppc64_rma_size / 4; - - if (!debug_pagealloc_enabled()) - return; - linear_map_hash_count =3D memblock_end_of_DRAM() >> PAGE_SHIFT; - if (unlikely(linear_map_hash_count > max_hash_count)) { - pr_info("linear map size (%llu) greater than 4 times RMA region (%llu). = Disabling debug_pagealloc\n", - ((u64)linear_map_hash_count << PAGE_SHIFT), - ppc64_rma_size); - linear_map_hash_count =3D 0; + if (!hash_supports_debug_pagealloc()) return; - } + linear_map_hash_count =3D memblock_end_of_DRAM() >> PAGE_SHIFT; linear_map_hash_slots =3D memblock_alloc_try_nid( linear_map_hash_count, 1, MEMBLOCK_LOW_LIMIT, ppc64_rma_size, NUMA_NO_NODE); @@ -1076,7 +1077,7 @@ static void __init htab_init_page_sizes(void) bool aligned =3D true; init_hpte_page_sizes(); - if (!debug_pagealloc_enabled() && !kfence_early_init_enabled()) { + if (!hash_supports_debug_pagealloc() && !kfence_early_init_enabled()) { /* * Pick a size for the linear mapping. Currently, we only * support 16M, 1M and 4K which is the default -- 2.46.0