From nobody Mon Jun 29 07:49:26 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 60FEFC433F5 for ; Mon, 14 Feb 2022 12:03:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352300AbiBNMD5 (ORCPT ); Mon, 14 Feb 2022 07:03:57 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:33042 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234179AbiBNMDu (ORCPT ); Mon, 14 Feb 2022 07:03:50 -0500 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8463EE3D for ; Mon, 14 Feb 2022 04:03:43 -0800 (PST) Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 21EBeNu4031188; Mon, 14 Feb 2022 12:03:41 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=1VLSBXEHw9Z9IXuQW5DktTaYds/UsYLR7xUfcY8F4Ns=; b=FgzMYz4mzrHQYJ9iuUZFWggGObFfyP3Ofiy/47VoZTvCNfNt3by4CS2Yf3agnIgqFump YQNJJAZNG/LMl9tvf2MwJvC9JNtFq8eZNQL664YZhsRB5XTbNG4ZwarsZzpnN32WMQd4 2Hc/ojkRdt4DoR3cPexS11FCfR5o/0GpLACHU0bfFdgbf7mM2vEvNv0IjQ/G9/oqgfPG pGDSZX4B4YS1j4CY1FJYeU7P9DVbWo7BIr/NV0+uRKEskYpFKW0BDxOQaYD0dx+dA5nZ LVL+DVHS2Eru/mAS+LcX+PUIqCIh8gU0VXSCFIGyPbBKRzRsSiQ3KiZzW0o61c8sXP0p mg== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by mx0b-00069f02.pphosted.com with ESMTP id 3e64sbv328-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 14 Feb 2022 12:03:40 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 21EBuRce176470; Mon, 14 Feb 2022 12:03:39 GMT Received: from nam12-bn8-obe.outbound.protection.outlook.com (mail-bn8nam12lp2175.outbound.protection.outlook.com [104.47.55.175]) by aserp3030.oracle.com with ESMTP id 3e62xd07d4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 14 Feb 2022 12:03:39 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dRrZoU1YmYtZmJgZBEEub5VIMADi05M9SXjoO7ZJ+83nMm1Cpui02APKlCFvquxelq16op8rm4ZyQ9bmkFNtAF0H5Fh9cHXxwmu5zQuMAhzYjkTIVgSSReiM579ipj848/vaJ7NC3jlG6mAzeBFd+4eO+iOWFdjk3UVpmODC4rrxS6k9CS9A1b59CUofgJ12UXEviB6/1WHwHx+5WSEOBQRCOgfkUmm1L7npkv2mUs9F5+dIVSItV2oqtcoFswirVSuvw7AKGOkozcLP0DAZa//SCN/wPTQjevwcmmVFo898wsuko4/HlB5EzFt0uoohcT7mRZPpBUKTHfFhrzu1Eg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=1VLSBXEHw9Z9IXuQW5DktTaYds/UsYLR7xUfcY8F4Ns=; b=fM06uA4Ln4Dv78xLh/TsRt6IV2nRrNGiAF6RB+LcS6S/Fth7Jv2/A28cOYeZdRvT3UWp6XPCeU6uHXnEtDhsNvimgCwb9z0PsHQpVRzxsMPRb6q/Pc2ujVOwOh59UqA5DzrO5ltGgYT0hxweyOM1U1XD9xazvHA/Htj67YQbFN3g5kJcIjf6vWUspTiUCIj+mGH4kfN34U51Sh75CIe1jUxpix5lRY/rFzT5NLliA1v5jFS+36GXYZko24MU2/Y4r4g8wAhJ7t8ULh8wITZo5ZwjimSy3sOfp+rzldU9nKQVTLCwOT/qQNXfyhrWZYctYcrObaYEOhsrAYz3b2/axw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1VLSBXEHw9Z9IXuQW5DktTaYds/UsYLR7xUfcY8F4Ns=; b=NTYuyBzzkKmeNVEGN+n95a64bc5U5yo12uGHnEaGh4/Ymgud7thJ+1WbagrYV91lB9OLmuvpvpgHBcNylpdvq5z5c3GeAHE6bzcseFV8T22Cl3CLWDl05N9ZjwRsCFhYqPVPhPBXa8JH4i9SAmHF8n0j143C0C5WhaPKcVgp2lY= Received: from CO1PR10MB4468.namprd10.prod.outlook.com (2603:10b6:303:6c::24) by BN8PR10MB3538.namprd10.prod.outlook.com (2603:10b6:408:b3::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4975.17; Mon, 14 Feb 2022 12:03:37 +0000 Received: from CO1PR10MB4468.namprd10.prod.outlook.com ([fe80::a5d1:ed4:5ab6:e9b1]) by CO1PR10MB4468.namprd10.prod.outlook.com ([fe80::a5d1:ed4:5ab6:e9b1%4]) with mapi id 15.20.4975.019; Mon, 14 Feb 2022 12:03:37 +0000 From: Imran Khan To: tj@kernel.org, gregkh@linuxfoundation.org Cc: linux-kernel@vger.kernel.org Subject: [PATCH v6 1/7] kernfs: Introduce hashed mutexes to replace global kernfs_open_file_mutex. Date: Mon, 14 Feb 2022 23:03:16 +1100 Message-Id: <20220214120322.2402628-2-imran.f.khan@oracle.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220214120322.2402628-1-imran.f.khan@oracle.com> References: <20220214120322.2402628-1-imran.f.khan@oracle.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SY2PR01CA0016.ausprd01.prod.outlook.com (2603:10c6:1:14::28) To CO1PR10MB4468.namprd10.prod.outlook.com (2603:10b6:303:6c::24) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 54b24421-f8be-4d01-1506-08d9efb2082b X-MS-TrafficTypeDiagnostic: BN8PR10MB3538:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8882; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: stkj8wPVJ0eprKc/Ec/uQhz+17mICD/3+xDC0dQm9VWaDm7J2RzND0IwJ+bE2f9B0Lf9Z9jHcVWenh9RXGRliVvpt+axGqW1Xl85s5tcvgreQ328Gw0XdWBkkLu0e6fpIQfqwffQSAlHj817I0hE3M5SHwn0M9r/Rveqm2SizNkdRy37Qf7W9Z8VRcHbL/baJkkOf1eawF+JU/L6z8zml/8KZF1A/RmBkMAXONQvCm69/78hDwhpsbUvYk458yyK2Z30PjW8W9KogCnORUquDMAUGlpEDhlN4Iu/+kKoR5FftQ0ZDgj76OHaaYtUoSu1R7rmpanUd1+p4FlgSDU8MxhxkCrtOpSWZ4of/sT15orGgOZf1Sqrg/qKPhD9PcGdxBd5fXhdnLqgzD8Dhu8d8yeuzFPnG6uNE4I7RPlG9eWeauXBzPVuK7qjDrdMddeNAn7WOL3bySKGwTWBh68Xrc7MvduCmxFGS+JPl9+5grPn0f7+kSpj8XrYp7tIBNME+qfUayodsndhkgLSgsKLhVhY9XI/oSSwEC2bQjCv/VpfJOYcg+7dJpg3g8aYiit8w7VRbc9rqKrYvv571Wx+hzFdLbvEbVq0fYVTq80iLjNyhQWBzoLdCAHxXY0dg9tvaJ/SJCmNXkuYDLex/QPO6JnHsMx3xVyZ/0aPDu4acHv2cc/fp7v9ygycv2TbCdkR2ZecoNgBwLxbBazEGEUcPw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO1PR10MB4468.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(38350700002)(66946007)(316002)(38100700002)(2616005)(36756003)(6486002)(66556008)(4326008)(66476007)(103116003)(186003)(1076003)(26005)(6666004)(6512007)(2906002)(86362001)(508600001)(52116002)(83380400001)(5660300002)(8936002)(6506007)(8676002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?digoBS3M0Kxjyo117ya9S9FDhCT+jmP1m5FR40dswl+EC3nDPc04bdCi4W/D?= =?us-ascii?Q?62+jvyjZaQwm2UcLwzxeeN8jPnLlOSiCKQphayFfd3Nh74lZI/dZ6zOzZ0dA?= =?us-ascii?Q?IUC/Zs/aY8wbOJDE9d9MMArTGa164KKF9ow5OgvKO0H7658D2zOeoOgFZOHz?= =?us-ascii?Q?jTVTkbyoOSLgjEO7NPu6s8zTmThFGkM759j6LRd535QqYP4mJEergo5arvKW?= =?us-ascii?Q?qzkxnLw0EDqFdpYXTqMrvG0cmX1LGuGoHX6+2LLz1rHj9Q1drt/jzDr7ciXg?= =?us-ascii?Q?L+Gadc7mRJPUAbdJ99SbybilarxNHkLbD5HHWypifgaP8VZaZEvN1cAWhCbr?= =?us-ascii?Q?MYKwcndaisLyMl4HsEKIZi9QPmU0NOp+Y4a7Y8p/CU1Ej9dO4US7wPfYhk/U?= =?us-ascii?Q?oeaStaOXz1helBz89qp8H203W0v6WqogAmvWcJ2yEGFfa+LzdSTtAfX+BJiu?= =?us-ascii?Q?TMyMjvRSZIVzBItFx+XqIKJ7xoHDpLPNYEyPXy4njuaagBt+BPL+E9p6f2EN?= =?us-ascii?Q?1dgrysjYL3RQHOQT6IdySTQOobFWGx65ar070pT6SUmwH1d40ukiNsDJ7GO0?= =?us-ascii?Q?CY/VxBagRt2uU1ui7sXoWuukY1A4U0Usbt/uJGhR0V5NsR1o/OBUFfl4rC/Q?= =?us-ascii?Q?lNmGS6ROlqkc3gF/zNHXO3wnMNYYAX4g7IUTzTX+e7G1O/JGfUo8f4O2m6C2?= =?us-ascii?Q?YKu5USTcoO725azWr3rbjuNtR5Dkdp1Bb5UCtU+2Y5MsBlQj9SDP4L0QZWG0?= =?us-ascii?Q?oF9u2Hp2zYJVCyg6IHEH3sRdAOVbNAq0WH6tBLuAbV6gJpmz+VK04uUjFLef?= =?us-ascii?Q?IZqnk4YcOucsmgO9WgpmBvC7KBayf6/xuk27ZwKFWtX18s4uneOuenrzXo0X?= =?us-ascii?Q?Ck6T5NpvE/K2LZXj/cmP5VNVcsaJOlze5RkOkxjJOhDUmOVzTPeFX8d/To7C?= =?us-ascii?Q?FMblb7Zx+EqozSx9xNmNcHSAMSew4hm0NbnQFU0sGwsscZBMT3zuhsIKiQvP?= =?us-ascii?Q?c0PnuI3f8crBroqZuOgdukBO0xDo86jtSBvyheh7FCa848z3fJ6A4R6rUxU9?= =?us-ascii?Q?yLHrPmBgJOH1vH+1F8hNt/4FBYlECgoyQ03wYy1SKUEPM/jYnU7sxF0xIMph?= =?us-ascii?Q?k57NAgReW/azyOhZSjr7IZTDvV7GIK8N4mq3TW0NBcT19TI6TQOQGj+nxnbK?= =?us-ascii?Q?YuJT91HtL0o7vZLJiMcpXs3aheMEeIPFxz6iLihUTCpBVG3wWCsUjFyEC4VH?= =?us-ascii?Q?GVA/QOyfu9r6A/YvWeGpZbaydtFBsofv+sbTTCS97i4XgH4zi+NIG7RMzXdg?= =?us-ascii?Q?4w9mDypof8xln0JG19K/iOhJar4dbZbU/JJMtF7iwbd717L/LgtfzzGw+P76?= =?us-ascii?Q?BPjjNzL4gSRxMQ6Rzd251rFtdR1i//kYQSdcS3nD09NRWK8lOjjtv8JI9lI6?= =?us-ascii?Q?AEM1U8nO+fC3MZ8NHme9Go8/J7BtdyvkaCK/P2rRf8f0T5sxfEbUZrVTEwTK?= =?us-ascii?Q?RMH02b5SkLVRkyLFD3CavE5saKGcN5ux70O/03xMgtc59vGwv3vZLb6D3EOP?= =?us-ascii?Q?36OqjCBQpucuKhbCrpy07za4D6qeqf30hGXiLmW36Ks4EsOT1T5C1FOKgKV2?= =?us-ascii?Q?h2T1DxsE1Gjs9XdbhmTCteA=3D?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 54b24421-f8be-4d01-1506-08d9efb2082b X-MS-Exchange-CrossTenant-AuthSource: CO1PR10MB4468.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Feb 2022 12:03:37.0830 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Bjz5SiKA0j1EC9hpZBkyK2ix9qWSaGMW6h/QmFlPWyQCyCc58E9zI18NV6ZiBqLpm/etSrKRfNHtMSeSzH8Wnw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR10MB3538 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10257 signatures=673431 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 phishscore=0 bulkscore=0 malwarescore=0 adultscore=0 spamscore=0 mlxlogscore=999 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2201110000 definitions=main-2202140073 X-Proofpoint-GUID: bbt1JBy03B7JldXdFyPzp9-WgvuUShKA X-Proofpoint-ORIG-GUID: bbt1JBy03B7JldXdFyPzp9-WgvuUShKA Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" In current kernfs design a single mutex, kernfs_open_file_mutex, protects the list of kernfs_open_file instances corresponding to a sysfs attribute. So even if different tasks are opening or closing different sysfs files they can contend on osq_lock of this mutex. The contention is more apparent in large scale systems with few hundred CPUs where most of the CPUs have running tasks that are opening, accessing or closing sysfs files at any point of time. Using hashed mutexes in place of a single global mutex, can significantly reduce contention around global mutex and hence can provide better scalability. Moreover as these hashed mutexes are not part of kernfs_node objects we will not see any singnificant change in memory utilization of kernfs based file systems like sysfs, cgroupfs etc. This patch introduces hashed mutexes that can be used in place of above mentioned global mutex. It also provides interfaces needed to use hashed mutexes. The next patch makes use of these interfaces and replaces global mutex with hashed ones. Signed-off-by: Imran Khan --- fs/kernfs/kernfs-internal.h | 23 +++++++++++++++ fs/kernfs/mount.c | 13 ++++++++ include/linux/kernfs.h | 59 +++++++++++++++++++++++++++++++++++++ 3 files changed, 95 insertions(+) diff --git a/fs/kernfs/kernfs-internal.h b/fs/kernfs/kernfs-internal.h index f9cc912c31e1b..03e983953eda4 100644 --- a/fs/kernfs/kernfs-internal.h +++ b/fs/kernfs/kernfs-internal.h @@ -147,4 +147,27 @@ void kernfs_drain_open_files(struct kernfs_node *kn); */ extern const struct inode_operations kernfs_symlink_iops; =20 +/* + * kernfs locks + */ +extern struct kernfs_global_locks *kernfs_locks; + +static inline struct mutex *kernfs_open_file_mutex_ptr(struct kernfs_node = *kn) +{ + int idx =3D hash_ptr(kn, NR_KERNFS_LOCK_BITS); + + return &kernfs_locks->open_file_mutex[idx].lock; +} + +static inline struct mutex *kernfs_open_file_mutex_lock(struct kernfs_node= *kn) +{ + struct mutex *lock; + + lock =3D kernfs_open_file_mutex_ptr(kn); + + mutex_lock(lock); + + return lock; +} + #endif /* __KERNFS_INTERNAL_H */ diff --git a/fs/kernfs/mount.c b/fs/kernfs/mount.c index cfa79715fc1a7..fa3fa22c95b21 100644 --- a/fs/kernfs/mount.c +++ b/fs/kernfs/mount.c @@ -20,6 +20,7 @@ #include "kernfs-internal.h" =20 struct kmem_cache *kernfs_node_cache, *kernfs_iattrs_cache; +struct kernfs_global_locks *kernfs_locks; =20 static int kernfs_sop_show_options(struct seq_file *sf, struct dentry *den= try) { @@ -387,6 +388,17 @@ void kernfs_kill_sb(struct super_block *sb) kfree(info); } =20 +void __init kernfs_lock_init(void) +{ + int count; + + kernfs_locks =3D kmalloc(sizeof(struct kernfs_global_locks), GFP_KERNEL); + WARN_ON(!kernfs_locks); + + for (count =3D 0; count < NR_KERNFS_LOCKS; count++) + mutex_init(&kernfs_locks->open_file_mutex[count].lock); +} + void __init kernfs_init(void) { kernfs_node_cache =3D kmem_cache_create("kernfs_node_cache", @@ -397,4 +409,5 @@ void __init kernfs_init(void) kernfs_iattrs_cache =3D kmem_cache_create("kernfs_iattrs_cache", sizeof(struct kernfs_iattrs), 0, SLAB_PANIC, NULL); + kernfs_lock_init(); } diff --git a/include/linux/kernfs.h b/include/linux/kernfs.h index 861c4f0f8a29f..3f72d38d48e31 100644 --- a/include/linux/kernfs.h +++ b/include/linux/kernfs.h @@ -18,6 +18,7 @@ #include #include #include +#include =20 struct file; struct dentry; @@ -34,6 +35,62 @@ struct kernfs_fs_context; struct kernfs_open_node; struct kernfs_iattrs; =20 +/* + * NR_KERNFS_LOCK_BITS determines size (NR_KERNFS_LOCKS) of hash + * table of locks. + * Having a small hash table would impact scalability, since + * more and more kernfs_node objects will end up using same lock + * and having a very large hash table would waste memory. + * + * At the moment size of hash table of locks is being set based on + * the number of CPUs as follows: + * + * NR_CPU NR_KERNFS_LOCK_BITS NR_KERNFS_LOCKS + * 1 1 2 + * 2-3 2 4 + * 4-7 4 16 + * 8-15 6 64 + * 16-31 8 256 + * 32 and more 10 1024 + * + * The above relation between NR_CPU and number of locks is based + * on some internal experimentation which involved booting qemu + * with different values of smp, performing some sysfs operations + * on all CPUs and observing how increase in number of locks impacts + * completion time of these sysfs operations on each CPU. + */ +#ifdef CONFIG_SMP +#define NR_KERNFS_LOCK_BITS (2 * (ilog2(NR_CPUS < 32 ? NR_CPUS : 32))) +#else +#define NR_KERNFS_LOCK_BITS 1 +#endif + +#define NR_KERNFS_LOCKS (1 << NR_KERNFS_LOCK_BITS) + +/* + * There's one kernfs_open_file for each open file and one kernfs_open_node + * for each kernfs_node with one or more open files. + * + * filp->private_data points to seq_file whose ->private points to + * kernfs_open_file. + * kernfs_open_files are chained at kernfs_open_node->files, which is + * protected by kernfs_open_file_mutex.lock. + */ + +struct kernfs_open_file_mutex { + struct mutex lock; +} ____cacheline_aligned_in_smp; + +/* + * To reduce possible contention in sysfs access, arising due to single + * locks, use an array of locks and use kernfs_node object address as + * hash keys to get the index of these locks. + */ + +struct kernfs_global_locks { + struct kernfs_open_file_mutex open_file_mutex[NR_KERNFS_LOCKS]; +}; + enum kernfs_node_type { KERNFS_DIR =3D 0x0001, KERNFS_FILE =3D 0x0002, @@ -413,6 +470,8 @@ void kernfs_kill_sb(struct super_block *sb); =20 void kernfs_init(void); =20 +void kernfs_lock_init(void); + struct kernfs_node *kernfs_find_and_get_node_by_id(struct kernfs_root *roo= t, u64 id); #else /* CONFIG_KERNFS */ --=20 2.30.2 From nobody Mon Jun 29 07:49:26 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 060C0C433EF for ; Mon, 14 Feb 2022 12:03:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352340AbiBNMEA (ORCPT ); Mon, 14 Feb 2022 07:04:00 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:33044 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352194AbiBNMDv (ORCPT ); Mon, 14 Feb 2022 07:03:51 -0500 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 049E6D9C for ; Mon, 14 Feb 2022 04:03:44 -0800 (PST) Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 21EAvAmD018836; Mon, 14 Feb 2022 12:03:42 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=K8ffF/DIITezGYp1oGJzxiKDxgW5J76IxsHozq+1X54=; b=G7WBiuD9kzoPRjkbxHCwjAgbfRo4FkPH1+oMgkN2m76RvUMtaHeTtLxxPKHNPtf4v+mU HHzuv37ocBJ4zUTtFzHjNxOMOpWs9gjkJkr38FRSJRglVIkaMvbgShLc24rE72IHhyTO iSPTknPJIK6mR/YQe4+/ly1UfiSWWm+j8F00jL/A6tgzcFjMWw7rN+b7YScHx++MvRcm MLOELTJCBk5EXmaSOaSU3L9BAGOur1hyvX8NI+heSD3OkUljVFJpVuZe4fhfTgKj3gPz HlQZh449EPAygWWI9m61tO/kUd50jjUgqyWfstlI0mUYTcGxPGfkZyDzQInrXM3l2EkB TQ== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by mx0b-00069f02.pphosted.com with ESMTP id 3e65euc5qa-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 14 Feb 2022 12:03:41 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 21EBuOEG176340; Mon, 14 Feb 2022 12:03:40 GMT Received: from nam12-bn8-obe.outbound.protection.outlook.com (mail-bn8nam12lp2169.outbound.protection.outlook.com [104.47.55.169]) by aserp3030.oracle.com with ESMTP id 3e62xd07e9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 14 Feb 2022 12:03:40 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=lEWkD+zeDOG6liEwSrGyUH2PSG/JmGzO3VGBazv3ucK3FsAlMwcgWrFBy9Uuv2y7jBqrGFileOvUhwkKXUD8qnGistphiut0BJpKRrlxgLzpm+dFp4IOCvccIofafMoAxxTcjqgo7swOgbzi57i6vjK22iFFh3fhFvF/9SXpm1AKFx8hbkbGJ6VPeolp4202LYWkJ/GfAocNcp/0439sUkRaIXQepGDRb/Gi8WPOycKomiTuA1L6IHoikk1h3D35ohsamnciZHqQLlD1naGG8rfQzsOVNwcVaExybW22yUpJwa0Szh9w2bbgDQr0SCfxY6+rUDgzlEH2N5yuKkDxAQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=K8ffF/DIITezGYp1oGJzxiKDxgW5J76IxsHozq+1X54=; b=j5uENRAj41b5sUbAqqPnwRIjMaBVgmj+L2D08NRVBVg/YB51RVqJEaOtT86yZM/pji5lq0wYiyXi7xazCa1WtP5I0jW1in/Fwr4JK+oqIuOlwSEZI/h7xqKrNXtF6hvRbuDZEzSzJt9+qwnrjJa1LBheKfr9s9ymt/7oD6elnrhhc0VGzPy+ly1Sh6i1A9SA+daVjtcxYhhFAAU5DytuNtXL5TZxnGBj00j9j9KpLr7yL0FAqUfjbN1txJsiBk8/lP7/qolvKdfasz+WgMQxE6nVcXGFfi9b2PwVX54cytEX2naQ65dtj778fQ9bUO+Sasvb0bdwpPZHueK7nkToQw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=K8ffF/DIITezGYp1oGJzxiKDxgW5J76IxsHozq+1X54=; b=J+xohrpX1O4DSIfAvnpxiEevapXwkSgc96i1PyiXeAlTMcjC7JaL9EX8Sip8iqWpcGfygQbQHq2srpUOIxqcO2hfnJQ2dJcx72YaAWAwEp8PpWw5h17HFhTraiK4cc8cEHkPW4TNdDLQ1QMH6+HRp/r3ysD/xOYVkEpHdLCwteA= Received: from CO1PR10MB4468.namprd10.prod.outlook.com (2603:10b6:303:6c::24) by BN8PR10MB3538.namprd10.prod.outlook.com (2603:10b6:408:b3::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4975.17; Mon, 14 Feb 2022 12:03:38 +0000 Received: from CO1PR10MB4468.namprd10.prod.outlook.com ([fe80::a5d1:ed4:5ab6:e9b1]) by CO1PR10MB4468.namprd10.prod.outlook.com ([fe80::a5d1:ed4:5ab6:e9b1%4]) with mapi id 15.20.4975.019; Mon, 14 Feb 2022 12:03:38 +0000 From: Imran Khan To: tj@kernel.org, gregkh@linuxfoundation.org Cc: linux-kernel@vger.kernel.org Subject: [PATCH v6 2/7] kernfs: Replace global kernfs_open_file_mutex with hashed mutexes. Date: Mon, 14 Feb 2022 23:03:17 +1100 Message-Id: <20220214120322.2402628-3-imran.f.khan@oracle.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220214120322.2402628-1-imran.f.khan@oracle.com> References: <20220214120322.2402628-1-imran.f.khan@oracle.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SY2PR01CA0016.ausprd01.prod.outlook.com (2603:10c6:1:14::28) To CO1PR10MB4468.namprd10.prod.outlook.com (2603:10b6:303:6c::24) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 55acc43b-b1af-4ef2-b79d-08d9efb20914 X-MS-TrafficTypeDiagnostic: BN8PR10MB3538:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4941; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: dTlQx1oQI7nldJh53d48/6WavUwSh0ZlV2jQBkLXkM5qII37D3LRPoMJcWzduCvxBtnLSSBHbsn6+81+JaDBdlSn1MuEJ+3rByQofKClZfxVkkbIkSpKM/u5p/6ytAe1Wndr4nqAVeKPhlNE/gDqUa/IRbUqKHZs1lqohao3WSuJ49BOOKvz6E/BQ2Ef4bWSYoMclsh7b3jLyEhMyJlGXO2vXZdQwzYNlaOpqf8Kz+Ds2LMDHvoEQIXeCVw1D0ycTdvh7k+OHSnFqufWvGweN71q0il1nQN7OnfdZccwZXkSPq+7OTojbHua94jyZhayf6nu/IPoie9LNsNSAZBpMSB+vduVWBOItVvZT+AWnALhLpTyIdwUOvO3bv1iSPS4MSc8LOike0nhJfZ5Oq3FgR9tMtXzxmJKfpDLAUO1q8XoHQgPnJrONMVqSjyIftQg+TQAvwFYUpcwlA5ps4OtUCYi2skQe6S9fiy2d1mtbWvQAZ67TjJYng4UUP3BbDd4S8joGl7vgrIyC73wTPwHNswokSZIAh/PN4aYGOPtS6hlqW/b82TIj26d4pp+h1Ev+g0S6/hdlwb0AWVTFXYQgTtBUdhYN2V4jh0eaGEuNRHuIgDt32SzbA2JDj9BrHwAF08HEPFRL+6745z7fGl8JZWsub60C2nxLEjC43Scfcdx9SDQCUczFWYWvxlbl1gSHLoQaZaM9uoNZQGG9qyWzA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO1PR10MB4468.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(38350700002)(66946007)(316002)(38100700002)(2616005)(36756003)(6486002)(66556008)(4326008)(66476007)(103116003)(186003)(1076003)(26005)(6666004)(6512007)(2906002)(86362001)(508600001)(52116002)(83380400001)(5660300002)(8936002)(6506007)(8676002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?6jl0mws6S6A6vibLxrQXgd64U1BiyfVIthlnVNDeMBDZvNf382ZW7/YQ+kng?= =?us-ascii?Q?4HNiYAwgRzzh5h5uHyxzEM/QCux5qP9RAr3BYG1NaPFA+WZ9Y6z499Kyi9be?= =?us-ascii?Q?6FOpqiAcTflj0Y0khTU6uJbYxao0s8n0m8pqBJic5JbEspv7U99QKUKgDk/7?= =?us-ascii?Q?wg1poOpiOVktORn+jq+7Bzc9RjImjx0iZOMf4TnTKcfk7rlaJ6TI5OQJ9ax1?= =?us-ascii?Q?RR3Jjb8Evk3MQxMMnz7g0DYNQV2/6U35iP19a44i3eEs8eYU4DAQ52pFJxCV?= =?us-ascii?Q?bBIl8nEQ3Ts2qBEE5JoERkQmyjTS/19+bE63reeNc8KUWSsDbvxIaHq5XPyI?= =?us-ascii?Q?2o2o/IZabHPcknL8Y+98gZ5eZxrZJ/BOROb8KhKoUqgGkEzNfoV2+GTm5Lls?= =?us-ascii?Q?o5EUKH94pBH7jdF392IaAVhIKmCgdwhgapfGIFIMRN/3tAyMsdVo0Qm1tvJv?= =?us-ascii?Q?VBaq3yNVL0+tDmh9qCdm1eXYhNAlqC2gl2ECmGNO8xT/X7Zyt/z2/K/2cVWf?= =?us-ascii?Q?DGLhi9fzmlDy9uqS7vnV3QnsqQgAoqw4y9M9DnuQ7pJ9BcqChjwzZiJG/BMy?= =?us-ascii?Q?hrln55Pk0/aQsOD7VA6yR0HA3W056fruM+V/rgBim3rv8t+Td/6p/aDenHIc?= =?us-ascii?Q?jVSQbtHI3boUIEGjsg5Wa+z8HouNxmDAyCxOH23qnY2dMcoDWGUUlQ45WTiZ?= =?us-ascii?Q?rvIKqba3PtXMufLg5O2SXHBf7XopITZhgUk0gLSDvjgEwG9FzoU+CWY333qA?= =?us-ascii?Q?jfT43jShYPXVdV2OJ1vftX9tII8UCDo44ZGtacBbDY1P939ZTDO1x+IeEeF1?= =?us-ascii?Q?lDjLujoNGgCVjawvIhmWxg7/qDlnDAmyNuw+XgtKDCKmpc1s98hxg5gcqFje?= =?us-ascii?Q?Fy/GYqg3as03LOZ8XadQzy9H52w/IeQUo1Xm1eQ6ZSuOkYSCIySjmp6Ygg0U?= =?us-ascii?Q?3Vj55x6W/XIbMgYgjsJqq/LFx6ybLzJA2W/woDzSNa6OX9QVe8gHEeogoH8l?= =?us-ascii?Q?DarvGsJR/+4EeLUUj79pujmyE2I4bmVLkN3mKWAMtQrhHaJPy/DfWHJcixyU?= =?us-ascii?Q?JZP550fN6c5QQrUZLGLlCGVtOJBc2gyloqGRZiDfQNVAoo/tkX3I3f+HumuP?= =?us-ascii?Q?Ph3Y9CCr5inccMhwRlyH4lxa827RxDnbICetqi0ETvxsMn9e24WJBbZ+3mxQ?= =?us-ascii?Q?CLfttW8caC3oOckmVzrOHvz71+THBI4rDuhIv4iIoEGTO8ySwZa+g+1zQ/5c?= =?us-ascii?Q?rINZzPiWEdT2FXg4dxDXrV8PV1H1bgUNFh54h6dLu2MD3lgnKcA7v5M+w3w0?= =?us-ascii?Q?P+3YcW3xFqRtrbQZf/LUepIk+x8ctv5G9RoiIITqpwwqt9sLErRph9Ji9DEv?= =?us-ascii?Q?2+xqhqe7xvEY4trRwwTVaBVDjXdYhWJWgv7pManFgyVbyzqA6sNXEuOdSDQK?= =?us-ascii?Q?8emS1AA1HJyJ/MEj/fWRHYIK9X55OhnCTDIHe0tT15z1TpFKca7cpl49vPqK?= =?us-ascii?Q?7cQTfzdb1PSffem/9HWFxjDn+U1GE3NAAslSQFa/HLnsexWJYporoAOI1uEL?= =?us-ascii?Q?9uvQYihuqSrfDdIlt2AxevIJb0tie5/HHxHnjsIUo0ilydmvh9UH/SoLjLu2?= =?us-ascii?Q?rSzRvJDQuifjXksVVoCVSMc=3D?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 55acc43b-b1af-4ef2-b79d-08d9efb20914 X-MS-Exchange-CrossTenant-AuthSource: CO1PR10MB4468.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Feb 2022 12:03:38.6456 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 4PrsEBuMcYAbpbyxGJMmbS9W32BTLsS97uY8Hc2ysIIoKHWoPvlvyOGvVT3Iat5431ujGiMKqI/sXbgkPE6W7Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR10MB3538 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10257 signatures=673431 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 phishscore=0 bulkscore=0 malwarescore=0 adultscore=0 spamscore=0 mlxlogscore=999 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2201110000 definitions=main-2202140073 X-Proofpoint-ORIG-GUID: ISfrRqktHsDMElRr1jwyCxWbj9aGIEDc X-Proofpoint-GUID: ISfrRqktHsDMElRr1jwyCxWbj9aGIEDc Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Remove global kernfs_open_file_mutex, using hashed mutex and corresponding interface introduced in previous patch. Signed-off-by: Imran Khan --- fs/kernfs/file.c | 31 +++++++++++++++---------------- 1 file changed, 15 insertions(+), 16 deletions(-) diff --git a/fs/kernfs/file.c b/fs/kernfs/file.c index 9414a7a60a9f4..295fe67950346 100644 --- a/fs/kernfs/file.c +++ b/fs/kernfs/file.c @@ -19,18 +19,13 @@ #include "kernfs-internal.h" =20 /* - * There's one kernfs_open_file for each open file and one kernfs_open_node - * for each kernfs_node with one or more open files. - * * kernfs_node->attr.open points to kernfs_open_node. attr.open is * protected by kernfs_open_node_lock. * * filp->private_data points to seq_file whose ->private points to - * kernfs_open_file. kernfs_open_files are chained at - * kernfs_open_node->files, which is protected by kernfs_open_file_mutex. + * kernfs_open_file. */ static DEFINE_SPINLOCK(kernfs_open_node_lock); -static DEFINE_MUTEX(kernfs_open_file_mutex); =20 struct kernfs_open_node { atomic_t refcnt; @@ -524,9 +519,10 @@ static int kernfs_get_open_node(struct kernfs_node *kn, struct kernfs_open_file *of) { struct kernfs_open_node *on, *new_on =3D NULL; + struct mutex *mutex =3D NULL; =20 retry: - mutex_lock(&kernfs_open_file_mutex); + mutex =3D kernfs_open_file_mutex_lock(kn); spin_lock_irq(&kernfs_open_node_lock); =20 if (!kn->attr.open && new_on) { @@ -541,7 +537,7 @@ static int kernfs_get_open_node(struct kernfs_node *kn, } =20 spin_unlock_irq(&kernfs_open_node_lock); - mutex_unlock(&kernfs_open_file_mutex); + mutex_unlock(mutex); =20 if (on) { kfree(new_on); @@ -575,9 +571,10 @@ static void kernfs_put_open_node(struct kernfs_node *k= n, struct kernfs_open_file *of) { struct kernfs_open_node *on =3D kn->attr.open; + struct mutex *mutex =3D NULL; unsigned long flags; =20 - mutex_lock(&kernfs_open_file_mutex); + mutex =3D kernfs_open_file_mutex_lock(kn); spin_lock_irqsave(&kernfs_open_node_lock, flags); =20 if (of) @@ -589,7 +586,7 @@ static void kernfs_put_open_node(struct kernfs_node *kn, on =3D NULL; =20 spin_unlock_irqrestore(&kernfs_open_node_lock, flags); - mutex_unlock(&kernfs_open_file_mutex); + mutex_unlock(mutex); =20 kfree(on); } @@ -729,11 +726,11 @@ static void kernfs_release_file(struct kernfs_node *k= n, /* * @of is guaranteed to have no other file operations in flight and * we just want to synchronize release and drain paths. - * @kernfs_open_file_mutex is enough. @of->mutex can't be used + * @open_file_mutex is enough. @of->mutex can't be used * here because drain path may be called from places which can * cause circular dependency. */ - lockdep_assert_held(&kernfs_open_file_mutex); + lockdep_assert_held(kernfs_open_file_mutex_ptr(kn)); =20 if (!of->released) { /* @@ -750,11 +747,12 @@ static int kernfs_fop_release(struct inode *inode, st= ruct file *filp) { struct kernfs_node *kn =3D inode->i_private; struct kernfs_open_file *of =3D kernfs_of(filp); + struct mutex *lock =3D NULL; =20 if (kn->flags & KERNFS_HAS_RELEASE) { - mutex_lock(&kernfs_open_file_mutex); + lock =3D kernfs_open_file_mutex_lock(kn); kernfs_release_file(kn, of); - mutex_unlock(&kernfs_open_file_mutex); + mutex_unlock(lock); } =20 kernfs_put_open_node(kn, of); @@ -769,6 +767,7 @@ void kernfs_drain_open_files(struct kernfs_node *kn) { struct kernfs_open_node *on; struct kernfs_open_file *of; + struct mutex *mutex =3D NULL; =20 if (!(kn->flags & (KERNFS_HAS_MMAP | KERNFS_HAS_RELEASE))) return; @@ -781,7 +780,7 @@ void kernfs_drain_open_files(struct kernfs_node *kn) if (!on) return; =20 - mutex_lock(&kernfs_open_file_mutex); + mutex =3D kernfs_open_file_mutex_lock(kn); =20 list_for_each_entry(of, &on->files, list) { struct inode *inode =3D file_inode(of->file); @@ -793,7 +792,7 @@ void kernfs_drain_open_files(struct kernfs_node *kn) kernfs_release_file(kn, of); } =20 - mutex_unlock(&kernfs_open_file_mutex); + mutex_unlock(mutex); =20 kernfs_put_open_node(kn, NULL); } --=20 2.30.2 From nobody Mon Jun 29 07:49:26 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A01C6C433EF for ; Mon, 14 Feb 2022 12:04:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352356AbiBNMEE (ORCPT ); Mon, 14 Feb 2022 07:04:04 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:33126 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352297AbiBNMDy (ORCPT ); Mon, 14 Feb 2022 07:03:54 -0500 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 31CFBD9C for ; Mon, 14 Feb 2022 04:03:47 -0800 (PST) Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 21EBfExE031159; Mon, 14 Feb 2022 12:03:44 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=i72Zuk4HngpHHy83v0BKVClagpOwW0ASOJv7kdB+Xdo=; b=s1Ck9TAZ9iW1PkV66CtzFtt29xEAUS1LtlniyHhehhtIKJe/dWn0G/4IPiQGDqUvLX3C ijH4QNkM4EAafz8Sdag86ZMtgKWlrB7Lh1tcyzzxWX36R/FegK8gck1P5KZNDHetAKf8 sFdJj1rCiofGexxd7qd+qkERct4QUy0rUcAvc32dNlEb2Cx4iuOHUmcKeIy+2mAItUIE Jt/m99XmzQ7sc9cTFqJlAtt1trn3ljK3m1a+A0rbOb1jblmtZY9+K3Fjngvx9ftDS4GX TPmlcsAs3Mmqqlz5YH+fNbL3SFSxuv3G1wkSJNAoYGDOYst7jUmeU//Mk8fTbONLRtQl yA== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by mx0b-00069f02.pphosted.com with ESMTP id 3e64sbv32a-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 14 Feb 2022 12:03:44 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 21EBuM0C184636; Mon, 14 Feb 2022 12:03:42 GMT Received: from nam12-bn8-obe.outbound.protection.outlook.com (mail-bn8nam12lp2175.outbound.protection.outlook.com [104.47.55.175]) by userp3030.oracle.com with ESMTP id 3e620vusue-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 14 Feb 2022 12:03:42 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=k/qR6rZ60X57zG62xFyRv12WK0OP1M4sqX+L507IuIBccq0otR/wFU7wXTY+TBcoVeXFmB04vRDRiCAVFoTMcFrwpWNlHPN/98SpQoyJAOc62TvuxFuDmSJQCMPrGLyskEMinLMNoYVPW2AZS2IwGMNeUgybpSt6z7OXZMEXGghJs9w7JmZ+Tfh7bWxnssgJ5nZM9Rn07DyD/zqo4iKvMhGbVTJqXMK36d9t0zteRPFZcf69VY0W2W6yRDwekexiYWcKX730qMw9ZjtN4KKxmkR9rLkoA4y8jZf/HieTGxn8xj4kPpTmj/xvju0uNAb7SZ1Y0liwRvJJnzgoYMFe8A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=i72Zuk4HngpHHy83v0BKVClagpOwW0ASOJv7kdB+Xdo=; b=KIOSMSxnS30r7QgzSZUdRsNW86fgO8qUJIuA/3rMgMk0iDnIZ4LzZaKlHvFbsT7smJ5uIAV+EjRMJ07Zlje0Ke/TXy77Wed0mdVXLIIxHdZvd6tF66aEejXpV5wtJPqS8QCRrBr+fD8lDPzwyyx3MV3gckXKq+u+CJ/NDsUptyLN63LAZEGRjARpwESZMPc+OBIlWNajthKByKoISh+ghdlwkvbvhGW2OHAM3jv8jXXOBiXeKy3BFBfUQiKy2MTnYLm7J+ivb6KQmiLvzwQqHl1oBzhxrPSkoZDwmcpCaJJh2yKCnUwZXuFmubRy+uTwZFI76nNjxu2sQVmt1DsEsw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=i72Zuk4HngpHHy83v0BKVClagpOwW0ASOJv7kdB+Xdo=; b=YBIdVVWaVBl8biNpbJ8eHv22ntRSeYVWOnIec0pot0z1jgvWgPqqIn+l1vtllvq5hi6Ah149fZARL6l0hLq1dL2ILZc3L9v3p6nYVj1rG2uIgHA0l7pxksDpnAh4RHu3vEAWIe0pA8hm/o+Cg41Kz/+Ti2yiXYVs2TriTtKWVyQ= Received: from CO1PR10MB4468.namprd10.prod.outlook.com (2603:10b6:303:6c::24) by BN8PR10MB3538.namprd10.prod.outlook.com (2603:10b6:408:b3::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4975.17; Mon, 14 Feb 2022 12:03:40 +0000 Received: from CO1PR10MB4468.namprd10.prod.outlook.com ([fe80::a5d1:ed4:5ab6:e9b1]) by CO1PR10MB4468.namprd10.prod.outlook.com ([fe80::a5d1:ed4:5ab6:e9b1%4]) with mapi id 15.20.4975.019; Mon, 14 Feb 2022 12:03:40 +0000 From: Imran Khan To: tj@kernel.org, gregkh@linuxfoundation.org Cc: linux-kernel@vger.kernel.org Subject: [PATCH v6 3/7] kernfs: Introduce hashed spinlocks to replace global kernfs_open_node_lock. Date: Mon, 14 Feb 2022 23:03:18 +1100 Message-Id: <20220214120322.2402628-4-imran.f.khan@oracle.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220214120322.2402628-1-imran.f.khan@oracle.com> References: <20220214120322.2402628-1-imran.f.khan@oracle.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SY2PR01CA0016.ausprd01.prod.outlook.com (2603:10c6:1:14::28) To CO1PR10MB4468.namprd10.prod.outlook.com (2603:10b6:303:6c::24) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9a753bf1-d98f-475d-3749-08d9efb209f0 X-MS-TrafficTypeDiagnostic: BN8PR10MB3538:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:9508; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: W4bDjq0zdxHxemzMJw2VQccv6BXgOp+FwfQ9hizo+Kz5a4n0ed9bjju51Ux8iQFrFuvyRY0beMCkKArOTO95GFEXie6Np5TOWBpDGDpgPwDfqFcknwKuOcjCzKM8KzEt/dpfxBkENVPPDhY8D3N0PlvPTNBaqNEPjpl/WbJQASJX7lLtgLXJHv5CCKFIHb/p7oILyT2mz8SFZXF0Kw1y/UQZLnqiuPOC1t1RlvKq0PStzcvdUrhVfajLBS2ePOov3VFs9lZFzgydzvdpQHcI/nEIIZtWtGANxkyeHC77Cj82y/5FOYKOMsCl5JPWdBxQsCFzYfh4JtLGeEjOrw2wp+50Nou5NVisZL7ooerj+mHaUxLB5WXPERVpvVG7f36JBdXOovQYzzSlxZ7O12aKkWAbKRgVRcm10aTGSE/X+QSBy/c4iwsUqKz1itLSb2HXDN1LZPZhtdYCexDJu2kbOUOrod8oX5/2AnOJSsthsFrCfGKbJf1MeBWsd1iruqFQVNgqDqeBgdP/WQqUGUtY3acFk5J27rlVS1D3Js28ssFbZMd+E/bS1qOw5VUmGQBRkIKexZk3ZHTpEmy0enaw1f+Pe/T8cCjAHEhjUaCrOuCk40w6NvzVQNddljqN5ZiPzGi4XieZSo2EPZEg5dbF/3UvYQIew1sgf7oXD6hDzfBYCCvDtmIVigkicG7eCWebHywOWBZtnEvuzjdJl7cPmQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO1PR10MB4468.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(38350700002)(66946007)(316002)(38100700002)(2616005)(36756003)(6486002)(66556008)(4326008)(66476007)(103116003)(186003)(1076003)(26005)(6666004)(6512007)(2906002)(86362001)(508600001)(52116002)(83380400001)(5660300002)(8936002)(6506007)(8676002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?IAsOmEEFKV7/QsX9kABJqAuW0Ns5qTpJXE+GLSJcJi2ftHP+a2SuUu+Im5Ia?= =?us-ascii?Q?+DjzktnOXv33ZzUFuvJaKC6ohmnFyhEHfmfhKs6krhadg9f756pGUyhwTFcf?= =?us-ascii?Q?zHLkX4+cg+1o7hS9XKyV/2ChfGumzW3jRxW5UA3pd2aICkCZwcf+DLkzuC4f?= =?us-ascii?Q?qOSFKWeRdjLM15uDC7JTTl2c39SupOmHYCb+52nPWnh9WHepFSe73V9Hky5u?= =?us-ascii?Q?UzLXKsKOV8geYKnwdvfRyyz0XRVNsXQGwODnd+Gh5yscyi+A901CxLZfM+Av?= =?us-ascii?Q?jtWePLX09VdXSzlDKn49pPw5u8kAPSmGbWq1uQBgTUDR4GEqP3m+IxHO16nj?= =?us-ascii?Q?34YL7aLdJquIOQP9B9Pq5BX3WrBkhwYPL2TGnDR10aSLCc6oFmhYRHhADCak?= =?us-ascii?Q?cXYPSS0jwr+A7aoGJRY2A9sKtqfDif4fmsUNL3o28+2VvEb4qtG32YhP8qq7?= =?us-ascii?Q?yGrYl9w7DoJPTja0ICAm/mFHmbMUi1WsNyapoltYMY6tEJPyt6njvxP6/NKP?= =?us-ascii?Q?hgLM5dhJWV7R6dRgevK/Uq4aHY3AD69UYfrmqltiqAtzmri6ASjMxDJpJ8LG?= =?us-ascii?Q?PtyPJ0S9L/rntYcEjwAiLjT54i+3jjaiaqTVAF2fkHRiBU4OKvZwz18EwxOW?= =?us-ascii?Q?sbtJN7wvDVIrGIo9aGErhlQ8/X/gyiAeV6Hz6iRZT3wwpqXVIdeFXceiTCO5?= =?us-ascii?Q?w8TIysh+RMlNZgN52iLv+V0+UA+9YirOX6Ra99Sn1PrStvpd759hEqeJtl4c?= =?us-ascii?Q?+prNtCW5qYWzYPfcrbqZ3F9WFBICnvrsAQvXQ/GBs4zsD9e+Puxd+uP/hiet?= =?us-ascii?Q?Qnm7JPxE19jFWI7Ij8XEQSA/CMXbfZAy+uM4rf6kHOwPtbOaG9RjIi4RfOfE?= =?us-ascii?Q?5oZxVrAnKR+90V1qdL20WBGKbLhWrkulffDgXeTK7831WWrnPYPLZy/mz41S?= =?us-ascii?Q?yFxSwNsQdAOlqSlq0R8hNpHN3pSPMIHFBIv7H8BWkwN72msimyemUhPlo2LK?= =?us-ascii?Q?EqGb2dFYnR1b7B7ePzzyn3Q7Y1kEfun0dL9nc0krWuRYWT3STQjrzuaMRocm?= =?us-ascii?Q?shHDQzosQei0+LpVeG6fdVcXxNMHgTXVbzeZhy7exHcwH2KDVuhblPh0kyvR?= =?us-ascii?Q?yh6Waj6OiDzwJ+OJKySgd0xzKDZ4WK9LDltWWDiNRfH6505HtzaJzIWiIiCt?= =?us-ascii?Q?8GbQ8O8DaOQj07FRPeGIhbVwUYNEDFfvnYRnVBTpF49xBZWyePPI/hJGQqKb?= =?us-ascii?Q?NlmO2e5FLS0LANGvX4N2ElJFvBa5JYYyP2KVnbR5ii9KOoaERkfxNxWKlMl6?= =?us-ascii?Q?HXnxpC0LiN/SzXE6P/ew+6fMqFii0KC24v4C4BxFAiNr4z5tuL5v/fN7yxfD?= =?us-ascii?Q?sflmrjvq5rWVlC4DqWqQ2VGm1+MPzgFG67wUdk+Cl+nMi69gFkgY9TfaEkGU?= =?us-ascii?Q?94PmmKWNID+ZQFxjTrDXAgxzlBc2BEBEHvLv0gyBuF2amrtaBejVnrR3P2sU?= =?us-ascii?Q?s4k8HMvX0bFCpxedRAgnIpRllXiO/EQDCSRAoGmEZ9HCYpVsaG0MY0nZs+/x?= =?us-ascii?Q?wt06T4qrT7eUZYyHE0cPPQFlIqSkZb5QIE5P3ZwzuRBI/8ZwTu7ZAlbXTzzp?= =?us-ascii?Q?RPiSGeBZUawYKisobj6hhYY=3D?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9a753bf1-d98f-475d-3749-08d9efb209f0 X-MS-Exchange-CrossTenant-AuthSource: CO1PR10MB4468.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Feb 2022 12:03:40.0840 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: gS4vAMKIZPqHgYftqhVxizp5IDD3ICt54M7Ie3t0wp3CPMHQ11jbG1UabKka7yW34zyK9KBvIGI3/Zr0gxIwlQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR10MB3538 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10257 signatures=673431 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 mlxlogscore=999 bulkscore=0 spamscore=0 adultscore=0 mlxscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2201110000 definitions=main-2202140073 X-Proofpoint-GUID: NEBi6_SOzdKV8jckaolFRbUMMHo3s76e X-Proofpoint-ORIG-GUID: NEBi6_SOzdKV8jckaolFRbUMMHo3s76e Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" In current kernfs design a single spinlock, kernfs_open_node_lock, protects the kernfs_node->attr.open i.e kernfs_open_node instances corresponding to a sysfs attribute. So even if different tasks are opening or closing different sysfs files they can contend on this spinlock. The contention is more apparent in large scale systems with few hundred CPUs where most of the CPUs have running tasks that are opening, accessing or closing sysfs files at any point of time. Using hashed spinlocks in place of a single global spinlock, can significantly reduce contention around global spinlock and hence provide better scalability. Moreover as these hashed spinlocks are not part of kernfs_node objects we will not see any singnificant change in memory utilization of kernfs based file systems like sysfs, cgroupfs etc. This patch introduces hashed spinlocks that can be used in place of above mentioned global spinlock. It also provides interfaces needed to use hashed spinlocks. The next patch makes use of these interfaces and replaces global spinlock with hashed ones. Signed-off-by: Imran Khan --- fs/kernfs/kernfs-internal.h | 20 ++++++++++++++++++++ fs/kernfs/mount.c | 4 +++- include/linux/kernfs.h | 11 +++++++++-- 3 files changed, 32 insertions(+), 3 deletions(-) diff --git a/fs/kernfs/kernfs-internal.h b/fs/kernfs/kernfs-internal.h index 03e983953eda4..593395f325a18 100644 --- a/fs/kernfs/kernfs-internal.h +++ b/fs/kernfs/kernfs-internal.h @@ -170,4 +170,24 @@ static inline struct mutex *kernfs_open_file_mutex_loc= k(struct kernfs_node *kn) return lock; } =20 +static inline spinlock_t * +kernfs_open_node_spinlock_ptr(struct kernfs_node *kn) +{ + int idx =3D hash_ptr(kn, NR_KERNFS_LOCK_BITS); + + return &kernfs_locks->open_node_locks[idx].lock; +} + +static inline spinlock_t * +kernfs_open_node_spinlock(struct kernfs_node *kn) +{ + spinlock_t *lock; + + lock =3D kernfs_open_node_spinlock_ptr(kn); + + spin_lock_irq(lock); + + return lock; +} + #endif /* __KERNFS_INTERNAL_H */ diff --git a/fs/kernfs/mount.c b/fs/kernfs/mount.c index fa3fa22c95b21..809b738739b18 100644 --- a/fs/kernfs/mount.c +++ b/fs/kernfs/mount.c @@ -395,8 +395,10 @@ void __init kernfs_lock_init(void) kernfs_locks =3D kmalloc(sizeof(struct kernfs_global_locks), GFP_KERNEL); WARN_ON(!kernfs_locks); =20 - for (count =3D 0; count < NR_KERNFS_LOCKS; count++) + for (count =3D 0; count < NR_KERNFS_LOCKS; count++) { mutex_init(&kernfs_locks->open_file_mutex[count].lock); + spin_lock_init(&kernfs_locks->open_node_locks[count].lock); + } } =20 void __init kernfs_init(void) diff --git a/include/linux/kernfs.h b/include/linux/kernfs.h index 3f72d38d48e31..7ee0595b315a2 100644 --- a/include/linux/kernfs.h +++ b/include/linux/kernfs.h @@ -19,6 +19,7 @@ #include #include #include +#include =20 struct file; struct dentry; @@ -75,20 +76,26 @@ struct kernfs_iattrs; * kernfs_open_file. * kernfs_open_files are chained at kernfs_open_node->files, which is * protected by kernfs_open_file_mutex.lock. + * + * kernfs_node->attr.open points to kernfs_open_node. attr.open is + * protected by kernfs_open_node_lock.lock. */ - struct kernfs_open_file_mutex { struct mutex lock; } ____cacheline_aligned_in_smp; =20 +struct kernfs_open_node_lock { + spinlock_t lock; +} ____cacheline_aligned_in_smp; + /* * To reduce possible contention in sysfs access, arising due to single * locks, use an array of locks and use kernfs_node object address as * hash keys to get the index of these locks. */ - struct kernfs_global_locks { struct kernfs_open_file_mutex open_file_mutex[NR_KERNFS_LOCKS]; + struct kernfs_open_node_lock open_node_locks[NR_KERNFS_LOCKS]; }; =20 enum kernfs_node_type { --=20 2.30.2 From nobody Mon Jun 29 07:49:26 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A62D9C433EF for ; Mon, 14 Feb 2022 12:04:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352381AbiBNMET (ORCPT ); Mon, 14 Feb 2022 07:04:19 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:33134 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352250AbiBNMDz (ORCPT ); Mon, 14 Feb 2022 07:03:55 -0500 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7C6ABE3D for ; Mon, 14 Feb 2022 04:03:47 -0800 (PST) Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 21EBeNu5031188; Mon, 14 Feb 2022 12:03:45 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=mqUfa01y43OuC0nj75CxH9QN9ownWcUaB3x8mkvcYTQ=; b=qACH3jcVRxToCC9DLvCWs2Cad2t7RyDc78qj9igCpoW+zJy4Co0AoHeJaqf2DHTPL81d cFAvJV0FXimN4z+CuIMNs8fojGeBXO77uMbuBjP9PWrqp1ATg/SEqgKukINopRNFkD7o IwYAgvYBVgQaAtjz9mZ/6ZrxMAErUmXsrlTC/u+eTkpv41e/StfmUl91j+734N43KdHN lXpXYVeqp1FDGVu3AAu16NPpgVKRgy2OCIGB+1r/P/dPF3z2wWtq5fve/C3t/2Qxp9TY 7VOtLZeGvvI6dYaLWDNTmPIf/vGZ9zuCs6rItv3/VFj4Vf6eh8CFbelA9gUc0H8++4VP xg== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by mx0b-00069f02.pphosted.com with ESMTP id 3e64sbv32b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 14 Feb 2022 12:03:44 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 21EBuM0E184636; Mon, 14 Feb 2022 12:03:43 GMT Received: from nam12-bn8-obe.outbound.protection.outlook.com (mail-bn8nam12lp2175.outbound.protection.outlook.com [104.47.55.175]) by userp3030.oracle.com with ESMTP id 3e620vusue-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 14 Feb 2022 12:03:43 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=h34lCqLgVGT2aq3UQlFUA72vZNJYLxgVeztlY5y3e3Y/5WLREtxZXzjTo0PR6TlJ0ODjU2PHqbEXCKe2VT54B56f00Kcdtr+VLOQy88AwU5emGsYwLK/W9lWFbqe7I7iCd2NXO0noyzm7sNAnWU4sKCGbCgPyB7EbmPm4jMktWO+Ho+4RP0XEyqQ82AXAnKk1ZgsMUhxa8zgd0+20tGprekkTwXe2VzveKhkjLBvg3Bq1TYQE4zgj/QK969a8f/UUfZN+1vD25ECk5E3NANa+H+BHvYi+u4c0tDr1/n8ziQvb5KSbcH95JvyFuelbE1ce2B+mckdRgtSy+VTSisPMA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=mqUfa01y43OuC0nj75CxH9QN9ownWcUaB3x8mkvcYTQ=; b=n4XRNtjx7qJxFgvQpyjmomoft0/Wnf5jhGvRCT4zeMsfuZv3Gfid7yf6IzHEU2Zwt0nG/OKiH1IFD0nc5Q74MigpiZy0EOnvZ2WS8A7+qyDUQEanWcl+RWfvFjgKs6/NPWXuly05ih7tI4twAt12bvJMdYeztgfOJfowrMHBy92Ys/w7+KCzTsBmc8motkEdtQMJV6v9PhEyprPj9G3lC3FWZrymwYWapDnz0LMuSHkaM+1P5wnhVdUlcxs6PETU3ccJ1F+aD1Mj0J+sAmE6vHh5qWNiC26D8qt/25oitT/L8b2y9hJHYtVLk/thAppxs1CDUZIW61enKsHvBsq9jg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mqUfa01y43OuC0nj75CxH9QN9ownWcUaB3x8mkvcYTQ=; b=ZRT9E18TUUMfBg1WcT8pXW9TCh58Yhmq1q+7ou/49UC2QEAoPwDDNb5G6dqpiuNIlH+fW4VHB2t/qx1t7KaVRGCcfZgi01AOq5VVNOeos34HCEA3yj4/Iq7zH2mMiKt2yVr5vhFlEtWUonpwgl9gtVRFcy0aok83tEirYG5PvI8= Received: from CO1PR10MB4468.namprd10.prod.outlook.com (2603:10b6:303:6c::24) by BN8PR10MB3538.namprd10.prod.outlook.com (2603:10b6:408:b3::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4975.17; Mon, 14 Feb 2022 12:03:41 +0000 Received: from CO1PR10MB4468.namprd10.prod.outlook.com ([fe80::a5d1:ed4:5ab6:e9b1]) by CO1PR10MB4468.namprd10.prod.outlook.com ([fe80::a5d1:ed4:5ab6:e9b1%4]) with mapi id 15.20.4975.019; Mon, 14 Feb 2022 12:03:41 +0000 From: Imran Khan To: tj@kernel.org, gregkh@linuxfoundation.org Cc: linux-kernel@vger.kernel.org Subject: [PATCH v6 4/7] kernfs: Replace global kernfs_open_node_lock with hashed spinlocks. Date: Mon, 14 Feb 2022 23:03:19 +1100 Message-Id: <20220214120322.2402628-5-imran.f.khan@oracle.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220214120322.2402628-1-imran.f.khan@oracle.com> References: <20220214120322.2402628-1-imran.f.khan@oracle.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SY2PR01CA0016.ausprd01.prod.outlook.com (2603:10c6:1:14::28) To CO1PR10MB4468.namprd10.prod.outlook.com (2603:10b6:303:6c::24) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 45f865e2-51c3-44cf-b134-08d9efb20aea X-MS-TrafficTypeDiagnostic: BN8PR10MB3538:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2331; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 5OHqDIaw9kTd/qOi3ZhA93gPIJpCxfcCsrb8a67prMc0bnhG6K/Av+OrLMNCXbuIgAPUA/bGQwnjF96vGwmESy6Wp8yXtK+QQwxFzfdgbZt+Vcpns7y+DDioOrhZopyJPYcKj5TiTSq8ldkVLyBd8i92r3b65hhLwVoNlwyl0RXPoPrPGjjrZrC7mfHQh4ztJypNUGU5HbYVyyaUCBuC+kH6i2QUwM2iSTiZauQ8B0fx/YR99Su36vcJzZyvlPe1zIDKd2vAd6d+kK6duBMiQUGlO8r528/0Fs5WG7OnnaLHkUtgUILQ6tb5BRFflpELsS7dIYF+tQZYviXUU3S/GpL70KzX/SR51w33GU5DG2y9oR0IfF3nHIL+mqqNKk9zXMA/xox9i0v2A1OYChqfKJYufSjVKFtEJxZZztr27aXLFQWLyI1/432kfWeGrkQsAo7MFqijszKH+Ss49Vlawd6PpNV3XT59H6xGNFQ/N86/pa8A6jhJ42WgCRBzUq/lo5gKMdW3zy78y+fPffk3x7bI1vNfEgE9fsC702fKakum7zZb+tS64Sa6DOc8l9dGY8QFzuTXCOlf2I5n7nkXxZJHnYNcxP+7m+y5QIdEcfJp2mR4tZegTBTBiwfSkyBy7rLb48UYNNVQJbQ39rnOecEkJKkddL7SjbmANRBfVXMMIhR27gmXlvhmvlB9Cf3ri4azSfGUSgvO6609OrWFKA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO1PR10MB4468.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(38350700002)(66946007)(316002)(38100700002)(2616005)(36756003)(6486002)(66556008)(4326008)(66476007)(103116003)(186003)(1076003)(26005)(6666004)(6512007)(2906002)(86362001)(508600001)(52116002)(83380400001)(5660300002)(8936002)(6506007)(8676002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?ELYyL6QeAMkTnWWj/pRJLd7cgRkzVxus9bUGAD0a+7Ah/6KpYFTyvPx2XOil?= =?us-ascii?Q?k1G9J8wb4Nxx+aO05p8H17XEif1IIEqBSr4RekeOqnviTqJRkZa0+ufVuRTo?= =?us-ascii?Q?X7fZRcmjVqWFV5nbUL97aX8kvZnB5nf9kxHVBvJdkUvkRr9FIfhgheqCoFKz?= =?us-ascii?Q?NNJhR8h5NYjQ8altC368iOLI9lePhAjii9ueFTa1xN49Y7bh4pzruwqS0StE?= =?us-ascii?Q?gsA7tScea7/jWOSd7reP6WUv7f0BDuJDvT3cBIl/hw7dbRSPpS+x4IiaHsCr?= =?us-ascii?Q?DfIr7Z6pXWWXAqH3yTfGRpwgIiCMDUv6ieMHS5WC1M4mLoX+HKIbQDiKezvt?= =?us-ascii?Q?EFls7yFp3854DCMtsETKF81FSLWu/ECm2anJPKenzfwYLxIP1pBd1tTkUxNV?= =?us-ascii?Q?n/n3LCSrbx7S/mLBdJRQgD8YpksMGjBO2AbGqKdWmtRGfJZ9uJ6fR3PUiDKi?= =?us-ascii?Q?yk1vaPW1ywHAfYpuNpTlgdTbPomABlm2M64uS5FTcklL5V4EHUT+laYuBB6D?= =?us-ascii?Q?UWoFfcIG/9Y3Vz5iKLc1c8kEVhPEEqhsoPrvbHSKjsv+ymJs3E8TgScQZ707?= =?us-ascii?Q?e4sAFmm+DYFyngEcreaK+RCiCdP/ZNgbqE0S8Ms39aj6dNp37/WLXHQYLXLS?= =?us-ascii?Q?+qII+3aStxGER28WXnKOsUH7FmNkiX7wfSvcKZEvLI1wr6uvf0YGrEw/PbI/?= =?us-ascii?Q?wZixB6MLEDMSm1TbcpRmIacvc595eqHdKr59HO7KlUBOwvPTTvrnQTjoK3Hi?= =?us-ascii?Q?RIKrRNABXLPRHv1jX7WWP/wOptC022gKVxjDhBWhLi9YroNr9DrOnMJK86Cr?= =?us-ascii?Q?cUm7qlKkQ8YvFNZNmKw4pj7YvBvy84il9UUxOz0a2r3tgj5orQly2nHzU48F?= =?us-ascii?Q?vJ0YFNoTe7Uq7VO1FcDhQOxBomtBuG5ndjTmgm53+WW1HAUom+fl89OZmuqL?= =?us-ascii?Q?3sPsT2ZeAOjbHQFUsDJ77aTU3kpN5IYzBLSdkDXDecsNcX1Q2E7I/ACFQtU2?= =?us-ascii?Q?a1fxZGs7nGgETZJmsfLTY6OErhbaYqwCJ4c3VW0zrReI0o1qDjjh4woBZrl4?= =?us-ascii?Q?8JbKzVnUef/rQKKBJG5r/oDt0BS2pZujKmFnd03qvPOyBDO1JMENc1nHGR+v?= =?us-ascii?Q?vR9qKzCvL0sMUYsA/hZDY88qNhUH7ks3y36c4/70Og4LcMwopjkAOEgLKqx8?= =?us-ascii?Q?yyAQShKg5KsuPAjH/55n07XR52C8VJL6oEVeGFlEiaP3GRge348Xho+o6rjE?= =?us-ascii?Q?DTQZUmS3wwDiQI6kU7KzqxdgporUQBibuiA8Dxf6oxQVmUzlFLcv2HgNY2YP?= =?us-ascii?Q?vW2xQGu6Em/zJaGXWzRIHjNsNfkvS+QgSdamy3hURnblAXrRTToZqEC69Q1s?= =?us-ascii?Q?2cRyHIHhaL3+iNdwiXiEC0KetsHWg/QPIC3vZM3g7Cc9KCgmbpMD7BCaVG6b?= =?us-ascii?Q?RBs3S4hfu93kDFFcOmnMXBgOpYEbmCv9u3wU3k+5NvSYp38Df5kJcq2DWvKI?= =?us-ascii?Q?vVFDXI33zDYrzP57eu0rL7x6Va9EI15l2rqs8LqzzbtsB2f/RM/Mw06QS9aM?= =?us-ascii?Q?fXcF0x4X3vybA7B7sOWw4FmwLVuD0pEr0yeCkntdPDcuLx/Fvx9LxrtonjGL?= =?us-ascii?Q?HXEPpYdUABSsdsaW7+Z642c=3D?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 45f865e2-51c3-44cf-b134-08d9efb20aea X-MS-Exchange-CrossTenant-AuthSource: CO1PR10MB4468.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Feb 2022 12:03:41.6943 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 0pRXZhOFK4M5NSyYK8SaRtArOmMsG8caRqZTK+bWoqcgnfmQc4dp+kwapzCBUIObGyEd0UX12d8J5NqL6G5FxQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR10MB3538 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10257 signatures=673431 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 mlxlogscore=999 bulkscore=0 spamscore=0 adultscore=0 mlxscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2201110000 definitions=main-2202140073 X-Proofpoint-GUID: 10eXItNUOJtT8Jj75PrWVY-7fxTOoTlV X-Proofpoint-ORIG-GUID: 10eXItNUOJtT8Jj75PrWVY-7fxTOoTlV Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Remove global kernfs_open_node_lock, using hashed spinlock and corresponding interface introduced in previous patch. Signed-off-by: Imran Khan --- fs/kernfs/file.c | 30 +++++++++++++----------------- 1 file changed, 13 insertions(+), 17 deletions(-) diff --git a/fs/kernfs/file.c b/fs/kernfs/file.c index 295fe67950346..f3ecc6fe8aedc 100644 --- a/fs/kernfs/file.c +++ b/fs/kernfs/file.c @@ -18,15 +18,6 @@ =20 #include "kernfs-internal.h" =20 -/* - * kernfs_node->attr.open points to kernfs_open_node. attr.open is - * protected by kernfs_open_node_lock. - * - * filp->private_data points to seq_file whose ->private points to - * kernfs_open_file. - */ -static DEFINE_SPINLOCK(kernfs_open_node_lock); - struct kernfs_open_node { atomic_t refcnt; atomic_t event; @@ -520,10 +511,11 @@ static int kernfs_get_open_node(struct kernfs_node *k= n, { struct kernfs_open_node *on, *new_on =3D NULL; struct mutex *mutex =3D NULL; + spinlock_t *lock =3D NULL; =20 retry: mutex =3D kernfs_open_file_mutex_lock(kn); - spin_lock_irq(&kernfs_open_node_lock); + lock =3D kernfs_open_node_spinlock(kn); =20 if (!kn->attr.open && new_on) { kn->attr.open =3D new_on; @@ -536,7 +528,7 @@ static int kernfs_get_open_node(struct kernfs_node *kn, list_add_tail(&of->list, &on->files); } =20 - spin_unlock_irq(&kernfs_open_node_lock); + spin_unlock_irq(lock); mutex_unlock(mutex); =20 if (on) { @@ -572,10 +564,13 @@ static void kernfs_put_open_node(struct kernfs_node *= kn, { struct kernfs_open_node *on =3D kn->attr.open; struct mutex *mutex =3D NULL; + spinlock_t *lock =3D NULL; unsigned long flags; =20 mutex =3D kernfs_open_file_mutex_lock(kn); - spin_lock_irqsave(&kernfs_open_node_lock, flags); + lock =3D kernfs_open_node_spinlock_ptr(kn); + + spin_lock_irqsave(lock, flags); =20 if (of) list_del(&of->list); @@ -585,7 +580,7 @@ static void kernfs_put_open_node(struct kernfs_node *kn, else on =3D NULL; =20 - spin_unlock_irqrestore(&kernfs_open_node_lock, flags); + spin_unlock_irqrestore(lock, flags); mutex_unlock(mutex); =20 kfree(on); @@ -768,15 +763,16 @@ void kernfs_drain_open_files(struct kernfs_node *kn) struct kernfs_open_node *on; struct kernfs_open_file *of; struct mutex *mutex =3D NULL; + spinlock_t *lock =3D NULL; =20 if (!(kn->flags & (KERNFS_HAS_MMAP | KERNFS_HAS_RELEASE))) return; =20 - spin_lock_irq(&kernfs_open_node_lock); + lock =3D kernfs_open_node_spinlock(kn); on =3D kn->attr.open; if (on) atomic_inc(&on->refcnt); - spin_unlock_irq(&kernfs_open_node_lock); + spin_unlock_irq(lock); if (!on) return; =20 @@ -921,13 +917,13 @@ void kernfs_notify(struct kernfs_node *kn) return; =20 /* kick poll immediately */ - spin_lock_irqsave(&kernfs_open_node_lock, flags); + spin_lock_irqsave(kernfs_open_node_spinlock_ptr(kn), flags); on =3D kn->attr.open; if (on) { atomic_inc(&on->event); wake_up_interruptible(&on->poll); } - spin_unlock_irqrestore(&kernfs_open_node_lock, flags); + spin_unlock_irqrestore(kernfs_open_node_spinlock_ptr(kn), flags); =20 /* schedule work to kick fsnotify */ spin_lock_irqsave(&kernfs_notify_lock, flags); --=20 2.30.2 From nobody Mon Jun 29 07:49:26 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0AE5C433F5 for ; Mon, 14 Feb 2022 12:04:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231644AbiBNMEi (ORCPT ); Mon, 14 Feb 2022 07:04:38 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:33416 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234504AbiBNMEO (ORCPT ); Mon, 14 Feb 2022 07:04:14 -0500 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9552126D9 for ; Mon, 14 Feb 2022 04:03:50 -0800 (PST) Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 21EATJ9B021228; Mon, 14 Feb 2022 12:03:47 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=3/F1X8fgIatRPSR9zP22TRADXmiGAgrQjRh/YvEL8Zk=; b=mPQ60ZVszm7AOszwlFtibVBV3drMX3CTkFbx/n5RiTwrKSzwliKHmBwgqfsOaFdjF5Fd AuAG2ljd82bVRr7oyRounKqNanUo0ol0bSUb+8O3G8lJSSi584ucGYLwgzTwXPWskbgD m3+lFCZHeH9yVpFtG1yA7nKldKXVg4sLEKoJr9VimCjnn0DuK0yW/eap5rtB4bJ46VwW pCHn7qglSYQ2TB/Ou1zu0HQA0baAyWqW4R04lhaUcxYGPPlYwrQGE5JSvDpbDFx2L1t3 tSyEEjpKRbJZLMYXD+UPS6iMnKOQmD2CEv7h9ccn4T9MGD4NAhFq1KTPZ3sZoXuId6s4 Ag== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by mx0b-00069f02.pphosted.com with ESMTP id 3e63p24b9u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 14 Feb 2022 12:03:46 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 21EBu1eT067149; Mon, 14 Feb 2022 12:03:45 GMT Received: from nam10-bn7-obe.outbound.protection.outlook.com (mail-bn7nam10lp2101.outbound.protection.outlook.com [104.47.70.101]) by aserp3020.oracle.com with ESMTP id 3e6qkwjh3n-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 14 Feb 2022 12:03:45 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UOc8IvMxgdiS20PLsIOmidsAXWy5u8TW8FSFDwONne1IrQJTJ2EHnp1VYUAEvl1mNi7m5Y1svHCdw3GwllZqkkToKPLo4LcXsagWV5f/ZJa67fJInd20BFZhWsFAlJmeV2M53SgRK9S50wqn5BeO9HcCBABdrFGwmhneTwnNjHebcCDdC8mlVeI7mMr0HNl8owMpXRy9cfdPcsYwnCokq+6MQTTpoV/qbXfEVDuSrF7Ig79r9Fg+yLIk6GDIb/lz1hXqKECSwrn1b78/knlm4Of5HTq6i3LPnrsdf9ukZllBr5QlPZHqtul4aRCzWXkrC+0QtgnyFzUQS1p2lJYsJA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=3/F1X8fgIatRPSR9zP22TRADXmiGAgrQjRh/YvEL8Zk=; b=Agxk7dES2lCFKe7zmib/1XJVQd+QV40vGgnFrV3xYv/Q0Erz3cqqQboiNFa0QadMXcvI8cxzgkA1wGudt/7Qxpl3Wl9bjuBin479QQCoUK/yVTYTPWgevI9KQyP0Wq077U9XwqS/bvW8Ry52Zfng5D5ZQ7FUjeau2ygtSlK77jMMAYI9aLKOw6Mwyt6AclC06VBl2GHKXf/q65qqiFl+C5RH7eyo7uIAks5H7tAtaKTGa3UUE0h72WROIGkw24WZcxZu7Ea8DG0BxWoPUleHPhjd583QArAIHhCHR/wH9d9f7sd2XbUY1YQpP2lqCbKPGx7WqDP/wZj7L39cbAOEvQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3/F1X8fgIatRPSR9zP22TRADXmiGAgrQjRh/YvEL8Zk=; b=MdjkIWOcRtoOtNdct6RP2e2jy91A4gg5yf1hZ4PvuWa+nku4lRqAZgCsJ1T3e0LfEVAS0018QysUGjf0mK4kFRE4do9EVkIUAKpLiXXJCxNzH8ILaQlDML2yCEz8MtL3HA922YJ7CbXPQ60Gqv63bl+JtUHey+W15jvvHKFuV3I= Received: from CO1PR10MB4468.namprd10.prod.outlook.com (2603:10b6:303:6c::24) by MN2PR10MB4191.namprd10.prod.outlook.com (2603:10b6:208:1d1::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4975.11; Mon, 14 Feb 2022 12:03:43 +0000 Received: from CO1PR10MB4468.namprd10.prod.outlook.com ([fe80::a5d1:ed4:5ab6:e9b1]) by CO1PR10MB4468.namprd10.prod.outlook.com ([fe80::a5d1:ed4:5ab6:e9b1%4]) with mapi id 15.20.4975.019; Mon, 14 Feb 2022 12:03:43 +0000 From: Imran Khan To: tj@kernel.org, gregkh@linuxfoundation.org Cc: linux-kernel@vger.kernel.org Subject: [PATCH v6 5/7] kernfs: Use a per-fs rwsem to protect per-fs list of kernfs_super_info. Date: Mon, 14 Feb 2022 23:03:20 +1100 Message-Id: <20220214120322.2402628-6-imran.f.khan@oracle.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220214120322.2402628-1-imran.f.khan@oracle.com> References: <20220214120322.2402628-1-imran.f.khan@oracle.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SY2PR01CA0016.ausprd01.prod.outlook.com (2603:10c6:1:14::28) To CO1PR10MB4468.namprd10.prod.outlook.com (2603:10b6:303:6c::24) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 90fe73a0-bbb9-4833-75e6-08d9efb20bd4 X-MS-TrafficTypeDiagnostic: MN2PR10MB4191:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:51; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: C4ieElShc7y8ZAHr2CW2d/d6iN5+VrLr+7bcg1foFIHN3KLzbUBrODwzuoClOqoVcCKovihfSTfTIn0TwWPpg8IT9lTShFLdXnE3LpHkC/s6oVpcz93EC/3sLwHgnil0klQJBvy8siRZhVo35HnK8w9XJVcjY48BgGtuQWOa18htsSSeJ3ia8AIMhXItuj9TEixejdo9zjsKwX0saYotWr25HSgtBh4aUxYM6BEJxjj3hRFqN8WKwROCveiNeiCQ2vgf0ejQp1wxvd/D8Ng9rtnF0aLa/aJL3rKk7aKoTeYb9/6DkZwmxL8finsXbNjoWX/gQb4Lenp2+iabuvJqJSeQySoB1EuITwuxsyEi8w859luSa35tmuwEK8dFjcvuGCGhVD/4qVCuP7pMlkcSORLUNI/iKAz2/AZQble8KiN5sIuuQVhUG6g6wHL26iE8xn03DBE8DWTdh1SG7+sKUlnM3ZJ4OJxD8neQEGMAJlnb+ZEppClkBHU2CkVoWZ8O84mM7RCHNgckhQaFAUPl+ESQOKLgjQS52hQOVessFLDlJV4CHliN9Gd2RyaTROTR+dT0d7unx7dMAP5hMWhvlJs0iIqnf+BBgkTJeaxOpCNQgbxAI7nNB5k5TWw+wRzqrbLUqNS/ymWyLXYSUB4vPzJZ0n4qg0oBRvzX7dFb2UzKNqAV21IW4r+ya4Tudh4YUKIrexqGYvJoAXdQVCqqRw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO1PR10MB4468.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(86362001)(2616005)(103116003)(5660300002)(38350700002)(1076003)(26005)(186003)(6512007)(38100700002)(2906002)(6666004)(6506007)(8676002)(36756003)(66946007)(66556008)(66476007)(316002)(6486002)(83380400001)(8936002)(4326008)(52116002)(508600001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Yoa5lTx/sfdKBWNYfhsOnl4G98AtMNkS/18GonjXU6wbL/lvszbvqpzdivzF?= =?us-ascii?Q?11FQS7LkI2I8h59eYV4Box6KQFJQ2LE1VMJmBiOxHvww+iJwV1ilqLGG1l/w?= =?us-ascii?Q?zpUxecuarviEmu3oUyLW4xPU4MNyP6KEkYyuZcQkRS5ch+JxrqMktZRB2GTn?= =?us-ascii?Q?gRbDCmthoMN6i0jBMPw74M5TygY0qZtd7dpO6rgfgWUmHylxNUWTniF42409?= =?us-ascii?Q?Ax1Xz8w0XZM0e08IxvS5CL3zpGDVhKrxOEd5pXmHq+vx70km+SEEOb0Hui54?= =?us-ascii?Q?ipGTS6QmunOsGjB/dLW+XD4kDJ6dB+x+0HcCLKuMsEroFFfMJ+1GGJpAWAYW?= =?us-ascii?Q?ag29xYYbLu0sR0HxkvPr+ms3l+6GfYuHlX7f3utjetqWaSUN8DbYHj4x1Vas?= =?us-ascii?Q?Nz+wyfvOqwfPrJNdx/vUzZU1+Q17ek42zeY5bOGnu+UIJd9JSBmE+Mv9fH0E?= =?us-ascii?Q?9S3dl/ChChS80X4J8dXsVQ7Ii0oprCXmcykx2UOT/NPGDAUSGdNoaQ79q3Qq?= =?us-ascii?Q?++W8TDIx2J9TeZKFvfAPc/BKwYq6jxm+QLp0HWG1uF0G83ZuucgxoLavenUj?= =?us-ascii?Q?3jJyMi6p0KDtebD5BoSQ+39hz0VdT4i82DNsXLxy43yuRSUlmsxccvSOlJaO?= =?us-ascii?Q?ed55z9OEbytowHzGCaxEigrMYOeYtEYPeSpnFxjQxMIeL2E2VqvsHp7X1r3j?= =?us-ascii?Q?wLwCjFnTWfTbw28ySaTftuE5+XfQkFruQKxXefYmlcfRDC1fxMxLEnEvwOWa?= =?us-ascii?Q?eU/S6p2hunczCJjWrib/oK4ZuuwY7kLQExwSFZdm7IV6dlwiR0p31N1H+qMn?= =?us-ascii?Q?N3iisZ7ZMMOVjsvd9VQRWZTPORzl5aNYI4XgRsKrQvwuOMOPAu7IYj7N78zi?= =?us-ascii?Q?nc23edagPnoYy49eKIas0Wb+b5/GxChobKAHCdqS98OMzZGwkaFVwwBSlQ1D?= =?us-ascii?Q?H/p636Dfp7i13ByIZUkezfeJIFOWWoShAQSFsP4119TD7hQ68d8gvo9juyjQ?= =?us-ascii?Q?CtyC/OhQdvB4oZddGFgsDE6bROnrYcj7IdC/1/4TJyr/i7PsMRJyrkAV7wRq?= =?us-ascii?Q?RrzaNr27IhOeWyOmlhu9IjRqy9bc8bRSzZKO+/4Gvl/g+gox0m7Lon7bqTK3?= =?us-ascii?Q?AcFV9CVw8C4AugOF4fx1aAm5On1sjK87AO0StiNqMfz6E4yHqluJ9cqjnl5F?= =?us-ascii?Q?CDbYCAjO1fGKqGtxyKkGrLwOfj/WsEve0Qw7lhqvrXzk+QWK+mv7QZc6wptf?= =?us-ascii?Q?+8YCd1ZJlQ1tUhDQEhDABqkZ+3w40n2xiFWR6LFC+MOBLUV1dVR/F+A2DzlF?= =?us-ascii?Q?4b2UCqCBpgE3ms3hzodkJYcD4SkTWFOhV/e9emJUb3y1EsfoePOixi18Ok/A?= =?us-ascii?Q?N1SQuBhGEXsH4wlikyR3q1VNbN2nbCsaTew3XFMCy1356I9pj+mKnXoO+K0/?= =?us-ascii?Q?BzkN450tuE41gIT+Ci5mALxnW2aQxyq5ZdEF3MR9M3Ns/kKxCzINS/JQsEDp?= =?us-ascii?Q?qNDu2KCO3evqBXPLfkIbQmInmianJx67qeKUzgoK2m4Zy0tnvfU/RwUVuL/v?= =?us-ascii?Q?xjSdEE3yl89B+Y60rXP8fwXwQE4dmcEoN/TLf9dfBB8WL0agMgTmddpHZi5u?= =?us-ascii?Q?WonjKfANmlJHH+pC6iKlOWM=3D?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 90fe73a0-bbb9-4833-75e6-08d9efb20bd4 X-MS-Exchange-CrossTenant-AuthSource: CO1PR10MB4468.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Feb 2022 12:03:43.2587 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: H/NC50f+nSY553S+aifsRb9NaZupoRJbUrdABQ5FNMY3jdrFW8QkoAcRFI36gwqCGp4habd9McJxYaZpnhjOpg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB4191 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10257 signatures=673431 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 bulkscore=0 suspectscore=0 phishscore=0 malwarescore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2201110000 definitions=main-2202140073 X-Proofpoint-ORIG-GUID: dOLXksAU45Ceh0F1ieJLGK5upFKiwjVd X-Proofpoint-GUID: dOLXksAU45Ceh0F1ieJLGK5upFKiwjVd Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Right now per-fs kernfs_rwsem protects list of kernfs_super_info instances for a kernfs_root. Since kernfs_rwsem is used to synchronize several other operations across kernfs and since most of these operations don't impact kernfs_super_info, we can use a separate per-fs rwsem to synchronize access to list of kernfs_super_info. This helps in reducing contention around kernfs_rwsem and also allows operations that change/access list of kernfs_super_info to proceed without contending for kernfs_rwsem. Signed-off-by: Imran Khan --- fs/kernfs/dir.c | 1 + fs/kernfs/file.c | 2 ++ fs/kernfs/mount.c | 8 ++++---- include/linux/kernfs.h | 1 + 4 files changed, 8 insertions(+), 4 deletions(-) diff --git a/fs/kernfs/dir.c b/fs/kernfs/dir.c index e6d9772ddb4ca..dc769301ac96b 100644 --- a/fs/kernfs/dir.c +++ b/fs/kernfs/dir.c @@ -917,6 +917,7 @@ struct kernfs_root *kernfs_create_root(struct kernfs_sy= scall_ops *scops, idr_init(&root->ino_idr); init_rwsem(&root->kernfs_rwsem); INIT_LIST_HEAD(&root->supers); + init_rwsem(&root->supers_rwsem); =20 /* * On 64bit ino setups, id is ino. On 32bit, low 32bits are ino. diff --git a/fs/kernfs/file.c b/fs/kernfs/file.c index f3ecc6fe8aedc..af2046bc63aa1 100644 --- a/fs/kernfs/file.c +++ b/fs/kernfs/file.c @@ -859,6 +859,7 @@ static void kernfs_notify_workfn(struct work_struct *wo= rk) /* kick fsnotify */ down_write(&root->kernfs_rwsem); =20 + down_write(&root->supers_rwsem); list_for_each_entry(info, &kernfs_root(kn)->supers, node) { struct kernfs_node *parent; struct inode *p_inode =3D NULL; @@ -894,6 +895,7 @@ static void kernfs_notify_workfn(struct work_struct *wo= rk) =20 iput(inode); } + up_write(&root->supers_rwsem); =20 up_write(&root->kernfs_rwsem); kernfs_put(kn); diff --git a/fs/kernfs/mount.c b/fs/kernfs/mount.c index 809b738739b18..d35142226c340 100644 --- a/fs/kernfs/mount.c +++ b/fs/kernfs/mount.c @@ -347,9 +347,9 @@ int kernfs_get_tree(struct fs_context *fc) } sb->s_flags |=3D SB_ACTIVE; =20 - down_write(&root->kernfs_rwsem); + down_write(&root->supers_rwsem); list_add(&info->node, &info->root->supers); - up_write(&root->kernfs_rwsem); + up_write(&root->supers_rwsem); } =20 fc->root =3D dget(sb->s_root); @@ -376,9 +376,9 @@ void kernfs_kill_sb(struct super_block *sb) struct kernfs_super_info *info =3D kernfs_info(sb); struct kernfs_root *root =3D info->root; =20 - down_write(&root->kernfs_rwsem); + down_write(&root->supers_rwsem); list_del(&info->node); - up_write(&root->kernfs_rwsem); + up_write(&root->supers_rwsem); =20 /* * Remove the superblock from fs_supers/s_instances diff --git a/include/linux/kernfs.h b/include/linux/kernfs.h index 7ee0595b315a2..84653c609a5c0 100644 --- a/include/linux/kernfs.h +++ b/include/linux/kernfs.h @@ -265,6 +265,7 @@ struct kernfs_root { =20 wait_queue_head_t deactivate_waitq; struct rw_semaphore kernfs_rwsem; + struct rw_semaphore supers_rwsem; }; =20 struct kernfs_open_file { --=20 2.30.2 From nobody Mon Jun 29 07:49:26 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D1162C433F5 for ; Mon, 14 Feb 2022 12:04:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352471AbiBNME3 (ORCPT ); Mon, 14 Feb 2022 07:04:29 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:33418 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240274AbiBNMEO (ORCPT ); Mon, 14 Feb 2022 07:04:14 -0500 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9963C26C0 for ; Mon, 14 Feb 2022 04:03:51 -0800 (PST) Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 21EB2N25028540; Mon, 14 Feb 2022 12:03:48 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=JY/CkGNpD4zyGmkew+M7iWeOszH1sOgXdOhFW7wZEOk=; b=rSy+SwYCd74N+hcrpgMzeKB7kRkgQA0vsujUe6kRXcseF0XcNn+1FhEKfbY1jl07M4RK A5x91iAX5YnplrKC6Y7xL6MbPc1TFlaeWw7Nb6lI0k7mCiPHmvUiqLRhZjULT2r2SC/y uVzvguKKKNSgQF67LxXpGCD+TjGDfwJzcdeaW1njolJThG9SqgvjlpXickUtKSy4+79N EbD4GdJra3q0XfP74eAffwWB6hZk5S5YO/svmOfzssRu884Uq56lFMIlzk1bgUdWY+V1 XSUUZ+gZC/p1WMYzygX3CaWfznuNjuO0RzuB/Ynv7SnPjw935uXHg1wrYCJHWny+/Qs5 RQ== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by mx0b-00069f02.pphosted.com with ESMTP id 3e63g148k6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 14 Feb 2022 12:03:48 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 21EBuOLN176293; Mon, 14 Feb 2022 12:03:47 GMT Received: from nam10-bn7-obe.outbound.protection.outlook.com (mail-bn7nam10lp2107.outbound.protection.outlook.com [104.47.70.107]) by aserp3030.oracle.com with ESMTP id 3e62xd07jw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 14 Feb 2022 12:03:47 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Aauf4noz1eyns1UladE8KhVpBpNjT+bJ+x3NE6x4u3aRzWZDgguJF8ZeusA8I/l10FLaVFLVyfr9utH4pIa5HDs60B2udrUC0HsQFwrpYRl4N/ILkThmHfGChygYcgSI/wvGEMDTEZm7W7dSmTUDR3zrB1jcDmh7XsuT+I8Nx8P9RXtJn7vbthoib+kAmTHUjMLQ0rXEZnJ4bBGeOcOznIKNsnOeg3VnK61iAQ5Ce3uSxm6HQIpQES/S5m3IyMQigTorhWCrD084o1Arz5p1vVOY1JWZZMPr7+eQNP6abv+yDcV0FeT1FRMtnBGhCniK/JVukpFqYwAyt3xT1wpQvg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=JY/CkGNpD4zyGmkew+M7iWeOszH1sOgXdOhFW7wZEOk=; b=EfJDk6/rlmKM83aFe7IGd1TJFbdKQqKd9zP1touas/Ihjlcn3TMPbrWpN9nehz5YBY9cAMB9w/sXYLgCmG1QSu2efxsXxhA0VP6DxHuB5fhfcnyD9v9bHcCq2A7IFqr8TKnM8AyFsZgRadlH/S6MmZvKyhtQAaiHspDiyUlmYQTB4In9/n75G3fNkD7iLubFA5ewYvDy2kUCY0jSH/Up6scjFyPF0tSNwitU148QlSz9pdO0SM0FcPcwLfngIH63RnO5F5kRW93+EUQrmWTsbua7X/c4DWpBHRL0IV+AaFdcItWHLXq8KrZMnit8qCk6o9ePTU7R5xPUUV8n0Z1cuA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JY/CkGNpD4zyGmkew+M7iWeOszH1sOgXdOhFW7wZEOk=; b=AJqGNDbxgRWgbwE9QyBSK4yVScHUyVdak1Pe3oNLLND9G1Y6vvK7o3V1yDw6fOBXt0B3V0XhUFThpLSgRjtK5noWG89LEwmCAYcWgrWC53zayjqVrXbNfndHJT3SNaIm2KbEk00fnlTK5HQ+oxFZQxGXiHhml1BkjacjsnKUlGs= Received: from CO1PR10MB4468.namprd10.prod.outlook.com (2603:10b6:303:6c::24) by MN2PR10MB4191.namprd10.prod.outlook.com (2603:10b6:208:1d1::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4975.11; Mon, 14 Feb 2022 12:03:45 +0000 Received: from CO1PR10MB4468.namprd10.prod.outlook.com ([fe80::a5d1:ed4:5ab6:e9b1]) by CO1PR10MB4468.namprd10.prod.outlook.com ([fe80::a5d1:ed4:5ab6:e9b1%4]) with mapi id 15.20.4975.019; Mon, 14 Feb 2022 12:03:45 +0000 From: Imran Khan To: tj@kernel.org, gregkh@linuxfoundation.org Cc: linux-kernel@vger.kernel.org Subject: [PATCH v6 6/7] kernfs: Introduce hashed rw-sem to replace per-fs kernfs_rwsem. Date: Mon, 14 Feb 2022 23:03:21 +1100 Message-Id: <20220214120322.2402628-7-imran.f.khan@oracle.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220214120322.2402628-1-imran.f.khan@oracle.com> References: <20220214120322.2402628-1-imran.f.khan@oracle.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SY2PR01CA0016.ausprd01.prod.outlook.com (2603:10c6:1:14::28) To CO1PR10MB4468.namprd10.prod.outlook.com (2603:10b6:303:6c::24) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b3e4dd9b-230c-4d67-efcd-08d9efb20cce X-MS-TrafficTypeDiagnostic: MN2PR10MB4191:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8882; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: VSk4s1i1OrYw46TftMZM7aSMiYv4J0SvxSEl1+1qv8umLT1lbZbsByIIjNcPBaA0fmpb7I8/c2zlv/ACmmikpJ31O+lzfmwKUY1VyY02IdElIUciBTrXHkZnTShGlarKmpbY0mgRnomwha1vCVO/oCsPwhUFfR7DjKS3yAn8s43Hg8Z8/zLbSNRDnE+8stM5yoj/6GVC0UOafv9WgJij6L9lTkhJ1qUiZdH/yhXpkuSFi6xKK/LkdecZ10MSl3zWhfjeaHhSWk9sSbzfwkNlBM8nyug2MgOk6mXB69M9J/17JEkaDZqhePCGs8tUzvXcdDnckpCjyNciqZwgzZdtGVPuQ7a9DTXcj2qBgCZh2f6291CT+bAadvLsf8LbLS1+N0QTXyh0w+hMCV2RHM0udPZiYnrv9SOYNf4GCYewF0yp15A0QQpQGxq+PeoUWp3jpdDtACnqzU0z3AxgEA+OTpzPJTjgYndYNhnT9yZ/+6HO6VwypktD2DiRUqybKM8V/WSikxUCBiaTPj9g37QVinQ5zXwI3U6QY5zTPEXaNN/HVW06Z+RMVjWNqRuMp1ZwzazhA841GnTZAVhV/wDl1gGkmIhZlO1HR21aQ8CK+OjzWIgtSDPIndyGCpEkAU500LzpJ67VZpQZIIkvlrIQDO1J3c1MpRY3ZjRq23MsZHffvzggAAj4ofOqoZF2JsI6eIXd0W3fyK2irvKs6+VWHQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO1PR10MB4468.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(86362001)(2616005)(103116003)(5660300002)(38350700002)(1076003)(26005)(186003)(6512007)(38100700002)(2906002)(6666004)(6506007)(8676002)(36756003)(66946007)(30864003)(66556008)(66476007)(316002)(6486002)(83380400001)(8936002)(4326008)(52116002)(508600001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?AiodMEx///tDNmn8/yY0z8Mwot9tU0DD82vSGMgGPQbRap29lLphPFoubHb+?= =?us-ascii?Q?KGIOGo26560iW/87sZ1yiNOTYbjCDLU956xsVlV5LO+L6HdosCRvPykFQ6Pr?= =?us-ascii?Q?Ofr8xLb0LuFQv4o8ga3hjLBdh4YMB4+k6zzRuPzuoBvrgG4iYeB/XvGl929g?= =?us-ascii?Q?/Lhpki6NH4Wr29smCglitg+/5HO99G0PRohENLW82CHdszz5ePduXZw683PX?= =?us-ascii?Q?Ag5IeUNizCDT/DazHrB/96KjSzrFBnAHwAGD/O857DxqoeKHNVRN5n+l2sRt?= =?us-ascii?Q?zAaz5uDiRdtDJVcEqUYrgot1rXMBNFAV+tPhhyT2H6n91kwq8wbQ0TB6bpvz?= =?us-ascii?Q?/rF4IltIklQAqOwKapAwQ0Xnn/P+fsg41Ij40AhFveWe0HXIGWbq8CC5DUmD?= =?us-ascii?Q?lMRljeI/HJcvmngMq7jBqeo8+BwZ4xXe02OE5gGj7jjnqxfgUM0rCzBCp6dJ?= =?us-ascii?Q?c8Ey7IFapi9nxvoew+f15k5olTKgwbhTtL43DU8s47/JuZhJAhwjt3+xThhi?= =?us-ascii?Q?C6lrW/3/63bYTC6ndiK7aIWYNztfBDtP7zXbtf8klAiPyH18TCuESu5RxPfj?= =?us-ascii?Q?uTBkjdjO9AkQYTX/9SdJDyahib+9VQJ7/8yGasMPoFyMmw6bD/kGuAw/1snW?= =?us-ascii?Q?mYSFQMClsmUOqMRrd/lqZqZmyKnw7puxfbMb9+bNyraqlO6I+XDLJkyjoDCp?= =?us-ascii?Q?nrZiud1TyVbzluE5ZzV3syuuEVSUCwZvRjdF/+8EvYGCsn/sVmupIEc5tFDl?= =?us-ascii?Q?bg4qrosSb5U6hSCbHJieFWCGJRZl6KM01WUKmRq7jGEW+myR1mZ+OhPS9bUs?= =?us-ascii?Q?2pDfRvlLr08Vxpmbig+GBUy3EwQUAFSX7LubP/5s+C7/1ETpDU4j+OppBuK/?= =?us-ascii?Q?RUZpF3ghh2QGeJpLxonoPaSj7Ilp5OZAiwRgaaGnqvliM84t+y5YBKTvRktD?= =?us-ascii?Q?sT3VM77Cw5Fw0Dlh15IraLtd+vcjYUQfrFo6/N/ZKPRGdsD1FKRIPKyMCOqM?= =?us-ascii?Q?IFKMkcavflunTJpb0mwoveMWRH+6HOjacZg2KKNphtFHEKB8rv8IkDWoKHcp?= =?us-ascii?Q?s/UHD5vCj78hOJr89dI2LOW8/PXL/4+gignq6gUqNjOIEMIkeSU6mw3DWECv?= =?us-ascii?Q?CSv6sMTEhIDwwZxdcmE3xx/VEokiRWcwKhywNul0BXAw9z/i50y5I4QGKA9Q?= =?us-ascii?Q?6w3pymhOY9+l44Zl8D5/AxVQi/DRorM9Ko/d03nsZVzHc8Qd1h3zYasrm8bK?= =?us-ascii?Q?A1RIjsV9RRtIsFCgY9EFfPPt46uhT0/OCvA4OxMv1DVATZqrp9yfy26ayXck?= =?us-ascii?Q?pmySL0HLYeTWOjakL1lDxmWgnkT+kqlTqjX6yJyaKVSDFCWHORkPniGnTBUF?= =?us-ascii?Q?kAV4Br8myrWdX56zbMNSHFMUNEZmdrCTV9rno8If7zufW8yQso8SmDInn/Gn?= =?us-ascii?Q?wt1cOWSYaxFKyCBBPjOeqsWKPVcB7kCVy+MMNgKzqaG0HCFh2YtQXeGQkyRQ?= =?us-ascii?Q?GiXG0IDQNHOSv0IMqeKfi+/Lt36aTqS9Hu4q5KbIzZob2GysFioKqX6YGrwR?= =?us-ascii?Q?ZlvixlvBV31tGteRekLolqXiMNU2T/swGpFkygoRHwmZt0rl3nua6vInqlHS?= =?us-ascii?Q?6DDEYu1OIRCt0HMJe0U15GY=3D?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: b3e4dd9b-230c-4d67-efcd-08d9efb20cce X-MS-Exchange-CrossTenant-AuthSource: CO1PR10MB4468.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Feb 2022 12:03:45.0397 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: BiVhGKuisdhnxqFLf86sX6oXz25KjGEGjVksNdXkVW5fqAvaFSbgQpMphiXrOZG75Bb7pbCbKETueLI1GXEKnA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB4191 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10257 signatures=673431 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 phishscore=0 bulkscore=0 malwarescore=0 adultscore=0 spamscore=0 mlxlogscore=999 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2201110000 definitions=main-2202140073 X-Proofpoint-GUID: dQKN3ykCevxnCO5lzK14NdjKFmzOi_qI X-Proofpoint-ORIG-GUID: dQKN3ykCevxnCO5lzK14NdjKFmzOi_qI Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Having a single rwsem to synchronize all operations across a kernfs based file system (cgroup, sysfs etc.) does not scale well. The contention around this single rwsem becomes more apparent in large scale systems with few hundred CPUs where most of the CPUs have running tasks that are opening, accessing or closing sysfs files at any point of time. Using hashed rwsems in place of a per-fs rwsem, can significantly reduce contention around per-fs rwsem and hence provide better scalability. Moreover as these hashed rwsems are not part of kernfs_node objects we will not see any singnificant change in memory utilization of kernfs based file systems like sysfs, cgroupfs etc. This patch introduces hashed rwsems that can be used in place of per-fs rwsem. It also provides interfaces to use hashed rwsems and interfaces for lockdep annotation as well. The next patch makes use of these interfaces and replaces per-fs rwsem with hashed ones. Signed-off-by: Imran Khan --- fs/kernfs/dir.c | 1 - fs/kernfs/kernfs-internal.h | 560 ++++++++++++++++++++++++++++++++++++ fs/kernfs/mount.c | 1 + include/linux/kernfs.h | 2 + 4 files changed, 563 insertions(+), 1 deletion(-) diff --git a/fs/kernfs/dir.c b/fs/kernfs/dir.c index dc769301ac96b..0dac58f8091c9 100644 --- a/fs/kernfs/dir.c +++ b/fs/kernfs/dir.c @@ -25,7 +25,6 @@ static DEFINE_SPINLOCK(kernfs_idr_lock); /* root->ino_idr= */ =20 static bool kernfs_active(struct kernfs_node *kn) { - lockdep_assert_held(&kernfs_root(kn)->kernfs_rwsem); return atomic_read(&kn->active) >=3D 0; } =20 diff --git a/fs/kernfs/kernfs-internal.h b/fs/kernfs/kernfs-internal.h index 593395f325a18..ba89de378f240 100644 --- a/fs/kernfs/kernfs-internal.h +++ b/fs/kernfs/kernfs-internal.h @@ -19,6 +19,17 @@ #include #include =20 +/* + * kernfs_rwsem locking pattern: + * + * KERNFS_RWSEM_LOCK_SELF: lock kernfs_node only. + * KERNFS_RWSEM_LOCK_SELF_AND_PARENT: lock kernfs_node and its parent. + */ +enum kernfs_rwsem_lock_pattern { + KERNFS_RWSEM_LOCK_SELF, + KERNFS_RWSEM_LOCK_SELF_AND_PARENT +}; + struct kernfs_iattrs { kuid_t ia_uid; kgid_t ia_gid; @@ -190,4 +201,553 @@ kernfs_open_node_spinlock(struct kernfs_node *kn) return lock; } =20 +static inline struct rw_semaphore *kernfs_rwsem_ptr(struct kernfs_node *kn) +{ + int idx =3D hash_ptr(kn, NR_KERNFS_LOCK_BITS); + + return &kernfs_locks->kernfs_rwsem[idx]; +} + +static inline void kernfs_rwsem_assert_held(struct kernfs_node *kn) +{ + lockdep_assert_held(kernfs_rwsem_ptr(kn)); +} + +static inline void kernfs_rwsem_assert_held_write(struct kernfs_node *kn) +{ + lockdep_assert_held_write(kernfs_rwsem_ptr(kn)); +} + +static inline void kernfs_rwsem_assert_held_read(struct kernfs_node *kn) +{ + lockdep_assert_held_read(kernfs_rwsem_ptr(kn)); +} + +/** + * down_write_kernfs_rwsem_for_two_nodes() - Acquire hashed rwsem for 2 no= des + * + * @kn1: kernfs_node for which hashed rwsem needs to be taken + * @kn2: kernfs_node for which hashed rwsem needs to be taken + * + * In certain cases we need to acquire hashed rwsem for 2 nodes that don't= have a + * parent child relationship. This is one of the cases of nested locking i= nvolving + * hashed rwsem and rwsem with lower address is acquired first. + */ +static inline void down_write_kernfs_rwsem_for_two_nodes(struct kernfs_nod= e *kn1, + struct kernfs_node *kn2) +{ + struct rw_semaphore *rwsem1 =3D kernfs_rwsem_ptr(kn1); + struct rw_semaphore *rwsem2 =3D kernfs_rwsem_ptr(kn2); + + if (rwsem1 =3D=3D rwsem2) + down_write_nested(rwsem1, 0); + else { + if (rwsem1 < rwsem2) { + down_write_nested(rwsem1, 0); + down_write_nested(rwsem2, 1); + } else { + down_write_nested(rwsem2, 0); + down_write_nested(rwsem1, 1); + } + } + kernfs_get(kn1); + kernfs_get(kn2); +} + +/** + * up_write_kernfs_rwsem_for_two_nodes() - Release hashed rwsem for 2 nodes + * + * @kn1: kernfs_node for which hashed rwsem needs to be released + * @kn2: kernfs_node for which hashed rwsem needs to be released + * + * In case of nested locking, rwsem with higher address is released first. + */ +static inline void up_write_kernfs_rwsem_for_two_nodes(struct kernfs_node = *kn1, + struct kernfs_node *kn2) +{ + struct rw_semaphore *rwsem1 =3D kernfs_rwsem_ptr(kn1); + struct rw_semaphore *rwsem2 =3D kernfs_rwsem_ptr(kn2); + + if (rwsem1 =3D=3D rwsem2) + up_write(rwsem1); + else { + if (rwsem1 > rwsem2) { + up_write(rwsem1); + up_write(rwsem2); + } else { + up_write(rwsem2); + up_write(rwsem1); + } + } + + kernfs_put(kn1); + kernfs_put(kn2); +} + +/** + * down_read_kernfs_rwsem_for_two_nodes() - Acquire hashed rwsem for 2 nod= es + * + * @kn1: kernfs_node for which hashed rwsem needs to be taken + * @kn2: kernfs_node for which hashed rwsem needs to be taken + * + * In certain cases we need to acquire hashed rwsem for 2 nodes that don't= have a + * parent child relationship. This is one of the cases of nested locking i= nvolving + * hashed rwsem and rwsem with lower address is acquired first. + */ +static inline void down_read_kernfs_rwsem_for_two_nodes(struct kernfs_node= *kn1, + struct kernfs_node *kn2) +{ + struct rw_semaphore *rwsem1 =3D kernfs_rwsem_ptr(kn1); + struct rw_semaphore *rwsem2 =3D kernfs_rwsem_ptr(kn2); + + if (rwsem1 =3D=3D rwsem2) + down_read_nested(rwsem1, 0); + else { + if (rwsem1 < rwsem2) { + down_read_nested(rwsem1, 0); + down_read_nested(rwsem2, 1); + } else { + down_read_nested(rwsem2, 0); + down_read_nested(rwsem1, 1); + } + } + kernfs_get(kn1); + kernfs_get(kn2); +} + +/** + * up_read_kernfs_rwsem_for_two_nodes() - Release hashed rwsem for 2 nodes + * + * @kn1: kernfs_node for which hashed rwsem needs to be released + * @kn2: kernfs_node for which hashed rwsem needs to be released + * + * In case of nested locking, rwsem with higher address is released first. + */ +static inline void up_read_kernfs_rwsem_for_two_nodes(struct kernfs_node *= kn1, + struct kernfs_node *kn2) +{ + struct rw_semaphore *rwsem1 =3D kernfs_rwsem_ptr(kn1); + struct rw_semaphore *rwsem2 =3D kernfs_rwsem_ptr(kn2); + + if (rwsem1 =3D=3D rwsem2) + up_read(rwsem1); + else { + if (rwsem1 > rwsem2) { + up_read(rwsem1); + up_read(rwsem2); + } else { + up_read(rwsem2); + up_read(rwsem1); + } + } + + kernfs_put(kn1); + kernfs_put(kn2); +} + +/** + * down_write_kernfs_rwsem() - Acquire hashed rwsem + * + * @kn: kernfs_node for which hashed rwsem needs to be taken + * @ptrn: locking pattern i.e whether to lock only given node or to lock + * node and its parent as well + * + * In case of nested locking, rwsem with lower address is acquired first. + * + * Return: void + */ +static inline void down_write_kernfs_rwsem(struct kernfs_node *kn, + enum kernfs_rwsem_lock_pattern ptrn) +{ + struct rw_semaphore *p_rwsem =3D NULL; + struct rw_semaphore *rwsem =3D kernfs_rwsem_ptr(kn); + int lock_parent =3D 0; + + if (ptrn =3D=3D KERNFS_RWSEM_LOCK_SELF_AND_PARENT && kn->parent) + lock_parent =3D 1; + + if (lock_parent) + p_rwsem =3D kernfs_rwsem_ptr(kn->parent); + + if (!lock_parent || rwsem =3D=3D p_rwsem) { + down_write_nested(rwsem, 0); + kernfs_get(kn); + kn->unlock_parent =3D 0; + } else { + /** + * In case of nested locking, locks are taken in order of their + * addresses. lock with lower address is taken first, followed + * by lock with higher address. + */ + if (rwsem < p_rwsem) { + down_write_nested(rwsem, 0); + down_write_nested(p_rwsem, 1); + } else { + down_write_nested(p_rwsem, 0); + down_write_nested(rwsem, 1); + } + kernfs_get(kn); + kernfs_get(kn->parent); + kn->unlock_parent =3D 1; + } +} + +/** + * up_write_kernfs_rwsem() - Release hashed rwsem + * + * @kn: kernfs_node for which hashed rwsem was taken + * + * In case of nested locking, rwsem with higher address is released first. + * + * Return: void + */ +static inline void up_write_kernfs_rwsem(struct kernfs_node *kn) +{ + struct rw_semaphore *p_rwsem =3D NULL; + struct rw_semaphore *rwsem =3D kernfs_rwsem_ptr(kn); + + if (kn->unlock_parent) { + kn->unlock_parent =3D 0; + p_rwsem =3D kernfs_rwsem_ptr(kn->parent); + if (rwsem > p_rwsem) { + up_write(rwsem); + up_write(p_rwsem); + } else { + up_write(p_rwsem); + up_write(rwsem); + } + kernfs_put(kn->parent); + } else + up_write(rwsem); + + kernfs_put(kn); +} + +/** + * down_read_kernfs_rwsem() - Acquire hashed rwsem + * + * @kn: kernfs_node for which hashed rwsem needs to be taken + * @ptrn: locking pattern i.e whether to lock only given node or to lock + * node and its parent as well + * + * In case of nested locking, rwsem with lower address is acquired first. + * + * Return: void + */ +static inline void down_read_kernfs_rwsem(struct kernfs_node *kn, + enum kernfs_rwsem_lock_pattern ptrn) +{ + struct rw_semaphore *p_rwsem =3D NULL; + struct rw_semaphore *rwsem =3D kernfs_rwsem_ptr(kn); + int lock_parent =3D 0; + + if (ptrn =3D=3D KERNFS_RWSEM_LOCK_SELF_AND_PARENT && kn->parent) + lock_parent =3D 1; + + if (lock_parent) + p_rwsem =3D kernfs_rwsem_ptr(kn->parent); + + if (!lock_parent || rwsem =3D=3D p_rwsem) { + down_read_nested(rwsem, 0); + kernfs_get(kn); + kn->unlock_parent =3D 0; + } else { + /** + * In case of nested locking, locks are taken in order of their + * addresses. lock with lower address is taken first, followed + * by lock with higher address. + */ + if (rwsem < p_rwsem) { + down_read_nested(rwsem, 0); + down_read_nested(p_rwsem, 1); + } else { + down_read_nested(p_rwsem, 0); + down_read_nested(rwsem, 1); + } + kernfs_get(kn); + kernfs_get(kn->parent); + kn->unlock_parent =3D 1; + } +} + +/** + * up_read_kernfs_rwsem() - Release hashed rwsem + * + * @kn: kernfs_node for which hashed rwsem was taken + * + * In case of nested locking, rwsem with higher address is released first. + * + * Return: void + */ +static inline void up_read_kernfs_rwsem(struct kernfs_node *kn) +{ + struct rw_semaphore *p_rwsem =3D NULL; + struct rw_semaphore *rwsem =3D kernfs_rwsem_ptr(kn); + + if (kn->unlock_parent) { + kn->unlock_parent =3D 0; + p_rwsem =3D kernfs_rwsem_ptr(kn->parent); + if (rwsem > p_rwsem) { + up_read(rwsem); + up_read(p_rwsem); + } else { + up_read(p_rwsem); + up_read(rwsem); + } + kernfs_put(kn->parent); + } else + up_read(rwsem); + + kernfs_put(kn); +} + +static inline void kernfs_swap_rwsems(struct rw_semaphore **array, int i, = int j) +{ + struct rw_semaphore *tmp; + + tmp =3D array[i]; + array[i] =3D array[j]; + array[j] =3D tmp; +} + +static inline void kernfs_sort_rwsems(struct rw_semaphore **array) +{ + if (array[0] > array[1]) + kernfs_swap_rwsems(array, 0, 1); + + if (array[0] > array[2]) + kernfs_swap_rwsems(array, 0, 2); + + if (array[1] > array[2]) + kernfs_swap_rwsems(array, 1, 2); +} + +/** + * down_write_kernfs_rwsem_rename_ns() - take hashed rwsem during + * rename or similar operations. + * + * @kn: kernfs_node of interest + * @current_parent: current parent of kernfs_node of interest + * @new_parent: about to become new parent of kernfs_node + * + * During rename or similar operations the parent of a node changes, + * and this means we will see different parents of a kernfs_node at + * the time of taking and releasing its or its parent's hashed rwsem. + * This function separately takes locks corresponding to node, and + * corresponding to its current and future parents (if needed). + * + * Return: void + */ +static inline void down_write_kernfs_rwsem_rename_ns(struct kernfs_node *k= n, + struct kernfs_node *current_parent, + struct kernfs_node *new_parent) +{ + struct rw_semaphore *array[3]; + + array[0] =3D kernfs_rwsem_ptr(kn); + array[1] =3D kernfs_rwsem_ptr(current_parent); + array[2] =3D kernfs_rwsem_ptr(new_parent); + + if (array[0] =3D=3D array[1] && array[0] =3D=3D array[2]) { + /* All 3 nodes hash to same rwsem */ + down_write_nested(array[0], 0); + } else { + /** + * All 3 nodes are not hashing to the same rwsem, so sort the + * array. + */ + kernfs_sort_rwsems(array); + + if (array[0] =3D=3D array[1] || array[1] =3D=3D array[2]) { + /** + * Two nodes hash to same rwsem, and these + * will occupy consecutive places in array after + * sorting. + */ + down_write_nested(array[0], 0); + down_write_nested(array[2], 1); + } else { + /* All 3 nodes hashe to different rwsems */ + down_write_nested(array[0], 0); + down_write_nested(array[1], 1); + down_write_nested(array[2], 2); + } + } + + kernfs_get(kn); + kernfs_get(current_parent); + kernfs_get(new_parent); +} + +/** + * up_write_kernfs_rwsem_rename_ns() - release hashed rwsem during + * rename or similar operations. + * + * @kn: kernfs_node of interest + * @current_parent: current parent of kernfs_node of interest + * @old_parent: old parent of kernfs_node + * + * During rename or similar operations the parent of a node changes, + * and this means we will see different parents of a kernfs_node at + * the time of taking and releasing its or its parent's hashed rwsem. + * This function separately releases locks corresponding to node, and + * corresponding to its current and old parents (if needed). + * + * Return: void + */ +static inline void up_write_kernfs_rwsem_rename_ns(struct kernfs_node *kn, + struct kernfs_node *current_parent, + struct kernfs_node *old_parent) +{ + struct rw_semaphore *array[3]; + + array[0] =3D kernfs_rwsem_ptr(kn); + array[1] =3D kernfs_rwsem_ptr(current_parent); + array[2] =3D kernfs_rwsem_ptr(old_parent); + + if (array[0] =3D=3D array[1] && array[0] =3D=3D array[2]) { + /* All 3 nodes hash to same rwsem */ + up_write(array[0]); + } else { + /** + * All 3 nodes are not hashing to the same rwsem, so sort the + * array. + */ + kernfs_sort_rwsems(array); + + if (array[0] =3D=3D array[1] || array[1] =3D=3D array[2]) { + /** + * Two nodes hash to same rwsem, and these + * will occupy consecutive places in array after + * sorting. + */ + up_write(array[2]); + up_write(array[0]); + } else { + /* All 3 nodes hashe to different rwsems */ + up_write(array[2]); + up_write(array[1]); + up_write(array[0]); + } + } + + kernfs_put(old_parent); + kernfs_put(current_parent); + kernfs_put(kn); +} + +/** + * down_read_kernfs_rwsem_rename_ns() - take hashed rwsem during + * rename or similar operations. + * + * @kn: kernfs_node of interest + * @current_parent: current parent of kernfs_node of interest + * @new_parent: about to become new parent of kernfs_node + * + * During rename or similar operations the parent of a node changes, + * and this means we will see different parents of a kernfs_node at + * the time of taking and releasing its or its parent's hashed rwsem. + * This function separately takes locks corresponding to node, and + * corresponding to its current and future parents (if needed). + * + * Return: void + */ +static inline void down_read_kernfs_rwsem_rename_ns(struct kernfs_node *kn, + struct kernfs_node *current_parent, + struct kernfs_node *new_parent) +{ + struct rw_semaphore *array[3]; + + array[0] =3D kernfs_rwsem_ptr(kn); + array[1] =3D kernfs_rwsem_ptr(current_parent); + array[2] =3D kernfs_rwsem_ptr(new_parent); + + if (array[0] =3D=3D array[1] && array[0] =3D=3D array[2]) { + /* All 3 nodes hash to same rwsem */ + down_read_nested(array[0], 0); + } else { + /** + * All 3 nodes are not hashing to the same rwsem, so sort the + * array. + */ + kernfs_sort_rwsems(array); + + if (array[0] =3D=3D array[1] || array[1] =3D=3D array[2]) { + /** + * Two nodes hash to same rwsem, and these + * will occupy consecutive places in array after + * sorting. + */ + down_read_nested(array[0], 0); + down_read_nested(array[2], 1); + } else { + /* All 3 nodes hashe to different rwsems */ + down_read_nested(array[0], 0); + down_read_nested(array[1], 1); + down_read_nested(array[2], 2); + } + } + + kernfs_get(kn); + kernfs_get(current_parent); + kernfs_get(new_parent); +} + +/** + * up_read_kernfs_rwsem_rename_ns() - release hashed rwsem during + * rename or similar operations. + * + * @kn: kernfs_node of interest + * @current_parent: current parent of kernfs_node of interest + * @old_parent: old parent of kernfs_node + * + * During rename or similar operations the parent of a node changes, + * and this means we will see different parents of a kernfs_node at + * the time of taking and releasing its or its parent's hashed rwsem. + * This function separately releases locks corresponding to node, and + * corresponding to its current and old parents (if needed). + * + * Return: void + */ +static inline void up_read_kernfs_rwsem_rename_ns(struct kernfs_node *kn, + struct kernfs_node *current_parent, + struct kernfs_node *old_parent) +{ + struct rw_semaphore *array[3]; + + array[0] =3D kernfs_rwsem_ptr(kn); + array[1] =3D kernfs_rwsem_ptr(current_parent); + array[2] =3D kernfs_rwsem_ptr(old_parent); + + if (array[0] =3D=3D array[1] && array[0] =3D=3D array[2]) { + /* All 3 nodes hash to same rwsem */ + up_read(array[0]); + } else { + /** + * All 3 nodes are not hashing to the same rwsem, so sort the + * array. + */ + kernfs_sort_rwsems(array); + + if (array[0] =3D=3D array[1] || array[1] =3D=3D array[2]) { + /** + * Two nodes hash to same rwsem, and these + * will occupy consecutive places in array after + * sorting. + */ + up_read(array[2]); + up_read(array[0]); + } else { + /* All 3 nodes hashe to different rwsems */ + up_read(array[2]); + up_read(array[1]); + up_read(array[0]); + } + } + + kernfs_put(old_parent); + kernfs_put(current_parent); + kernfs_put(kn); +} + #endif /* __KERNFS_INTERNAL_H */ diff --git a/fs/kernfs/mount.c b/fs/kernfs/mount.c index d35142226c340..d28f8a3eeb215 100644 --- a/fs/kernfs/mount.c +++ b/fs/kernfs/mount.c @@ -398,6 +398,7 @@ void __init kernfs_lock_init(void) for (count =3D 0; count < NR_KERNFS_LOCKS; count++) { mutex_init(&kernfs_locks->open_file_mutex[count].lock); spin_lock_init(&kernfs_locks->open_node_locks[count].lock); + init_rwsem(&kernfs_locks->kernfs_rwsem[count]); } } =20 diff --git a/include/linux/kernfs.h b/include/linux/kernfs.h index 84653c609a5c0..8451f153b2291 100644 --- a/include/linux/kernfs.h +++ b/include/linux/kernfs.h @@ -96,6 +96,7 @@ struct kernfs_open_node_lock { struct kernfs_global_locks { struct kernfs_open_file_mutex open_file_mutex[NR_KERNFS_LOCKS]; struct kernfs_open_node_lock open_node_locks[NR_KERNFS_LOCKS]; + struct rw_semaphore kernfs_rwsem[NR_KERNFS_LOCKS]; }; =20 enum kernfs_node_type { @@ -206,6 +207,7 @@ struct kernfs_node { */ struct kernfs_node *parent; const char *name; + u8 unlock_parent; /* release parent's rwsem */ =20 struct rb_node rb; =20 --=20 2.30.2 From nobody Mon Jun 29 07:49:26 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5110AC433FE for ; Mon, 14 Feb 2022 12:04:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352501AbiBNMEc (ORCPT ); Mon, 14 Feb 2022 07:04:32 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:33576 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352413AbiBNMEY (ORCPT ); Mon, 14 Feb 2022 07:04:24 -0500 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 95AE72ACB for ; Mon, 14 Feb 2022 04:03:53 -0800 (PST) Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 21EBFZAa003324; Mon, 14 Feb 2022 12:03:50 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=6ZbqituaPBbpmGQTIuBKBgUjl94uAwBM2fLOWqyDrQU=; b=XiHUkttW0Sx1NJtqa1IutiU/UkTzWD4IiKZ9NZ+dbSbmT/WqQPLd90+2tnEDIRH7hi+8 zE/vBlcqpN8GeCDy9ZCUmjHhkk1YXTPyQcbAG3mVoDHKam0BfWSKf9YNID9l2GU4dauU TdUNEezdL4CPFXpwigmUVcQeBNNThL5d3xf8DPvtScE2o6RHBY1y2pWAahDMRlpCxAa6 f0FFtfxWE5xmpx0iKB7ZJlPPt1TyWYEPXUlw8WctWLNXruDbmtf5Cblq5Qwe8cLtWjDY oboCGC8Hy6CGLes4JPzwBACJ7nUD+w2oZow7szVoYbHzoqOLnHRA8OEj/1cxaVm9SLkw Ow== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by mx0b-00069f02.pphosted.com with ESMTP id 3e63ad4872-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 14 Feb 2022 12:03:49 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 21EBuRcg176470; Mon, 14 Feb 2022 12:03:49 GMT Received: from nam10-bn7-obe.outbound.protection.outlook.com (mail-bn7nam10lp2109.outbound.protection.outlook.com [104.47.70.109]) by aserp3030.oracle.com with ESMTP id 3e62xd07mf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 14 Feb 2022 12:03:48 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=R4fPlk2qJBdY2mdBDm5h1TF0Jy5ZhEeEnHX9arB/Mg5DG67LkbRKOHLtiJ2YsztLKfPuNNeXt6ctW66zlcRPGw7NVmEldtGzgvw5VWpmLgLUCN1liR9macZtpapobpFrKJwQuDunVVZCRsLStTL4MSN5aEdH1t0+ZLHFH69OkzYRx7KfjLgvaDRlqsnOeMLGMDoC5+uKI39fbo2LrXQVy2ltXmz9ziwaDkaeCqHr/SsPNuK6LFqkZT3D9tZqmglDSRWstYv7kAKrq6zyL46uZ6u8UNMrzH1CI32Cq+yP/0R8hUH9lUZ3qlR7ZbCEEEjuV4thbKTz/sJDd3z18qN0+w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=6ZbqituaPBbpmGQTIuBKBgUjl94uAwBM2fLOWqyDrQU=; b=YUmLFbt2TeicbAhPh4WqkTCdeJEmofU+TQDuzOO5Iqr7KAjAN4dVGKBqpgFnytJdDZky/Fonalbi/zOtdVC19rocw4pHMmt9Al4e+6zbS9JfDKFYEEuVyvMLfsot2FsrZLHun1G54eCWTH4XqHTJx/IjkIu4Oum3an+N7aiyUmZAizTp/IHmgLMI3rs/WJlkZYXZzG7yRg3JcB8A0Zgeudrp1GSdBrKXEphcsfVh+1oXRzzAunqFc9DP9LhTe9b/Niv96wt2u5ZhFwWRWBCt2lN9i0S2TKWwZ1X37dutqK10x3X5wF4Fdy2nCl0VdWVI6CUNsJ6Nt+BdqdO+j4BL8g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6ZbqituaPBbpmGQTIuBKBgUjl94uAwBM2fLOWqyDrQU=; b=V0uVd4X/s7yXJYpaHJWw1sN9fosbEA1YsAtVwfCiTQ4/ApRLCUJ+P1Xtf1JP54CDwy58GhKoSdmqjio6DO1O9LiPUJYS3zE9SmGaM13jNg/x/CAM7ewMQYwx8ySIbk2LKO7TaxczYMTiZjjJD5aI1PkorExNDovQst8t8GXPgN4= Received: from CO1PR10MB4468.namprd10.prod.outlook.com (2603:10b6:303:6c::24) by MN2PR10MB4191.namprd10.prod.outlook.com (2603:10b6:208:1d1::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4975.11; Mon, 14 Feb 2022 12:03:46 +0000 Received: from CO1PR10MB4468.namprd10.prod.outlook.com ([fe80::a5d1:ed4:5ab6:e9b1]) by CO1PR10MB4468.namprd10.prod.outlook.com ([fe80::a5d1:ed4:5ab6:e9b1%4]) with mapi id 15.20.4975.019; Mon, 14 Feb 2022 12:03:46 +0000 From: Imran Khan To: tj@kernel.org, gregkh@linuxfoundation.org Cc: linux-kernel@vger.kernel.org Subject: [PATCH v6 7/7] kernfs: Replace per-fs rwsem with hashed ones. Date: Mon, 14 Feb 2022 23:03:22 +1100 Message-Id: <20220214120322.2402628-8-imran.f.khan@oracle.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220214120322.2402628-1-imran.f.khan@oracle.com> References: <20220214120322.2402628-1-imran.f.khan@oracle.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SY2PR01CA0016.ausprd01.prod.outlook.com (2603:10c6:1:14::28) To CO1PR10MB4468.namprd10.prod.outlook.com (2603:10b6:303:6c::24) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 11ef8a10-83a4-45e1-08c1-08d9efb20dc7 X-MS-TrafficTypeDiagnostic: MN2PR10MB4191:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6430; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 5IwbQQnAY+jKVUSC1ZmxoSw9P2arF/IiON6vD/q+GGujLqj0seToLriFQ1cYqUGciD4PFtJPgrNwnzVWcl5L3wvKCR75ZcVWQEbv1Ri3AyrH7GV0wEMu+689gxuQ/eGPC+8RLjc8Z4Poc+Z/L0rH2WgnAqAa+MHGiAqJZnxFR09L7E1sXvBTx+7dPSjAeyj/XyJZrge7qz7UVREBLPk2Qi1fNjnikhhkDbcXo+G0tRWDIbhvU5cYBPPI6F9u9jxLlVgP0ekstFB8MxfZ5uFPYMdSJs/Ept2RYtZ3aYAh5kWhMCH2D6GDsNMHrYYID4UTaxr0Eq2ddUk47YuLmQmd5joKI8NeuzMqv5uwN2AqS/R6g4KUMtPXIxJLpqaJWBBnwMAaaqfFGagU4BfHZP95PPJUK2aglmpbEJEjuMKtoUXgRuR6hqT/ZfWo7XBb/nmRP/ysi2Q8A064dr6ewxS4Im24kdHY/sWyJPTR2QYua85VcCD9MAAIrgBnr40fp0Molo5pRVhNaDopyoq1J/dRXaPnTBu1tRzPshl32DBhQS+n3vwgXn8tfP21qOu1n0blH7mGlgEsegFeJPBxsN0FSntoPdl3hTbT//DfwvX8lJGVB+IPqQeT4LBa/2cwUaEAuQQkxYmGwCCHu0XTOELXqrkQp20nh4CbvHM+8Fay4Cic1JnW03H8ksP2msZjFKy8MRt/utD2MWgpgGxHNSd18w== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO1PR10MB4468.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(86362001)(2616005)(103116003)(5660300002)(38350700002)(1076003)(26005)(186003)(6512007)(38100700002)(2906002)(6666004)(6506007)(8676002)(36756003)(66946007)(30864003)(66556008)(66476007)(316002)(6486002)(83380400001)(8936002)(4326008)(52116002)(508600001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?VpqSr8n04e44g0FnQRe4Vc42hvHVAaCgSpWCcp+T2hoUOGkPPLfIHVFY0hGh?= =?us-ascii?Q?hijJRpOPCNRclSs3v0sH+Rc8GYPc4UPixYxSEhtA7aocD0K/g6xeFogpHQGj?= =?us-ascii?Q?Zt/LmdXJiEbPkpQ6oaPn5onF42FCQ8sW6N5pmx2ZoKvUcGh8FAnjVPivteI6?= =?us-ascii?Q?2F/YIrO4mw8lJU4QOCrs6IlYS6SY2p/PFUpksOfdmACVYVTx9LZBiV4j5bUp?= =?us-ascii?Q?/YlZR5Xg3/D3xWNgYQUvu0uXN8vVAtcpYOI11xZlg2QWty7Xdl+dtH7aqyZK?= =?us-ascii?Q?UISpwUz8OI0wljqrBmN6MTppQ+bmVErvJ+6ye9BMMsrP2sFbZKlfR/QsCOMa?= =?us-ascii?Q?HfqgFZ2QjJpFyCX5m/SOnPzXxbyi8PxRXeLv/pdx65IWFuNY8tMeWLPbaroD?= =?us-ascii?Q?ZjOXgIBohoCQJzubMXlnAOjA8SGhfKy+FnEMQ75UqS8BWSbJfF8zRhTd6v2z?= =?us-ascii?Q?miq0M/D/AaGRYFKVF7jjUhUYf2oFmUVEmsZ55bFL41k76XmMqwL1JX93n36e?= =?us-ascii?Q?Z7bB1Dkchy45rmZ2JF7pzaQH5IihC5qWYIkTvVSfneUrLS4HA14Sny2mGL/Q?= =?us-ascii?Q?uiylonXnfVLJPX7q9TIXpQBtsKtyf8Kxp0/MpXw9OcL0UQFOYHXcXwTfep1l?= =?us-ascii?Q?kPxrlvlILGJTHFdkL6MB4gfJO+usVvf3/Rh2jO1fJaXLSjqJ6aKa37vxY+ei?= =?us-ascii?Q?DYeyDSDKY27cMW5Ltx9kNfnozXM3PV78YgaBgIe4uGaYsteFxorgqM9if0aU?= =?us-ascii?Q?lKZ1jD85wP2vdOfQjvXNTWF3+pB5kz+FN471m2OjBp9aeAZND8Bg6BXlNUx0?= =?us-ascii?Q?lrUuxW5aFb7iwSfK0Gicr/nUtauYfjuCErRpay1DSboJbPKKSl2yrUF1PNZg?= =?us-ascii?Q?zPzOxreOoaL/6r1d4UqGEen9dThyJ1Vp46QLBI9+yTUCWolnplXFS3TTQwac?= =?us-ascii?Q?gTQmJrdjqYDdccgmdSlq0XAgLO3/3Hpnf2oXEhs6pG2YdVCTpVvDXzGXOtRg?= =?us-ascii?Q?UbgUhCXoEv9LSeCe5DoXH2nAiWBS3pKmnxaaGUnhOpLNO4kKG3MbZolifZ33?= =?us-ascii?Q?qkhs2frX2VMpuezWpS+8K8jvyspEoNRp5Zf8hz3/ZC2qtryc7WhsO3oZH8/R?= =?us-ascii?Q?r2lkU5UlcyAM4mRVC1nJfqscvylcwXs20h2WqpMD321KUqAXwO0jdYripQvG?= =?us-ascii?Q?IFjUUO2+5Pa7+cBM2S/9eWF4EjPbo6ONxmqmv1URE1FjZhvN4JneCHKu+LVi?= =?us-ascii?Q?wkofqMQBjxEXbQVe8LTYueg25slijswlv01K1I8Kx+9CK75HLz8Xef+M6uon?= =?us-ascii?Q?XJSj9VVwtaVEM1CvPrJNizQNoE9prn0SZORiNB/YnIS9dl2EvCOuLKcZvHyu?= =?us-ascii?Q?SFaVZW4QiJujCtkCP/RdEzNUliiTITfeX1ErDjP7zZXtcp70htor8YBB/dQM?= =?us-ascii?Q?uohs+XNIr0cv7tz752I80sQ/PbxfL/REUVSBtv8dU1jSD/y6HXPTEHANq1JB?= =?us-ascii?Q?Vg1Kz3ZSO+VSNQp7RAaaY3/3vj4NwXRwGrBT/Q8i9Gq80de2IjDtQFCjf0qV?= =?us-ascii?Q?2jXakVmIQPRDRBrCjm9bk7icL6o8DWGIXpcNBjU9VPZGTchWteuK7pV5m1ue?= =?us-ascii?Q?PasgA7WRfCfzQBzf+R9CPlw=3D?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 11ef8a10-83a4-45e1-08c1-08d9efb20dc7 X-MS-Exchange-CrossTenant-AuthSource: CO1PR10MB4468.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Feb 2022 12:03:46.6968 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 5/Bl0OZPMR5iQ6z/FWrLjNLE892gOr9KGPKVbOyfKd9WTRvBA6m/VH7d3MoCL1m5grTknMgPH87GK4FO/jbvvw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB4191 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10257 signatures=673431 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 phishscore=0 bulkscore=0 malwarescore=0 adultscore=0 spamscore=0 mlxlogscore=999 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2201110000 definitions=main-2202140073 X-Proofpoint-GUID: 2iCrymktlswEzhwZCsBOMFUsulxt9iBh X-Proofpoint-ORIG-GUID: 2iCrymktlswEzhwZCsBOMFUsulxt9iBh Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Remove per-fs rwsem, using hashed rwsem and corresponding interface introduced in previous patch. Also as we are removing use of per-fs rwsem, we no longer need lockdep checkings under kernfs_active. Wherever needed lockdep checkings can be made on a per-node basis using interfaces provided in previous patch. Signed-off-by: Imran Khan --- fs/kernfs/dir.c | 130 ++++++++++++++++++------------------ fs/kernfs/file.c | 4 +- fs/kernfs/inode.c | 22 +++--- fs/kernfs/kernfs-internal.h | 1 - fs/kernfs/mount.c | 5 +- fs/kernfs/symlink.c | 5 +- 6 files changed, 80 insertions(+), 87 deletions(-) diff --git a/fs/kernfs/dir.c b/fs/kernfs/dir.c index 0dac58f8091c9..42404f8bc6acd 100644 --- a/fs/kernfs/dir.c +++ b/fs/kernfs/dir.c @@ -449,21 +449,22 @@ void kernfs_put_active(struct kernfs_node *kn) /** * kernfs_drain - drain kernfs_node * @kn: kernfs_node to drain + * @anc: ancestor of kernfs_node to drain * * Drain existing usages and nuke all existing mmaps of @kn. Mutiple * removers may invoke this function concurrently on @kn and all will * return after draining is complete. */ -static void kernfs_drain(struct kernfs_node *kn) - __releases(&kernfs_root(kn)->kernfs_rwsem) - __acquires(&kernfs_root(kn)->kernfs_rwsem) +static void kernfs_drain(struct kernfs_node *kn, struct kernfs_node *anc) + __releases(kernfs_rwsem_ptr(anc)) + __acquires(kernfs_rwsem_ptr(anc)) { struct kernfs_root *root =3D kernfs_root(kn); =20 - lockdep_assert_held_write(&root->kernfs_rwsem); + kernfs_rwsem_assert_held_write(anc); WARN_ON_ONCE(kernfs_active(kn)); =20 - up_write(&root->kernfs_rwsem); + up_write_kernfs_rwsem(anc); =20 if (kernfs_lockdep(kn)) { rwsem_acquire(&kn->dep_map, 0, 0, _RET_IP_); @@ -482,7 +483,7 @@ static void kernfs_drain(struct kernfs_node *kn) =20 kernfs_drain_open_files(kn); =20 - down_write(&root->kernfs_rwsem); + down_write_kernfs_rwsem(anc, KERNFS_RWSEM_LOCK_SELF); } =20 /** @@ -717,12 +718,16 @@ struct kernfs_node *kernfs_find_and_get_node_by_id(st= ruct kernfs_root *root, int kernfs_add_one(struct kernfs_node *kn) { struct kernfs_node *parent =3D kn->parent; - struct kernfs_root *root =3D kernfs_root(parent); struct kernfs_iattrs *ps_iattr; bool has_ns; int ret; =20 - down_write(&root->kernfs_rwsem); + /** + * The node being added is not active at this point of time and may + * be activated later depending on CREATE_DEACTIVATED flag. So at + * this point of time just locking the parent is enough. + */ + down_write_kernfs_rwsem(parent, KERNFS_RWSEM_LOCK_SELF); =20 ret =3D -EINVAL; has_ns =3D kernfs_ns_enabled(parent); @@ -753,7 +758,7 @@ int kernfs_add_one(struct kernfs_node *kn) ps_iattr->ia_mtime =3D ps_iattr->ia_ctime; } =20 - up_write(&root->kernfs_rwsem); + up_write_kernfs_rwsem(parent); =20 /* * Activate the new node unless CREATE_DEACTIVATED is requested. @@ -767,7 +772,7 @@ int kernfs_add_one(struct kernfs_node *kn) return 0; =20 out_unlock: - up_write(&root->kernfs_rwsem); + up_write_kernfs_rwsem(parent); return ret; } =20 @@ -788,7 +793,7 @@ static struct kernfs_node *kernfs_find_ns(struct kernfs= _node *parent, bool has_ns =3D kernfs_ns_enabled(parent); unsigned int hash; =20 - lockdep_assert_held(&kernfs_root(parent)->kernfs_rwsem); + kernfs_rwsem_assert_held(parent); =20 if (has_ns !=3D (bool)ns) { WARN(1, KERN_WARNING "kernfs: ns %s in '%s' for '%s'\n", @@ -820,7 +825,7 @@ static struct kernfs_node *kernfs_walk_ns(struct kernfs= _node *parent, size_t len; char *p, *name; =20 - lockdep_assert_held_read(&kernfs_root(parent)->kernfs_rwsem); + kernfs_rwsem_assert_held_read(parent); =20 /* grab kernfs_rename_lock to piggy back on kernfs_pr_cont_buf */ spin_lock_irq(&kernfs_rename_lock); @@ -859,12 +864,11 @@ struct kernfs_node *kernfs_find_and_get_ns(struct ker= nfs_node *parent, const char *name, const void *ns) { struct kernfs_node *kn; - struct kernfs_root *root =3D kernfs_root(parent); =20 - down_read(&root->kernfs_rwsem); + down_read_kernfs_rwsem(parent, KERNFS_RWSEM_LOCK_SELF); kn =3D kernfs_find_ns(parent, name, ns); kernfs_get(kn); - up_read(&root->kernfs_rwsem); + up_read_kernfs_rwsem(parent); =20 return kn; } @@ -884,12 +888,11 @@ struct kernfs_node *kernfs_walk_and_get_ns(struct ker= nfs_node *parent, const char *path, const void *ns) { struct kernfs_node *kn; - struct kernfs_root *root =3D kernfs_root(parent); =20 - down_read(&root->kernfs_rwsem); + down_read_kernfs_rwsem(parent, KERNFS_RWSEM_LOCK_SELF); kn =3D kernfs_walk_ns(parent, path, ns); kernfs_get(kn); - up_read(&root->kernfs_rwsem); + up_read_kernfs_rwsem(parent); =20 return kn; } @@ -914,7 +917,6 @@ struct kernfs_root *kernfs_create_root(struct kernfs_sy= scall_ops *scops, return ERR_PTR(-ENOMEM); =20 idr_init(&root->ino_idr); - init_rwsem(&root->kernfs_rwsem); INIT_LIST_HEAD(&root->supers); init_rwsem(&root->supers_rwsem); =20 @@ -1045,7 +1047,6 @@ struct kernfs_node *kernfs_create_empty_dir(struct ke= rnfs_node *parent, static int kernfs_dop_revalidate(struct dentry *dentry, unsigned int flags) { struct kernfs_node *kn; - struct kernfs_root *root; =20 if (flags & LOOKUP_RCU) return -ECHILD; @@ -1061,13 +1062,12 @@ static int kernfs_dop_revalidate(struct dentry *den= try, unsigned int flags) parent =3D kernfs_dentry_node(dentry->d_parent); if (parent) { spin_unlock(&dentry->d_lock); - root =3D kernfs_root(parent); - down_read(&root->kernfs_rwsem); + down_read_kernfs_rwsem(parent, KERNFS_RWSEM_LOCK_SELF); if (kernfs_dir_changed(parent, dentry)) { - up_read(&root->kernfs_rwsem); + up_read_kernfs_rwsem(parent); return 0; } - up_read(&root->kernfs_rwsem); + up_read_kernfs_rwsem(parent); } else spin_unlock(&dentry->d_lock); =20 @@ -1078,8 +1078,7 @@ static int kernfs_dop_revalidate(struct dentry *dentr= y, unsigned int flags) } =20 kn =3D kernfs_dentry_node(dentry); - root =3D kernfs_root(kn); - down_read(&root->kernfs_rwsem); + down_read_kernfs_rwsem(kn, KERNFS_RWSEM_LOCK_SELF); =20 /* The kernfs node has been deactivated */ if (!kernfs_active(kn)) @@ -1098,10 +1097,10 @@ static int kernfs_dop_revalidate(struct dentry *den= try, unsigned int flags) kernfs_info(dentry->d_sb)->ns !=3D kn->ns) goto out_bad; =20 - up_read(&root->kernfs_rwsem); + up_read_kernfs_rwsem(kn); return 1; out_bad: - up_read(&root->kernfs_rwsem); + up_read_kernfs_rwsem(kn); return 0; } =20 @@ -1115,28 +1114,29 @@ static struct dentry *kernfs_iop_lookup(struct inod= e *dir, { struct kernfs_node *parent =3D dir->i_private; struct kernfs_node *kn; - struct kernfs_root *root; struct inode *inode =3D NULL; const void *ns =3D NULL; =20 - root =3D kernfs_root(parent); - down_read(&root->kernfs_rwsem); + down_read_kernfs_rwsem(parent, KERNFS_RWSEM_LOCK_SELF); if (kernfs_ns_enabled(parent)) ns =3D kernfs_info(dir->i_sb)->ns; =20 kn =3D kernfs_find_ns(parent, dentry->d_name.name, ns); + up_read_kernfs_rwsem(parent); /* attach dentry and inode */ if (kn) { /* Inactive nodes are invisible to the VFS so don't * create a negative. */ + down_read_kernfs_rwsem(kn, KERNFS_RWSEM_LOCK_SELF); if (!kernfs_active(kn)) { - up_read(&root->kernfs_rwsem); + up_read_kernfs_rwsem(kn); return NULL; } inode =3D kernfs_get_inode(dir->i_sb, kn); if (!inode) inode =3D ERR_PTR(-ENOMEM); + up_read_kernfs_rwsem(kn); } /* * Needed for negative dentry validation. @@ -1144,9 +1144,10 @@ static struct dentry *kernfs_iop_lookup(struct inode= *dir, * or transforms from positive dentry in dentry_unlink_inode() * called from vfs_rmdir(). */ + down_read_kernfs_rwsem(parent, KERNFS_RWSEM_LOCK_SELF); if (!IS_ERR(inode)) kernfs_set_rev(parent, dentry); - up_read(&root->kernfs_rwsem); + up_read_kernfs_rwsem(parent); =20 /* instantiate and hash (possibly negative) dentry */ return d_splice_alias(inode, dentry); @@ -1269,7 +1270,7 @@ static struct kernfs_node *kernfs_next_descendant_pos= t(struct kernfs_node *pos, { struct rb_node *rbn; =20 - lockdep_assert_held_write(&kernfs_root(root)->kernfs_rwsem); + kernfs_rwsem_assert_held_write(root); =20 /* if first iteration, visit leftmost descendant which may be root */ if (!pos) @@ -1304,9 +1305,8 @@ static struct kernfs_node *kernfs_next_descendant_pos= t(struct kernfs_node *pos, void kernfs_activate(struct kernfs_node *kn) { struct kernfs_node *pos; - struct kernfs_root *root =3D kernfs_root(kn); =20 - down_write(&root->kernfs_rwsem); + down_write_kernfs_rwsem(kn, KERNFS_RWSEM_LOCK_SELF); =20 pos =3D NULL; while ((pos =3D kernfs_next_descendant_post(pos, kn))) { @@ -1320,15 +1320,13 @@ void kernfs_activate(struct kernfs_node *kn) pos->flags |=3D KERNFS_ACTIVATED; } =20 - up_write(&root->kernfs_rwsem); + up_write_kernfs_rwsem(kn); } =20 static void __kernfs_remove(struct kernfs_node *kn) { struct kernfs_node *pos; =20 - lockdep_assert_held_write(&kernfs_root(kn)->kernfs_rwsem); - /* * Short-circuit if non-root @kn has already finished removal. * This is for kernfs_remove_self() which plays with active ref @@ -1341,12 +1339,16 @@ static void __kernfs_remove(struct kernfs_node *kn) =20 /* prevent any new usage under @kn by deactivating all nodes */ pos =3D NULL; + + down_write_kernfs_rwsem(kn, KERNFS_RWSEM_LOCK_SELF); while ((pos =3D kernfs_next_descendant_post(pos, kn))) if (kernfs_active(pos)) atomic_add(KN_DEACTIVATED_BIAS, &pos->active); + up_write_kernfs_rwsem(kn); =20 /* deactivate and unlink the subtree node-by-node */ do { + down_write_kernfs_rwsem(kn, KERNFS_RWSEM_LOCK_SELF); pos =3D kernfs_leftmost_descendant(kn); =20 /* @@ -1364,10 +1366,15 @@ static void __kernfs_remove(struct kernfs_node *kn) * error paths without worrying about draining. */ if (kn->flags & KERNFS_ACTIVATED) - kernfs_drain(pos); + kernfs_drain(pos, kn); else WARN_ON_ONCE(atomic_read(&kn->active) !=3D KN_DEACTIVATED_BIAS); =20 + up_write_kernfs_rwsem(kn); + + if (pos->parent) + down_write_kernfs_rwsem(pos->parent, KERNFS_RWSEM_LOCK_SELF); + /* * kernfs_unlink_sibling() succeeds once per node. Use it * to decide who's responsible for cleanups. @@ -1385,6 +1392,9 @@ static void __kernfs_remove(struct kernfs_node *kn) kernfs_put(pos); } =20 + if (pos->parent) + up_write_kernfs_rwsem(pos->parent); + kernfs_put(pos); } while (pos !=3D kn); } @@ -1397,11 +1407,7 @@ static void __kernfs_remove(struct kernfs_node *kn) */ void kernfs_remove(struct kernfs_node *kn) { - struct kernfs_root *root =3D kernfs_root(kn); - - down_write(&root->kernfs_rwsem); __kernfs_remove(kn); - up_write(&root->kernfs_rwsem); } =20 /** @@ -1487,9 +1493,8 @@ void kernfs_unbreak_active_protection(struct kernfs_n= ode *kn) bool kernfs_remove_self(struct kernfs_node *kn) { bool ret; - struct kernfs_root *root =3D kernfs_root(kn); =20 - down_write(&root->kernfs_rwsem); + down_write_kernfs_rwsem(kn, KERNFS_RWSEM_LOCK_SELF); kernfs_break_active_protection(kn); =20 /* @@ -1503,9 +1508,11 @@ bool kernfs_remove_self(struct kernfs_node *kn) */ if (!(kn->flags & KERNFS_SUICIDAL)) { kn->flags |=3D KERNFS_SUICIDAL; + up_write_kernfs_rwsem(kn); __kernfs_remove(kn); kn->flags |=3D KERNFS_SUICIDED; ret =3D true; + down_write_kernfs_rwsem(kn, KERNFS_RWSEM_LOCK_SELF); } else { wait_queue_head_t *waitq =3D &kernfs_root(kn)->deactivate_waitq; DEFINE_WAIT(wait); @@ -1517,9 +1524,9 @@ bool kernfs_remove_self(struct kernfs_node *kn) atomic_read(&kn->active) =3D=3D KN_DEACTIVATED_BIAS) break; =20 - up_write(&root->kernfs_rwsem); + up_write_kernfs_rwsem(kn); schedule(); - down_write(&root->kernfs_rwsem); + down_write_kernfs_rwsem(kn, KERNFS_RWSEM_LOCK_SELF_AND_PARENT); } finish_wait(waitq, &wait); WARN_ON_ONCE(!RB_EMPTY_NODE(&kn->rb)); @@ -1532,7 +1539,7 @@ bool kernfs_remove_self(struct kernfs_node *kn) */ kernfs_unbreak_active_protection(kn); =20 - up_write(&root->kernfs_rwsem); + up_write_kernfs_rwsem(kn); return ret; } =20 @@ -1549,7 +1556,6 @@ int kernfs_remove_by_name_ns(struct kernfs_node *pare= nt, const char *name, const void *ns) { struct kernfs_node *kn; - struct kernfs_root *root; =20 if (!parent) { WARN(1, KERN_WARNING "kernfs: can not remove '%s', no directory\n", @@ -1557,15 +1563,15 @@ int kernfs_remove_by_name_ns(struct kernfs_node *pa= rent, const char *name, return -ENOENT; } =20 - root =3D kernfs_root(parent); - down_write(&root->kernfs_rwsem); + down_write_kernfs_rwsem(parent, KERNFS_RWSEM_LOCK_SELF); =20 kn =3D kernfs_find_ns(parent, name, ns); + + up_write_kernfs_rwsem(parent); + if (kn) __kernfs_remove(kn); =20 - up_write(&root->kernfs_rwsem); - if (kn) return 0; else @@ -1583,7 +1589,6 @@ int kernfs_rename_ns(struct kernfs_node *kn, struct k= ernfs_node *new_parent, const char *new_name, const void *new_ns) { struct kernfs_node *old_parent; - struct kernfs_root *root; const char *old_name =3D NULL; int error; =20 @@ -1591,8 +1596,7 @@ int kernfs_rename_ns(struct kernfs_node *kn, struct k= ernfs_node *new_parent, if (!kn->parent) return -EINVAL; =20 - root =3D kernfs_root(kn); - down_write(&root->kernfs_rwsem); + down_write_kernfs_rwsem_rename_ns(kn, kn->parent, new_parent); =20 error =3D -ENOENT; if (!kernfs_active(kn) || !kernfs_active(new_parent) || @@ -1646,7 +1650,7 @@ int kernfs_rename_ns(struct kernfs_node *kn, struct k= ernfs_node *new_parent, =20 error =3D 0; out: - up_write(&root->kernfs_rwsem); + up_write_kernfs_rwsem_rename_ns(kn, new_parent, old_parent); return error; } =20 @@ -1717,14 +1721,12 @@ static int kernfs_fop_readdir(struct file *file, st= ruct dir_context *ctx) struct dentry *dentry =3D file->f_path.dentry; struct kernfs_node *parent =3D kernfs_dentry_node(dentry); struct kernfs_node *pos =3D file->private_data; - struct kernfs_root *root; const void *ns =3D NULL; =20 if (!dir_emit_dots(file, ctx)) return 0; =20 - root =3D kernfs_root(parent); - down_read(&root->kernfs_rwsem); + down_read_kernfs_rwsem(parent, KERNFS_RWSEM_LOCK_SELF); =20 if (kernfs_ns_enabled(parent)) ns =3D kernfs_info(dentry->d_sb)->ns; @@ -1741,12 +1743,12 @@ static int kernfs_fop_readdir(struct file *file, st= ruct dir_context *ctx) file->private_data =3D pos; kernfs_get(pos); =20 - up_read(&root->kernfs_rwsem); + up_read_kernfs_rwsem(parent); if (!dir_emit(ctx, name, len, ino, type)) return 0; - down_read(&root->kernfs_rwsem); + down_read_kernfs_rwsem(parent, KERNFS_RWSEM_LOCK_SELF); } - up_read(&root->kernfs_rwsem); + up_read_kernfs_rwsem(parent); file->private_data =3D NULL; ctx->pos =3D INT_MAX; return 0; diff --git a/fs/kernfs/file.c b/fs/kernfs/file.c index af2046bc63aa1..03d8f0d087cb8 100644 --- a/fs/kernfs/file.c +++ b/fs/kernfs/file.c @@ -857,7 +857,7 @@ static void kernfs_notify_workfn(struct work_struct *wo= rk) =20 root =3D kernfs_root(kn); /* kick fsnotify */ - down_write(&root->kernfs_rwsem); + down_write_kernfs_rwsem(kn, KERNFS_RWSEM_LOCK_SELF); =20 down_write(&root->supers_rwsem); list_for_each_entry(info, &kernfs_root(kn)->supers, node) { @@ -897,7 +897,7 @@ static void kernfs_notify_workfn(struct work_struct *wo= rk) } up_write(&root->supers_rwsem); =20 - up_write(&root->kernfs_rwsem); + up_write_kernfs_rwsem(kn); kernfs_put(kn); goto repeat; } diff --git a/fs/kernfs/inode.c b/fs/kernfs/inode.c index 3d783d80f5daa..4de65f9c21d85 100644 --- a/fs/kernfs/inode.c +++ b/fs/kernfs/inode.c @@ -99,11 +99,10 @@ int __kernfs_setattr(struct kernfs_node *kn, const stru= ct iattr *iattr) int kernfs_setattr(struct kernfs_node *kn, const struct iattr *iattr) { int ret; - struct kernfs_root *root =3D kernfs_root(kn); =20 - down_write(&root->kernfs_rwsem); + down_write_kernfs_rwsem(kn, KERNFS_RWSEM_LOCK_SELF); ret =3D __kernfs_setattr(kn, iattr); - up_write(&root->kernfs_rwsem); + up_write_kernfs_rwsem(kn); return ret; } =20 @@ -112,14 +111,12 @@ int kernfs_iop_setattr(struct user_namespace *mnt_use= rns, struct dentry *dentry, { struct inode *inode =3D d_inode(dentry); struct kernfs_node *kn =3D inode->i_private; - struct kernfs_root *root; int error; =20 if (!kn) return -EINVAL; =20 - root =3D kernfs_root(kn); - down_write(&root->kernfs_rwsem); + down_write_kernfs_rwsem(kn, KERNFS_RWSEM_LOCK_SELF); error =3D setattr_prepare(&init_user_ns, dentry, iattr); if (error) goto out; @@ -132,7 +129,7 @@ int kernfs_iop_setattr(struct user_namespace *mnt_usern= s, struct dentry *dentry, setattr_copy(&init_user_ns, inode, iattr); =20 out: - up_write(&root->kernfs_rwsem); + up_write_kernfs_rwsem(kn); return error; } =20 @@ -187,14 +184,13 @@ int kernfs_iop_getattr(struct user_namespace *mnt_use= rns, { struct inode *inode =3D d_inode(path->dentry); struct kernfs_node *kn =3D inode->i_private; - struct kernfs_root *root =3D kernfs_root(kn); =20 - down_read(&root->kernfs_rwsem); + down_read_kernfs_rwsem(kn, KERNFS_RWSEM_LOCK_SELF); spin_lock(&inode->i_lock); kernfs_refresh_inode(kn, inode); generic_fillattr(&init_user_ns, inode, stat); spin_unlock(&inode->i_lock); - up_read(&root->kernfs_rwsem); + up_read_kernfs_rwsem(kn); =20 return 0; } @@ -278,21 +274,19 @@ int kernfs_iop_permission(struct user_namespace *mnt_= userns, struct inode *inode, int mask) { struct kernfs_node *kn; - struct kernfs_root *root; int ret; =20 if (mask & MAY_NOT_BLOCK) return -ECHILD; =20 kn =3D inode->i_private; - root =3D kernfs_root(kn); =20 - down_read(&root->kernfs_rwsem); + down_read_kernfs_rwsem(kn, KERNFS_RWSEM_LOCK_SELF); spin_lock(&inode->i_lock); kernfs_refresh_inode(kn, inode); ret =3D generic_permission(&init_user_ns, inode, mask); spin_unlock(&inode->i_lock); - up_read(&root->kernfs_rwsem); + up_read_kernfs_rwsem(kn); =20 return ret; } diff --git a/fs/kernfs/kernfs-internal.h b/fs/kernfs/kernfs-internal.h index ba89de378f240..f19b180557559 100644 --- a/fs/kernfs/kernfs-internal.h +++ b/fs/kernfs/kernfs-internal.h @@ -133,7 +133,6 @@ int __kernfs_setattr(struct kernfs_node *kn, const stru= ct iattr *iattr); /* * dir.c */ -extern struct rw_semaphore kernfs_rwsem; extern const struct dentry_operations kernfs_dops; extern const struct file_operations kernfs_dir_fops; extern const struct inode_operations kernfs_dir_iops; diff --git a/fs/kernfs/mount.c b/fs/kernfs/mount.c index d28f8a3eeb215..2816750f798e2 100644 --- a/fs/kernfs/mount.c +++ b/fs/kernfs/mount.c @@ -237,7 +237,6 @@ struct dentry *kernfs_node_dentry(struct kernfs_node *k= n, static int kernfs_fill_super(struct super_block *sb, struct kernfs_fs_cont= ext *kfc) { struct kernfs_super_info *info =3D kernfs_info(sb); - struct kernfs_root *kf_root =3D kfc->root; struct inode *inode; struct dentry *root; =20 @@ -257,9 +256,9 @@ static int kernfs_fill_super(struct super_block *sb, st= ruct kernfs_fs_context *k sb->s_shrink.seeks =3D 0; =20 /* get root inode, initialize and unlock it */ - down_read(&kf_root->kernfs_rwsem); + down_read_kernfs_rwsem(info->root->kn, KERNFS_RWSEM_LOCK_SELF); inode =3D kernfs_get_inode(sb, info->root->kn); - up_read(&kf_root->kernfs_rwsem); + up_read_kernfs_rwsem(info->root->kn); if (!inode) { pr_debug("kernfs: could not get root inode\n"); return -ENOMEM; diff --git a/fs/kernfs/symlink.c b/fs/kernfs/symlink.c index 0ab13824822f7..24d0f64460bda 100644 --- a/fs/kernfs/symlink.c +++ b/fs/kernfs/symlink.c @@ -113,12 +113,11 @@ static int kernfs_getlink(struct inode *inode, char *= path) struct kernfs_node *kn =3D inode->i_private; struct kernfs_node *parent =3D kn->parent; struct kernfs_node *target =3D kn->symlink.target_kn; - struct kernfs_root *root =3D kernfs_root(parent); int error; =20 - down_read(&root->kernfs_rwsem); + down_read_kernfs_rwsem(parent, KERNFS_RWSEM_LOCK_SELF); error =3D kernfs_get_target_path(parent, target, path); - up_read(&root->kernfs_rwsem); + up_read_kernfs_rwsem(parent); =20 return error; } --=20 2.30.2