From nobody Wed Apr 29 12:25:12 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 73E41C43334 for ; Tue, 14 Jun 2022 03:44:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237628AbiFNDoL (ORCPT ); Mon, 13 Jun 2022 23:44:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41358 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231129AbiFNDoI (ORCPT ); Mon, 13 Jun 2022 23:44:08 -0400 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A28E938BCA for ; Mon, 13 Jun 2022 20:44:06 -0700 (PDT) Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 25E0xTtX027234; Tue, 14 Jun 2022 02:08:57 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=OfSN8q+CnVo23NDqHh0S/Z9/fDUvVHplbpLyWz8PKi8=; b=p4Al/4pvgGmIuhONO/0LbTihf9lwgs4xRoA3lHwWn4cLIlaHaY8wtZVT9zhDi7bmAAMx rDtd+5jVwozGsMuzWOMukc/V/hfx+yJvg5WKu8/vGdNoXOMJ2LKFjYDMmgf3RbtwxbO0 OyGUCnhuBKzynsIKt5M5uj73T9D7Fl31FN8AfWjxgBryIJ6SVwhGzi8HlI1EAWd6/Ry6 Xt5BZltfAzJioW3q28mLhUuxGlHcLQF3FOuSOIVIsGt+v3V9wPLIgAccCepioBeFLHbP ooGC5AjN965hpbjMZqvAIFe9u0qvoXzPcByqxFyDuwkBbJ1+AhT8c8bZZofCwpfGfBc6 /w== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3gmjns4j5h-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 14 Jun 2022 02:08:57 +0000 Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 25E20I5m025507; Tue, 14 Jun 2022 02:08:56 GMT Received: from nam11-co1-obe.outbound.protection.outlook.com (mail-co1nam11lp2175.outbound.protection.outlook.com [104.47.56.175]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com with ESMTP id 3gpgyerbhm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 14 Jun 2022 02:08:56 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=irDVzOQl2q3tZlQnbC/piQliljout/W9gyMu3SoXAK18oK6DG65yBggOkgdU3U79v4cRDa54odnBfEK9udeEcH34XAlfsS+w12d205GpPTBv3lkhesY4L+jC6lbTz3AhHzic7hcz0vw7u6N2sPJ1/4b4OyF+gXQVKu2mn53FgXj4iDgxqBM9zxdA90IzyMP/+f7ANxxFYZ8Gv7x6O9N7TMWRbDFcYTtrZeuh1sLtVLbdjUH5flWSWtq2X2MJSkzoLjOfrSMd0TwjEzwF5os/TbMovj77WEqwFhHF/F5Mzr0v3sK5Uh3Fbgvj9+nlFCk9Ff20clnaTYC6hKuiGISYYw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=OfSN8q+CnVo23NDqHh0S/Z9/fDUvVHplbpLyWz8PKi8=; b=iu877/C6NvA8VbMFbgR4rTsFcC/G8dsq5CEdyfSjpZkVG95ALgjMYAxKImVGdmFxmVmFLq3SPAGrqp1P1J5EBPyakd4RYR22n2YtbLbp1jCYdFu9Qcl/6DHlMX/3OjtJ3Tdxu1LxmO3Kq4QDx3FVwiK8/U7u44SWxMGWJaNfFMhKWgLJGGoahAp4kbQWwU69/j1qkyTri2U1egt4DJPWizMxhrWXY0M7ctRcGM0epLLwZOa6SKAU8asmOtXFhkgqpEjEJYDKgJx1MsHtI76OvevVsi9wrFp2a1wTB8L433MjCxzjXr3IEh64C6qEbTbY9Vw7SmwJKtqpZUhGLUwmtw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OfSN8q+CnVo23NDqHh0S/Z9/fDUvVHplbpLyWz8PKi8=; b=Sa9t++hBSwBa7NK8ZdxaqxyEYXQJBf1xkxxRSjtN3Hw5VILJ5AvkDvEfAhTlJYALrEYkiWcu3kjy9dfdp6YQXaBlJfkiRBmyjSh7t2gJ0ZSImV05woTYuCtUV3qep6u4r+H17h+3P5GcSK6MLW0MxPHf/0o7rs13D57FVKlGRq8= Received: from CO1PR10MB4468.namprd10.prod.outlook.com (2603:10b6:303:6c::24) by MWHPR1001MB2398.namprd10.prod.outlook.com (2603:10b6:301:2f::37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5332.20; Tue, 14 Jun 2022 02:08:54 +0000 Received: from CO1PR10MB4468.namprd10.prod.outlook.com ([fe80::e9bd:ba52:c1c2:11e0]) by CO1PR10MB4468.namprd10.prod.outlook.com ([fe80::e9bd:ba52:c1c2:11e0%9]) with mapi id 15.20.5332.021; Tue, 14 Jun 2022 02:08:54 +0000 From: Imran Khan To: tj@kernel.org, gregkh@linuxfoundation.org, viro@zeniv.linux.org.uk Cc: linux-kernel@vger.kernel.org Subject: [PATCH v5 1/4] kernfs: make ->attr.open RCU protected. Date: Tue, 14 Jun 2022 12:08:37 +1000 Message-Id: <20220614020840.546013-2-imran.f.khan@oracle.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220614020840.546013-1-imran.f.khan@oracle.com> References: <20220614020840.546013-1-imran.f.khan@oracle.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SY6PR01CA0036.ausprd01.prod.outlook.com (2603:10c6:10:eb::23) To CO1PR10MB4468.namprd10.prod.outlook.com (2603:10b6:303:6c::24) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1e72f9f3-ed75-4400-4725-08da4daad4f7 X-MS-TrafficTypeDiagnostic: MWHPR1001MB2398:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 2Zgia1u/tcfTt41mFXO4JSkpgpGjWW4kfZdINBxhOBnSMEm52yUIMi+E3OfT3b3Icl5Ycq9Wz7BouGUEc5SLNLUYPFFTcAQf60lL6GGOAKSL3/X75U06eq4tVm+zW5kLnmHOcS32BeKtyng9MAHjV4E3iCjXVpMjeuYO9VHllnXt3xqdsopDUd6vgSycck5rbPWu8SHRB+KYbxcxAHSimlPrNmwVcNmqMewgKra/WAoJRJIbyeJzPxMNZnL5+odHXEvE2B4VUvS7vYOeFa18Guz85U30BsSkZOyl6CEAzJcuvaAkMCOeuWxnCUzQlfUEtdlMaLHBlsvCvi53xfSowaWWG9BAUBF9rIyEbwQpqcwQCzAcLCkMxS0ctlANMAj0ZapnyD0/lC0Vurtu6pWgYyAd087Rzlj9YdIX6Cr/TsVjbJfdESc+fCwM0yv/6J4i6LbX6HtrKBFqpLisql2IFHPbjYqF3cYRYdl0x7LCOGjIidVbANZcW6OX1kfCMu9dUk3kn3d+BqIfSX3I/wTLIKGZyS/f09rvcS2I66QM/o1vM4piFEOl2U20SgsrI5MScoH2ev7UJm4cOG6sxNTB8JcfoRLtch5+TbaELu1s940Vj4lcxJOckaoLtXfuuSX6Clqi9VgT0JowpqVb5j1xGNCrk5qO03/F+8lv1peqKajhVnJ2Y14mlsMRUaAjMugKhSuT6tik3YekH6/DU+0ssg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO1PR10MB4468.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230016)(366004)(66556008)(66946007)(1076003)(66476007)(8676002)(4326008)(186003)(5660300002)(2616005)(36756003)(52116002)(6506007)(6666004)(83380400001)(8936002)(26005)(6512007)(86362001)(103116003)(2906002)(38350700002)(6486002)(38100700002)(316002)(508600001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?fYtLC5lsYzlOeRquz6uRZvRt1jccWXuRUNBqb+yLO9pJa9q1UUwDa9MzXBc+?= =?us-ascii?Q?eVH+ly5dT54NfIzLJDHwCvVYXHQMtgFwj0DMFkKGjDtfyaaP6FFTeA0dUcEC?= =?us-ascii?Q?1baFVuZz4esU/ZDYU0tPw3z5S5aZYXdKKprjoeHzzMhvYlnPugDN4xJGPalB?= =?us-ascii?Q?bLWV80BblQnqgJ7XM00NnU0K3vPqFEs3itvHJjrJsPTu+C19gdAzueLYicm9?= =?us-ascii?Q?g371bbFEYHSh4O9zYugFS1ZOv7FVqtSnXVSw4JaRXN7kpfzBX5qZuMovX672?= =?us-ascii?Q?Bd+U7SBwLXZ0nE4BoOeFcBjF0w/yjvFQC8VIQJZ6v8TbRQgH8pMZ7p6Eez2B?= =?us-ascii?Q?rb7Fn2yasY1ml7cLbEMmjdND4I5CHZlHePAfKqZeMliIIbtXohM8veeR/eMB?= =?us-ascii?Q?mCMMwkMgt4w0Bh0gMYWvi8Ybsn9/KZGe0sI363AtzxY0r8Vih6lCA5Sjzlat?= =?us-ascii?Q?KCkt+plITXJhiKQnaSIuXRIzwAlRqr7oUd1N8bNzHWCGcHUyV/U3yHEYHj64?= =?us-ascii?Q?4quX3Urxx3lDspZvL98KNJgNarQfp15/sGhGxbGBEAbwYf5cHQRCsjD5DafV?= =?us-ascii?Q?F5cnuV5YhLQre58DCUXpnUwb8a4+HW5SkafXNmWWIYu9k4kgYvKDlz1keXQ1?= =?us-ascii?Q?eegt2cV/evw8HYlkdC8spy6B1BJCTKWhjkYiibAHK1SEZRWKtBHdiGJBxRGP?= =?us-ascii?Q?gZltQ3POmz8HB+geAx0BriuzsUjgXUROPVnxoQBZUwjd7OLJg4tv4cibymaL?= =?us-ascii?Q?ZLkDNPwfr9DNVuVpxYk6zAGd4gmw6cvu/oYvQ2FiRhkRe4AUSfRF8dr1BTXg?= =?us-ascii?Q?943NPO2l3kCIhKbP0z6Bc419HxuZ6s8vwCumVFOTxdArDcb5ZuPHZjGuoX3o?= =?us-ascii?Q?S1ZUlWaKbmuO+c6f5LDktUS4bs+NCvjFsVkMOeYNqJOZlZgvZ4WVIo2hkajx?= =?us-ascii?Q?79afSvUj/Rjsk1CMO6YarXrxzdzMWD9sVwMGRT1kh3K8KuWP1oPuuY4/BY3L?= =?us-ascii?Q?FVHAtTHEZtSETw3mJj7kGHRjA1qtEpYX7U02ak3nGFVAQxo8HyUONJxMfovH?= =?us-ascii?Q?2zbSa/CSuNGxN0d8LK6X1tCmCXh4MgHem0eVGxKERYut11EP1Y83eIXnni0P?= =?us-ascii?Q?0GJtzJMKUBufzPQ+PYnE8c3KPxl0c0YsY86j8p0nlx9eOXDnFI0tAJNIocUO?= =?us-ascii?Q?DCKZgH9Br0OafXbU9il8OxnjjchMxExiyvrThsHq+fbKPLaU6f9AQl8TCaeH?= =?us-ascii?Q?Fla8Pi3nCm2DP4VT5byzr0vqUbbUZoVWZqDcWLVueP57hyRGB/z4wkhe8Fda?= =?us-ascii?Q?X4FhkwNET3pPxh2cw3fy2mRQ3BYyg4Vn15mnYaWGFuby+yfyowkDbaqoboKf?= =?us-ascii?Q?bD2mW3aO6G/UegQXIL7VNeBpxS+4aZ1jDOWOAw0xVf2mjvoBg/TTfwRmFWkB?= =?us-ascii?Q?jgWi0oJNmiKzLXm5veqhkEmFTMB5JbViD0Cg79Iy1RqCS6mISmJoP5D1cDEB?= =?us-ascii?Q?zF2z8S0+GxczYxYOgBdbXT6m3q12GBmhHpAcFMqa1FpomIwAyf8W3FpHKodm?= =?us-ascii?Q?sHGV40OIwrdHZthf2MjLmNhXEdrQ+vIhmdQLFWHUph8TwRRs+ScfnRDQtFEq?= =?us-ascii?Q?L3JKWsVaXZllHqBUs9jU/7c4ZztAuM0doA3OeawrBoV1eRi8dh6yLbaARjqT?= =?us-ascii?Q?8blKSwcjg8YASbPxVm/BI1lh5pn+5rAB3FwcdMUWiLbQvE2ZdgYTGtBJ4NpS?= =?us-ascii?Q?cT8ra3/H54IRMXSQEmgou6xGf3itJv4=3D?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1e72f9f3-ed75-4400-4725-08da4daad4f7 X-MS-Exchange-CrossTenant-AuthSource: CO1PR10MB4468.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Jun 2022 02:08:54.0194 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: gME25gI8/y27nM3rWHfwYjhLsJS1hB1ucFwo7jZ4tWsJrEaN9LF0pqR1IUDpltcrsF0jqV1rWsde3nLcRZAhng== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR1001MB2398 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.517,18.0.874 definitions=2022-06-14_01:2022-06-13,2022-06-13 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 suspectscore=0 adultscore=0 phishscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2204290000 definitions=main-2206140007 X-Proofpoint-GUID: 7ehe1OOc1R6gL_qI2oHkzCsEVmFXwHSB X-Proofpoint-ORIG-GUID: 7ehe1OOc1R6gL_qI2oHkzCsEVmFXwHSB Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" After removal of kernfs_open_node->refcnt in the previous patch, kernfs_open_node_lock can be removed as well by making ->attr.open RCU protected. kernfs_put_open_node can delegate freeing to ->attr.open to RCU and other readers of ->attr.open can do so under rcu_read_(un)lock. Suggested by: Al Viro Signed-off-by: Imran Khan --- fs/kernfs/file.c | 142 ++++++++++++++++++++++++++++------------- include/linux/kernfs.h | 2 +- 2 files changed, 97 insertions(+), 47 deletions(-) diff --git a/fs/kernfs/file.c b/fs/kernfs/file.c index e3abfa843879c..42ee5dbe351d3 100644 --- a/fs/kernfs/file.c +++ b/fs/kernfs/file.c @@ -23,16 +23,16 @@ * for each kernfs_node with one or more open files. * * kernfs_node->attr.open points to kernfs_open_node. attr.open is - * protected by kernfs_open_node_lock. + * RCU protected. * * filp->private_data points to seq_file whose ->private points to * kernfs_open_file. kernfs_open_files are chained at * kernfs_open_node->files, which is protected by kernfs_open_file_mutex. */ -static DEFINE_SPINLOCK(kernfs_open_node_lock); static DEFINE_MUTEX(kernfs_open_file_mutex); =20 struct kernfs_open_node { + struct rcu_head rcu_head; atomic_t event; wait_queue_head_t poll; struct list_head files; /* goes through kernfs_open_file.list */ @@ -51,6 +51,47 @@ struct kernfs_open_node { static DEFINE_SPINLOCK(kernfs_notify_lock); static struct kernfs_node *kernfs_notify_list =3D KERNFS_NOTIFY_EOL; =20 +/** + * kernfs_deref_open_node - Get kernfs_open_node corresponding to @kn. + * + * @of: associated kernfs_open_file instance or NULL if unknown. + * @kn: target kernfs_node. + * + * Fetch and return ->attr.open of @kn if @of->list is not empty or if + * kernfs_open_file_mutex is held. + * + * We can rely on either of the following 2 conditions to ensure that + * dereferencing ->attr.open outside RCU read-side critical section is saf= e. + * + * 1. If @of->list is not empty we can safely assume that @of is on + * @kn->attr.open->files list and this guarantees that @kn->attr.open + * will not vanish i.e. dereferencing outside RCU read-side critical + * section is safe here. + * + * or + * + * 2. Update of ->attr.open happens under kernfs_open_file_mutex. So as lo= ng as + * the current updater (caller) is holding this mutex, other updaters will= not + * be able to change ->attr.open and this means that we can safely deref + * ->attr.open outside RCU read-side critical section. + * + * The caller needs to make sure that either @of->list is not empty or + * kernfs_open_file_mutex is held. + */ +static struct kernfs_open_node * +kernfs_deref_open_node(struct kernfs_open_file *of, struct kernfs_node *kn) +{ + struct kernfs_open_node *on; + bool deref_ok; + + deref_ok =3D (of ? !list_empty(&of->list) : false) || + lockdep_is_held(&kernfs_open_file_mutex); + + on =3D rcu_dereference_check(kn->attr.open, deref_ok); + + return on; +} + static struct kernfs_open_file *kernfs_of(struct file *file) { return ((struct seq_file *)file->private_data)->private; @@ -156,8 +197,12 @@ static void kernfs_seq_stop(struct seq_file *sf, void = *v) static int kernfs_seq_show(struct seq_file *sf, void *v) { struct kernfs_open_file *of =3D sf->private; + struct kernfs_open_node *on =3D kernfs_deref_open_node(of, of->kn); =20 - of->event =3D atomic_read(&of->kn->attr.open->event); + if (!on) + return -EINVAL; + + of->event =3D atomic_read(&on->event); =20 return of->kn->attr.ops->seq_show(sf, v); } @@ -180,6 +225,7 @@ static ssize_t kernfs_file_read_iter(struct kiocb *iocb= , struct iov_iter *iter) struct kernfs_open_file *of =3D kernfs_of(iocb->ki_filp); ssize_t len =3D min_t(size_t, iov_iter_count(iter), PAGE_SIZE); const struct kernfs_ops *ops; + struct kernfs_open_node *on; char *buf; =20 buf =3D of->prealloc_buf; @@ -201,7 +247,15 @@ static ssize_t kernfs_file_read_iter(struct kiocb *ioc= b, struct iov_iter *iter) goto out_free; } =20 - of->event =3D atomic_read(&of->kn->attr.open->event); + on =3D kernfs_deref_open_node(of, of->kn); + if (!on) { + len =3D -EINVAL; + mutex_unlock(&of->mutex); + goto out_free; + } + + of->event =3D atomic_read(&on->event); + ops =3D kernfs_ops(of->kn); if (ops->read) len =3D ops->read(of, buf, len, iocb->ki_pos); @@ -519,36 +573,29 @@ static int kernfs_get_open_node(struct kernfs_node *k= n, { struct kernfs_open_node *on, *new_on =3D NULL; =20 - retry: mutex_lock(&kernfs_open_file_mutex); - spin_lock_irq(&kernfs_open_node_lock); - - if (!kn->attr.open && new_on) { - kn->attr.open =3D new_on; - new_on =3D NULL; - } - - on =3D kn->attr.open; - if (on) - list_add_tail(&of->list, &on->files); - - spin_unlock_irq(&kernfs_open_node_lock); - mutex_unlock(&kernfs_open_file_mutex); + on =3D kernfs_deref_open_node(NULL, kn); =20 if (on) { - kfree(new_on); + list_add_tail(&of->list, &on->files); + mutex_unlock(&kernfs_open_file_mutex); return 0; + } else { + /* not there, initialize a new one */ + new_on =3D kmalloc(sizeof(*new_on), GFP_KERNEL); + if (!new_on) { + mutex_unlock(&kernfs_open_file_mutex); + return -ENOMEM; + } + atomic_set(&new_on->event, 1); + init_waitqueue_head(&new_on->poll); + INIT_LIST_HEAD(&new_on->files); + list_add_tail(&of->list, &new_on->files); + rcu_assign_pointer(kn->attr.open, new_on); } + mutex_unlock(&kernfs_open_file_mutex); =20 - /* not there, initialize a new one and retry */ - new_on =3D kmalloc(sizeof(*new_on), GFP_KERNEL); - if (!new_on) - return -ENOMEM; - - atomic_set(&new_on->event, 1); - init_waitqueue_head(&new_on->poll); - INIT_LIST_HEAD(&new_on->files); - goto retry; + return 0; } =20 /** @@ -567,24 +614,25 @@ static int kernfs_get_open_node(struct kernfs_node *k= n, static void kernfs_unlink_open_file(struct kernfs_node *kn, struct kernfs_open_file *of) { - struct kernfs_open_node *on =3D kn->attr.open; - unsigned long flags; + struct kernfs_open_node *on; =20 mutex_lock(&kernfs_open_file_mutex); - spin_lock_irqsave(&kernfs_open_node_lock, flags); + + on =3D kernfs_deref_open_node(NULL, kn); + if (!on) { + mutex_unlock(&kernfs_open_file_mutex); + return; + } =20 if (of) list_del(&of->list); =20 - if (list_empty(&on->files)) - kn->attr.open =3D NULL; - else - on =3D NULL; + if (list_empty(&on->files)) { + rcu_assign_pointer(kn->attr.open, NULL); + kfree_rcu(on, rcu_head); + } =20 - spin_unlock_irqrestore(&kernfs_open_node_lock, flags); mutex_unlock(&kernfs_open_file_mutex); - - kfree(on); } =20 static int kernfs_fop_open(struct inode *inode, struct file *file) @@ -774,17 +822,16 @@ void kernfs_drain_open_files(struct kernfs_node *kn) * check under kernfs_open_file_mutex will ensure bailing out if * ->attr.open became NULL while waiting for the mutex. */ - if (!kn->attr.open) + if (!rcu_access_pointer(kn->attr.open)) return; =20 mutex_lock(&kernfs_open_file_mutex); - if (!kn->attr.open) { + on =3D kernfs_deref_open_node(NULL, kn); + if (!on) { mutex_unlock(&kernfs_open_file_mutex); return; } =20 - on =3D kn->attr.open; - list_for_each_entry(of, &on->files, list) { struct inode *inode =3D file_inode(of->file); =20 @@ -815,7 +862,10 @@ void kernfs_drain_open_files(struct kernfs_node *kn) __poll_t kernfs_generic_poll(struct kernfs_open_file *of, poll_table *wait) { struct kernfs_node *kn =3D kernfs_dentry_node(of->file->f_path.dentry); - struct kernfs_open_node *on =3D kn->attr.open; + struct kernfs_open_node *on =3D kernfs_deref_open_node(of, kn); + + if (!on) + return EPOLLERR; =20 poll_wait(of->file, &on->poll, wait); =20 @@ -922,13 +972,13 @@ void kernfs_notify(struct kernfs_node *kn) return; =20 /* kick poll immediately */ - spin_lock_irqsave(&kernfs_open_node_lock, flags); - on =3D kn->attr.open; + rcu_read_lock(); + on =3D rcu_dereference(kn->attr.open); if (on) { atomic_inc(&on->event); wake_up_interruptible(&on->poll); } - spin_unlock_irqrestore(&kernfs_open_node_lock, flags); + rcu_read_unlock(); =20 /* schedule work to kick fsnotify */ spin_lock_irqsave(&kernfs_notify_lock, flags); diff --git a/include/linux/kernfs.h b/include/linux/kernfs.h index e2ae15a6225e8..13f54f078a52a 100644 --- a/include/linux/kernfs.h +++ b/include/linux/kernfs.h @@ -114,7 +114,7 @@ struct kernfs_elem_symlink { =20 struct kernfs_elem_attr { const struct kernfs_ops *ops; - struct kernfs_open_node *open; + struct kernfs_open_node __rcu *open; loff_t size; struct kernfs_node *notify_next; /* for kernfs_notify() */ }; --=20 2.30.2 From nobody Wed Apr 29 12:25:12 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 671C5CCA47B for ; Tue, 14 Jun 2022 02:22:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1353843AbiFNCW0 (ORCPT ); Mon, 13 Jun 2022 22:22:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57652 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1353438AbiFNCP1 (ORCPT ); Mon, 13 Jun 2022 22:15:27 -0400 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3CF4A3DA6F for ; Mon, 13 Jun 2022 19:09:22 -0700 (PDT) Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 25E0cW6r004207; Tue, 14 Jun 2022 02:09:05 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=9U886SywQHCDCtftkwq8pGqpEbVahLlNzCzGICxBm3w=; b=Joxaj+mVMukpGxWChC6GQYc1Baa/2pb1Y+lfZYFEbHEL7n/segDSN6CQYPGoXaNxCOc1 LMJn7y1x2n7bpurQkoWu2m5kitJ4d629Q0XOslF2Lov66LyVXsbkOooEXefzcrq1AEkF TARA6R79QtwuwTW8xv0/GL2jtL4tU7O+JkTYFtg5vkn/vsJKrWWAdJzgVFsTcXrInrFE NRfY9sSZcbKkQysZlnLyHfazmo96Ec//hfB+ZvrVJgIdFjyQpZwSRn+iAdjtafZuC9+S 9s7oCNSoIQiDMbWKBAvDidCMoDT93SExaQG/pyWu5NoOL22SnlKrTqkpzlNMsijnU1tB qQ== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3gmhu2mmu5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 14 Jun 2022 02:09:05 +0000 Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 25E20I5n025507; Tue, 14 Jun 2022 02:08:57 GMT Received: from nam11-co1-obe.outbound.protection.outlook.com (mail-co1nam11lp2175.outbound.protection.outlook.com [104.47.56.175]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com with ESMTP id 3gpgyerbhm-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 14 Jun 2022 02:08:57 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ogx5Hm5E4EAeC0MsOvB2AFokQ2lAsr4EoSXvG9xQzQofHCCvzj13mi7L63l6NOBgxJYXMjIDiSMtJb8Kg8Z08H6HezRmUxP5GhqVz1aPUD8xpxPKnvOsgu7eH2EuXyzgWBIuVm4zYlS2QHbMWcBaooCih2dJHjG/jx/BoZHUmnjhAA2UTshsDrdhAZKVvGiKtDg3enb86HRw2aFezQRy46Ajv95prNnd7LnIJvf2rXQsbo3X1j+GJVTQ7xhto2SEGsx3o6YRZlmePA9vQf2XycY6dVo8tMPcHY8i/ZNyR+sUpm9xnUy8uUlzSn2EpStqbseHyV2jAxX27bGOppZbnA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=9U886SywQHCDCtftkwq8pGqpEbVahLlNzCzGICxBm3w=; b=BdV+UJB24Wcgl3QEueRx5CvZQPE2FqY6f8mfwNnzK+G5oftXdHLAsZ4WxGI+8ZYrBVasoJaFjhNct2PMWLXr+kCFR2E+5UiTpbVCuCmjDzs0B7J5SN01TFCWfWMYcdfn9dVjYgLFUZokhw2xySQut7Jy5ODnq4RFXy4ow2VQO+kRXyMwOJ/iVZk7VifmVoXvwz5/4m/0U3ORVGKdaeghamsZpkkgPcGNTlH9YzTQayLiBkVAXMK8RKFlNTOq+zH1y+F1mlckr9lmTQ0Cfrp9dzyDGvr1AD2aXnczNnHwDep5Z3TfR8YetczEeeZi4GIT4TyOiGQZjNottut/dLdwZA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9U886SywQHCDCtftkwq8pGqpEbVahLlNzCzGICxBm3w=; b=MAJR+M1iJIfheYzhUWe7YpM/hlznCGvfAVpbAT93tAqA5X8/RKi+TcvigzxJLSlGa16Q3A14lQHzZa0Q57uNfQhi7MnhlDMQEhSshrSM9QFROlC+fhLjC40M2YQrZc1H0EzBedxeCdZppI1eMtGPXzSgAdgmx/4LhptmQiCARW0= Received: from CO1PR10MB4468.namprd10.prod.outlook.com (2603:10b6:303:6c::24) by MWHPR1001MB2398.namprd10.prod.outlook.com (2603:10b6:301:2f::37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5332.20; Tue, 14 Jun 2022 02:08:55 +0000 Received: from CO1PR10MB4468.namprd10.prod.outlook.com ([fe80::e9bd:ba52:c1c2:11e0]) by CO1PR10MB4468.namprd10.prod.outlook.com ([fe80::e9bd:ba52:c1c2:11e0%9]) with mapi id 15.20.5332.021; Tue, 14 Jun 2022 02:08:55 +0000 From: Imran Khan To: tj@kernel.org, gregkh@linuxfoundation.org, viro@zeniv.linux.org.uk Cc: linux-kernel@vger.kernel.org Subject: [PATCH v5 2/4] kernfs: Change kernfs_notify_list to llist. Date: Tue, 14 Jun 2022 12:08:38 +1000 Message-Id: <20220614020840.546013-3-imran.f.khan@oracle.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220614020840.546013-1-imran.f.khan@oracle.com> References: <20220614020840.546013-1-imran.f.khan@oracle.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SY6PR01CA0036.ausprd01.prod.outlook.com (2603:10c6:10:eb::23) To CO1PR10MB4468.namprd10.prod.outlook.com (2603:10b6:303:6c::24) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 06f3ffe3-a5cb-4b92-8a90-08da4daad5f1 X-MS-TrafficTypeDiagnostic: MWHPR1001MB2398:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: SXirPnmT1JQfHqLrBV/XvVgM92TKmn8S6MoJHLQhKPGPPQ5xnuyjKsoh1VTdNmy5tczPCjdRbMOorT/+mhDwWRTeeYhtSLewi8OweQp7i3MQ0Vt1ZOKesBBB/9H4v4bzTu+DqU2U5W9Z5mk3vWecqsmVwv8dYvz85UmAP9nHhjsmu6VoJ2p/ZuMfFWzapNf/MxnCTYuMW8LS6ZBx/upChBCTtRWyw6dFerq/RaZlK2Jc9AlfCzAs52GZ4iC3YNqfuXtXR7UMznIUd7aXbERiQb0ykbHqk+6B2ahj+3jC0u0Lah8ciy+a2adiFguO77GXLGzhDzyn6OkxZ0EluDa/k0Oiq26fn2Qt0VDGoouXTadGCoNvMTVSZYV4/dKoRi8LyJmW5u7W6y8YF9rUZ8KtVQz4fJrJqbBL0dlEJBXnTclVdBPLkZXTDU/sZGrV+jz4o922+X0gKKILLwTHY/roSZmTz/A5D4EeMXMm7DMEnMvpdb5LBFNYEF0kuRkSizB/QfYQtkJcjHjg0YWeBLXu21o/xadssnYtiMk01ITYijboz4XUioDTnYgeGAnymztkJPfMUWE8Fsy1dL5UyxMeip7TvCqMehJD0gzByJHJfgPz3aN11Oxd5Yaf2v7tHbFAQLcMmP/E8b7Y3iZU/V1A/OVWfLI5jbqkoEPYC8bpm0Q9m9JAERX2BbJkxALX1V2oTWpnc/DLRxZlMBHwQjaQhg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO1PR10MB4468.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230016)(366004)(66556008)(66946007)(1076003)(66476007)(8676002)(4326008)(186003)(5660300002)(2616005)(36756003)(52116002)(6506007)(6666004)(83380400001)(8936002)(26005)(6512007)(86362001)(103116003)(2906002)(38350700002)(6486002)(38100700002)(316002)(508600001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?+M19bHIGUJbVDThaJ6uZBjROBPDvbk/KTxrdJhAC5py3KUcrJdBOId9ikYFD?= =?us-ascii?Q?+VtZ1lP5kWW8v9pOn0t9Fg7lCeSefTaFdmrXh6dITwGdjBJ9lkgbuURgeNSy?= =?us-ascii?Q?HOUef+6N83NFFv2QEktAnTj4Vhbbu3JM+++KJ/LTPbWvL37bmKpL8WAv6gfP?= =?us-ascii?Q?yZCGA7w4xCbUyDGW5OFepHA0R8XTiyAnLM6SSPhCHd3/3ZGdHaYI9Xa7xePv?= =?us-ascii?Q?dcc0uJTicFQjcwQGvv2fAT93mERKo4Pwr5W4hfVzfBEA0TzOizdsTBtLTBHD?= =?us-ascii?Q?7rNcymL3uZiFbYIGnqi/XmQjU+Iy6paUsvLkpTFCFDq/XufzK88oSGlErnYa?= =?us-ascii?Q?80OH6BN0BGcqu+i/Sk+Jx2TtO4Zjo/jhptRX0u2K/kjxHE8x1wnyYx6+BmiG?= =?us-ascii?Q?VVcf/lhGdYsIz/b5SA7yzi5qyBm1p7ED3fFjCqn5jbb1tLloA1r3+yCzPKDr?= =?us-ascii?Q?MPrDAubhr8Eyxbq+gJhXEXszezoYpWWKmoeUCQGivAsTr5sIrOua+QxTFAHS?= =?us-ascii?Q?gVDLB7D7pjYM4RW+Ft4EqfmC36ub9Vf02nB+lbfYHK2mVJ1ti4zbbF3z6poI?= =?us-ascii?Q?8ldfATE8sS/e6BFUWwm/4o8pflNJsUPgAnZHAmciRBD8JpAqriAyn8Pb0//I?= =?us-ascii?Q?4yg16vYM/jTh6eP3LUbS2vyfc2MklZ2EnbqzdIBrd49HRP4P8gYy+NSgfH67?= =?us-ascii?Q?v0NjgMkKNcqglTE4YTVsTxAkVwQpiGO8zh1C+B3Tu7ICMv4mKol479Wd+7V7?= =?us-ascii?Q?G1TYRdXaMpVaMc3L0oQ5dGtS7lmwhZFkNTGbaKqoqWufQf6b5zf2VGpbAcuV?= =?us-ascii?Q?W47LtdHozp2onAjkC0PdjgZ0iJNh9JF/qqmTNvnxF0Xhn/CZ+SJsiRpvbtuq?= =?us-ascii?Q?Ki6N8ibb37sDnFV+Wq0KlJVtHathi80VchsdRrYZfw8E3GeKZKLOd+gF1BN7?= =?us-ascii?Q?zqK68VmNhEfF6Pl7EbV8giPRKWr92Is/88x8XXsP8hjve7HWteILP4L5naon?= =?us-ascii?Q?98BYmYwhisBNHVvdMbsLfwDsCYr/KHOomEHlFy0YdgPM7XEBRTDImkJ92U33?= =?us-ascii?Q?lHiRo4zRriZtYFusPWtrcy8NGBh4HGXEOpYjnHYIVa62ODL8yZE1OylQ4Vtu?= =?us-ascii?Q?uf3c2DPHbDf4zeXL3ZehkjTQXFEIVzSExB8s0MvQKbVVKUSqNfsQ7CD88j0S?= =?us-ascii?Q?SJuRZOTneEMgoVpiHXQKPeqi91tDbDnnw8ExBxbMc7rlMUwA7dtlpOrXgFfU?= =?us-ascii?Q?T/TcOzWnLBnnon8VDoeFLPSJ8sff4uyjAwK3SZG91J6J0M7lI78gHWJ6VdWi?= =?us-ascii?Q?UTIhZSSL6p5vBUndsE46IJU7s/+AVnCSc8YOkYsWFlnZBIRFJiTXcsYa4wo6?= =?us-ascii?Q?N0x0fODnEt8ar5BrfGUVXKIArhQGHZYviSRXrylFR4w1TsZHWGnxLJbE4UDi?= =?us-ascii?Q?Fh07o7XKS5280RYH0I37HLjtv1eTLuRSQqlfmSVos8E1GMZ5eOwo8lNGgFkR?= =?us-ascii?Q?hyFDSyJDih51tcGqbI/hiWyyhfJfNAHzTgOeNlfRRdYDdmd+DuOR4GfFh7lX?= =?us-ascii?Q?lcdDHzgRYUxqw2ShpYIkw54lURsGfosS0ONGI+DwrcjsRgYf2ivO9iHx7caO?= =?us-ascii?Q?t5vYrdyC6VGDTAe5Lo50jqTrJ2QH0/OhdlCCi6i0+E8uzf+1MQgNwBgVVWZm?= =?us-ascii?Q?J9osoZcdMl5uY9c7U+ZatDV0i0nQOMFQxe+kwXOp+pzZRXzJsRtSKV1yykH2?= =?us-ascii?Q?KhOHeY+6LSLJ1qsUHeLC2YJ0zze8LgE=3D?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 06f3ffe3-a5cb-4b92-8a90-08da4daad5f1 X-MS-Exchange-CrossTenant-AuthSource: CO1PR10MB4468.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Jun 2022 02:08:55.6908 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: rls1CQlPAhI11h91bA9Lq+n5GcHMUrb0tg6sK/sAhJG1eOBeX4D/dZZOhGRFL76rGdpY9x7hwT+4UPY3TqxFoQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR1001MB2398 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.517,18.0.874 definitions=2022-06-14_01:2022-06-13,2022-06-13 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 suspectscore=0 adultscore=0 phishscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2204290000 definitions=main-2206140007 X-Proofpoint-GUID: LFyTaHnqSwY8N4F9Lr6YoGO52Kv5aTAD X-Proofpoint-ORIG-GUID: LFyTaHnqSwY8N4F9Lr6YoGO52Kv5aTAD Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" At present kernfs_notify_list is implemented as a singly linked list of kernfs_node(s), where last element points to itself and value of ->attr.next tells if node is present on the list or not. Both addition and deletion to list happen under kernfs_notify_lock. Change kernfs_notify_list to llist so that addition to list can heppen locklessly. Suggested by: Al Viro Signed-off-by: Imran Khan Acked-by: Tejun Heo --- fs/kernfs/file.c | 47 ++++++++++++++++++------------------------ include/linux/kernfs.h | 2 +- 2 files changed, 21 insertions(+), 28 deletions(-) diff --git a/fs/kernfs/file.c b/fs/kernfs/file.c index 42ee5dbe351d3..0cc288dcf51d9 100644 --- a/fs/kernfs/file.c +++ b/fs/kernfs/file.c @@ -38,18 +38,16 @@ struct kernfs_open_node { struct list_head files; /* goes through kernfs_open_file.list */ }; =20 -/* - * kernfs_notify() may be called from any context and bounces notifications - * through a work item. To minimize space overhead in kernfs_node, the - * pending queue is implemented as a singly linked list of kernfs_nodes. - * The list is terminated with the self pointer so that whether a - * kernfs_node is on the list or not can be determined by testing the next - * pointer for NULL. +/** + * attribute_to_node - get kernfs_node object corresponding to a kernfs at= tribute + * @ptr: &struct kernfs_elem_attr + * @type: struct kernfs_node + * @member: name of member (i.e attr) */ -#define KERNFS_NOTIFY_EOL ((void *)&kernfs_notify_list) +#define attribute_to_node(ptr, type, member) \ + container_of(ptr, type, member) =20 -static DEFINE_SPINLOCK(kernfs_notify_lock); -static struct kernfs_node *kernfs_notify_list =3D KERNFS_NOTIFY_EOL; +static LLIST_HEAD(kernfs_notify_list); =20 /** * kernfs_deref_open_node - Get kernfs_open_node corresponding to @kn. @@ -898,18 +896,16 @@ static void kernfs_notify_workfn(struct work_struct *= work) struct kernfs_node *kn; struct kernfs_super_info *info; struct kernfs_root *root; + struct llist_node *free; + struct kernfs_elem_attr *attr; repeat: /* pop one off the notify_list */ - spin_lock_irq(&kernfs_notify_lock); - kn =3D kernfs_notify_list; - if (kn =3D=3D KERNFS_NOTIFY_EOL) { - spin_unlock_irq(&kernfs_notify_lock); + free =3D llist_del_first(&kernfs_notify_list); + if (free =3D=3D NULL) return; - } - kernfs_notify_list =3D kn->attr.notify_next; - kn->attr.notify_next =3D NULL; - spin_unlock_irq(&kernfs_notify_lock); =20 + attr =3D llist_entry(free, struct kernfs_elem_attr, notify_next); + kn =3D attribute_to_node(attr, struct kernfs_node, attr); root =3D kernfs_root(kn); /* kick fsnotify */ down_write(&root->kernfs_rwsem); @@ -965,12 +961,14 @@ static void kernfs_notify_workfn(struct work_struct *= work) void kernfs_notify(struct kernfs_node *kn) { static DECLARE_WORK(kernfs_notify_work, kernfs_notify_workfn); - unsigned long flags; struct kernfs_open_node *on; =20 if (WARN_ON(kernfs_type(kn) !=3D KERNFS_FILE)) return; =20 + /* Because we are using llist for kernfs_notify_list */ + WARN_ON_ONCE(in_nmi()); + /* kick poll immediately */ rcu_read_lock(); on =3D rcu_dereference(kn->attr.open); @@ -981,14 +979,9 @@ void kernfs_notify(struct kernfs_node *kn) rcu_read_unlock(); =20 /* schedule work to kick fsnotify */ - spin_lock_irqsave(&kernfs_notify_lock, flags); - if (!kn->attr.notify_next) { - kernfs_get(kn); - kn->attr.notify_next =3D kernfs_notify_list; - kernfs_notify_list =3D kn; - schedule_work(&kernfs_notify_work); - } - spin_unlock_irqrestore(&kernfs_notify_lock, flags); + kernfs_get(kn); + llist_add(&kn->attr.notify_next, &kernfs_notify_list); + schedule_work(&kernfs_notify_work); } EXPORT_SYMBOL_GPL(kernfs_notify); =20 diff --git a/include/linux/kernfs.h b/include/linux/kernfs.h index 13f54f078a52a..2dd9c8df0f4f6 100644 --- a/include/linux/kernfs.h +++ b/include/linux/kernfs.h @@ -116,7 +116,7 @@ struct kernfs_elem_attr { const struct kernfs_ops *ops; struct kernfs_open_node __rcu *open; loff_t size; - struct kernfs_node *notify_next; /* for kernfs_notify() */ + struct llist_node notify_next; /* for kernfs_notify() */ }; =20 /* --=20 2.30.2 From nobody Wed Apr 29 12:25:12 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3ED13C433EF for ; Tue, 14 Jun 2022 02:48:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1354625AbiFNCsv (ORCPT ); Mon, 13 Jun 2022 22:48:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46084 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352311AbiFNCsP (ORCPT ); Mon, 13 Jun 2022 22:48:15 -0400 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B8C5B59960 for ; Mon, 13 Jun 2022 19:26:03 -0700 (PDT) Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 25E0fH1S027238; Tue, 14 Jun 2022 02:09:00 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=ADI9sXjaxTkOLxp23rB/WTc7Om4E0dUsEwTsb+uxY20=; b=OUkJVDsxDMM1f+9GVFiz5uLzwYZlHZFaCBVkAFKnAJ02faVo3Vwth33rr4bKnNM52XoI qZ8rQmHiWq1S/hAZYlXPix3el6GRuAJax/+N2cGXEAVSKkrUY+np9B21/q7haJ16Fx8G zKnRwMQt32mxc2xheW+PiiNO9Ry8XaoS+j2cIGGLkqWV0m4zovtCphhg2r6MM9zuNWu6 0uwJxTKvjgM/Jx/4Q7+RJBShqVzqQfTvQBISZX8DVDkwjyHbLorU2/Bq2JpftwyNZM3i eM2ztGxR6S/IostjT1IBw7fTh4Rl1XS28IKko8qVrv+QwwUi0lHwBErNw1Rb5u4+YjnZ CA== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3gmjns4j5k-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 14 Jun 2022 02:08:59 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 25E20iEL034822; Tue, 14 Jun 2022 02:08:59 GMT Received: from nam11-co1-obe.outbound.protection.outlook.com (mail-co1nam11lp2171.outbound.protection.outlook.com [104.47.56.171]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com with ESMTP id 3gph5xr466-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 14 Jun 2022 02:08:59 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eMHgo7qLsF1M+6vdwWRytEsNKLoZkLqb7zIgG+pJiJBb1dYTeT8LusEyq8XOfM0Xq+dWThf8rOavwlLCGZbXa0VWsyKIOJIQ9oUHKkPyH5F1f2tgcbKrxkXuOmPhFohyj/GsNJnfFxdpNdPCuoTOjkF2+VxwWCqfzZeMS7cerB96InRQq+PgfKE+kiFDpmw3g+PsfbfTsPQW1rcU3apvo8yAvlytZ0/kMPODBo9JF4ysh8dY/53QN2BJ8GcmF/Piqu/NI6wOb29oQTuKPItH2T2Hwf3Wa5I5d/GdJXdChQU8bQ9ARIzYMYmFmzYlR7XySLX9hCrId+5x+PngykJL8Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ADI9sXjaxTkOLxp23rB/WTc7Om4E0dUsEwTsb+uxY20=; b=TF/OTSEUYW48fbyW+lchbmheTGfJjR7OwYrCslF5dfyMIaG7rYl6QqNgQJNKPWlStSPq2igr3R650mMMuoRd0ITguHZcADIJox/ip3whUOO+34LD2aHOkeijupwZoDB1sSmi+0cz5dzk1gyXEfWpL6ZSfR1vT9i8rVgvlQGUaa7KkLsgZT+6xJKa5jY7bEG7nTXRToVBdYQytE0ntX9oSTI4rI5RIf1Nv6BbNwMWo7Z5GfCw9YepwkXDAa1UuU+lFm5JyjaXgzlrsSac+mFC/lfSpG/ePSBBCXN9bkzsI5HEF3NuJOOJbpPWWYTdEYzmwWjGRyqI8tscf3IaPCTihA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ADI9sXjaxTkOLxp23rB/WTc7Om4E0dUsEwTsb+uxY20=; b=K6RfACK4KmxEINrRwCnVnffS0sglenubtc8GRkFqXJ6OsqF9pZBIaobjEVW5Luk4JH4u0vQdOiX1r6ElK1DLEO4brROJ60vY4ntu3AVjRrIsaibXnMDOVBSnzJ/VwOgXa6ARQMm/u2cR847dyB8g7IMC40eedwLcVnpLb0P9xKw= Received: from CO1PR10MB4468.namprd10.prod.outlook.com (2603:10b6:303:6c::24) by MWHPR1001MB2398.namprd10.prod.outlook.com (2603:10b6:301:2f::37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5332.20; Tue, 14 Jun 2022 02:08:57 +0000 Received: from CO1PR10MB4468.namprd10.prod.outlook.com ([fe80::e9bd:ba52:c1c2:11e0]) by CO1PR10MB4468.namprd10.prod.outlook.com ([fe80::e9bd:ba52:c1c2:11e0%9]) with mapi id 15.20.5332.021; Tue, 14 Jun 2022 02:08:57 +0000 From: Imran Khan To: tj@kernel.org, gregkh@linuxfoundation.org, viro@zeniv.linux.org.uk Cc: linux-kernel@vger.kernel.org Subject: [PATCH v5 3/4] kernfs: Introduce interface to access global kernfs_open_file_mutex. Date: Tue, 14 Jun 2022 12:08:39 +1000 Message-Id: <20220614020840.546013-4-imran.f.khan@oracle.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220614020840.546013-1-imran.f.khan@oracle.com> References: <20220614020840.546013-1-imran.f.khan@oracle.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SY6PR01CA0036.ausprd01.prod.outlook.com (2603:10c6:10:eb::23) To CO1PR10MB4468.namprd10.prod.outlook.com (2603:10b6:303:6c::24) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 14ff8109-13af-4d22-049a-08da4daad6ec X-MS-TrafficTypeDiagnostic: MWHPR1001MB2398:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: C85d8qRNU4BkqjNgJ+wlyBz7hsNo0w64Wv8n/7aNdwSxg1PUj4IuWOlp9InjsWe1NsgS0kON5PIuOPkqBK4AFTyqSklrErCYAQN5N2WW8XqpptENX/34MoI7s4j9sgS8QdF3Xa+m0oYCPdWfF2YCdqPcqAWuZxRHP8pZD5CFZ1t/2uS4A2k2ychq3H0oSAhlMTu3X8aQBq5WgA4jXd3bNh6p8b4BfotC789rI1Weoll0j/x+vSfMOKTqrguHz7RA/bcgmDfzB6oUcYyr7VuXu+aRIGh+deoyWww2Q9vgvVp3OC6EJWIynFMa464ddsHUa5nakoNRrOQ8awDaAHGPwHZJXjsGNa41UGhLXaGzPgI9qvCFs9ONuc7IOQHNlRV2P3r9ZtgW/eUo9pvtmtJE3twya6Ytq5edv0Y0RWXxJtxFIklo1rEk1iSvHLZj+AQxJElMD5MGwZIndn0VPYIxHNKRgIRL/1eXyzJGkHNtvj3J4NQxIyKqAD1ZCS/C6e8aZkfRbDXYBL2u3SEyYl1gkXpy4DZH6xtDVlHJO1uEQ1G55V1FIvJKWohJy2jehe9OIROlemiUiVYDuisU0LzUcOONz9R9NF27f8GCoE/LmRvyHbtbIZ1Zh/lOhKnI/gMIlq6FqpFqpvzW6uohMoo8/w2H47sCMRnUq1LhlAV389yLeTO5C0pGDoCWkaraNiUATBVIvMbk3CTaAGMmKzbXJQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO1PR10MB4468.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230016)(366004)(66556008)(66946007)(1076003)(66476007)(8676002)(4326008)(186003)(5660300002)(2616005)(36756003)(52116002)(6506007)(6666004)(83380400001)(8936002)(26005)(6512007)(86362001)(103116003)(2906002)(38350700002)(6486002)(38100700002)(316002)(508600001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?6wnsL2q1+hakzd/ovhYkey5wf4DY1+ubm+/2iktqQqkZYYYLgYjrGaownkS9?= =?us-ascii?Q?PBtbOSsuVXeRTEuvOEX++aSX4oHk4I1H5yZI2rnG8KcUg1KqLMHLw6lTM/6j?= =?us-ascii?Q?ezjfAoJGVlnPgM5E/z5pO2jnGlT6vFxgCfdmjDeAe6GkjyJSx9kHrb8/kN5o?= =?us-ascii?Q?YQaoYlzmCsta0LfantqB3GDAVQLoNEBnGiQdP7D1AcU/OXGCQlT10E6Gs8tI?= =?us-ascii?Q?4MuxFUAutoSh0Ktjm1GDaJLxiVUoLX8D2kJtZj95TOxHO4uyJQ+StdLWByVc?= =?us-ascii?Q?30D7mlM7oZVOcoQsJ14PzkKuvWIRhWUXr2dgzWTfXBV/Wn+65NYyd6DWypv6?= =?us-ascii?Q?5NGE3UTBAJFgGr6u7k6ybjl4fBTNhwR80HAU8iFuuNekimASMYvygwYsd6DD?= =?us-ascii?Q?B+NesoonSQVm45V+rsbHGuaTkYOfNZQOn8Ye5H2A9FVezEpx0nP1/XaAQcb/?= =?us-ascii?Q?EVG8MGVtOj3rkqQj5bndRRG7M7PU02FL0FpWCny/VYnygqzFYpBt8O+KbbZR?= =?us-ascii?Q?7b7dg6uFyG9LxdxpqpOn+70DVjPK5KPLnz0KL71BwADrAfhE7D81EwHFncGo?= =?us-ascii?Q?AT4au4vpfywk8zeGVhEZD13wnPxtntyVM40m50ixqh2Ou/cATXqXA/SdLe0S?= =?us-ascii?Q?FxlasubOliQ4aGcZjAaNo2BM8NVw0GJ5EN8wbl0fYJwvwpfMKcAmJSlSpriD?= =?us-ascii?Q?m81NNI6AxMjtBhx1hONNDG8AqnyBmjXdA7K0hTGmF4j98SrCMSz12CcP4VEb?= =?us-ascii?Q?eJKUICly9Hp8xel4G+ul+sUYTk708QFdae8M2rVty1ygNZVlLAbur+zEDLGr?= =?us-ascii?Q?CBEa8yvJp2Bs0MmgOwkQriHefs6C4yMpAOVCQXZgPiAH5Lx0EyOOXLUMts8n?= =?us-ascii?Q?JWA7R0+W0vhA3FeclwJwh3KcdTESgtay9Km0mQGPkFTRHvE7jNEO7qytlEIS?= =?us-ascii?Q?jRXDqsHLvaVyLMoAYP94z76ZSOh+AM9ydnNAMdmmWvKb1MUkZSBYZLR/TZSV?= =?us-ascii?Q?NL1AUDc6slBCP7pRYpKAp8jd0MD4j6vcPNMWN+6iQkTuXOP0Za63wcX2xdCD?= =?us-ascii?Q?Jz9J0+mKPhcTa8AiKGetKHQyCb/w0VxOrkGYei8ksxJYXdKarjVbF0kD98sQ?= =?us-ascii?Q?doISmrLTzpJyQMjACf5us3FjusPVtyyQTaALYCVXLf2VNdOeUnWZtLmC9kUS?= =?us-ascii?Q?6OdS9BZgG4/Di3PQCF4v1PXWPKN8alunmKMIXVV8/3VOc6yauSDz+PFadVPn?= =?us-ascii?Q?iCo/O1/yX1DzB7i7RTQvtA6s6kUOUBSwIlHlQct5mzqeUvXMT172+0bVlqX7?= =?us-ascii?Q?5q6r3CfNugDbfVn5xjzPjprUAbSJCUCjhCF46iisegr2LUAmPbS/ChwHRydf?= =?us-ascii?Q?TKC8brgEF5Wgp5nG9deFcboqtRFpSvQNgojEtwjNw2hStfqUsL/eS+vNWq/b?= =?us-ascii?Q?Fd6GP3aMWYg50qBOoMFXisCW2IaWhLh7u/mXMP+OChZGrrTPMrOG1q6JYPuc?= =?us-ascii?Q?Bx7Nk/tYvQS29w/VnaKSpu/Z4XA/a5woIdSA7M8/BPqqOrovOLeEYxGfC9P7?= =?us-ascii?Q?GaWL5YzcSn0oI+0AShR3eBEUhvMVTt3ep4jdJIFBwr0ACwA1vYqIXyWAK5pN?= =?us-ascii?Q?8p+9I1MLntdj21L7enHSzFPdB4QI497kBrQRsmGFaJYo2DlQImUlWZGzi5/0?= =?us-ascii?Q?1W8nbBy4d3mhlfX1EFUFnLKbxYX7gRmCCtnWLKstXYQVDGJcPe8w/pQNKLI7?= =?us-ascii?Q?+lfbLdq0qqdmchiMzNbLmYRTifXfU48=3D?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 14ff8109-13af-4d22-049a-08da4daad6ec X-MS-Exchange-CrossTenant-AuthSource: CO1PR10MB4468.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Jun 2022 02:08:57.3011 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ysus6idDPLvHqRBZE5aucdjqwqh9Awq/pyTEyPimz2pDZs3bisw40cSWc2RQdb6Qx1DtwDCFTqDMWWzUuDVJrQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR1001MB2398 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.517,18.0.874 definitions=2022-06-14_01:2022-06-13,2022-06-13 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 phishscore=0 adultscore=0 spamscore=0 malwarescore=0 mlxscore=0 mlxlogscore=999 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2204290000 definitions=main-2206140007 X-Proofpoint-GUID: FJEgKmShpcVzUsbY8tXCDYwYm0orTAY9 X-Proofpoint-ORIG-GUID: FJEgKmShpcVzUsbY8tXCDYwYm0orTAY9 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This allows to change underlying mutex locking, without needing to change the users of the lock. For example next patch modifies this interface to use hashed mutexes in place of a single global kernfs_open_file_mutex. Signed-off-by: Imran Khan Acked-by: Tejun Heo --- fs/kernfs/file.c | 60 ++++++++++++++++++++++++++++++++---------------- 1 file changed, 40 insertions(+), 20 deletions(-) diff --git a/fs/kernfs/file.c b/fs/kernfs/file.c index 0cc288dcf51d9..205b5c71f30cb 100644 --- a/fs/kernfs/file.c +++ b/fs/kernfs/file.c @@ -49,6 +49,22 @@ struct kernfs_open_node { =20 static LLIST_HEAD(kernfs_notify_list); =20 +static inline struct mutex *kernfs_open_file_mutex_ptr(struct kernfs_node = *kn) +{ + return &kernfs_open_file_mutex; +} + +static inline struct mutex *kernfs_open_file_mutex_lock(struct kernfs_node= *kn) +{ + struct mutex *lock; + + lock =3D kernfs_open_file_mutex_ptr(kn); + + mutex_lock(lock); + + return lock; +} + /** * kernfs_deref_open_node - Get kernfs_open_node corresponding to @kn. * @@ -56,7 +72,7 @@ static LLIST_HEAD(kernfs_notify_list); * @kn: target kernfs_node. * * Fetch and return ->attr.open of @kn if @of->list is not empty or if - * kernfs_open_file_mutex is held. + * kernfs_open_file_mutex_ptr(kn) is held. * * We can rely on either of the following 2 conditions to ensure that * dereferencing ->attr.open outside RCU read-side critical section is saf= e. @@ -68,13 +84,13 @@ static LLIST_HEAD(kernfs_notify_list); * * or * - * 2. Update of ->attr.open happens under kernfs_open_file_mutex. So as lo= ng as - * the current updater (caller) is holding this mutex, other updaters will= not - * be able to change ->attr.open and this means that we can safely deref - * ->attr.open outside RCU read-side critical section. + * 2. Update of ->attr.open happens under kernfs_open_file_mutex_ptr(kn). = So as + * long as the current updater (caller) holds this mutex, other updaters c= an't + * change ->attr.open and this means that we can safely deref ->attr.open + * outside RCU read-side critical section. * * The caller needs to make sure that either @of->list is not empty or - * kernfs_open_file_mutex is held. + * kernfs_open_file_mutex_ptr(kn) is held. */ static struct kernfs_open_node * kernfs_deref_open_node(struct kernfs_open_file *of, struct kernfs_node *kn) @@ -83,7 +99,7 @@ kernfs_deref_open_node(struct kernfs_open_file *of, struc= t kernfs_node *kn) bool deref_ok; =20 deref_ok =3D (of ? !list_empty(&of->list) : false) || - lockdep_is_held(&kernfs_open_file_mutex); + lockdep_is_held(kernfs_open_file_mutex_ptr(kn)); =20 on =3D rcu_dereference_check(kn->attr.open, deref_ok); =20 @@ -570,19 +586,20 @@ static int kernfs_get_open_node(struct kernfs_node *k= n, struct kernfs_open_file *of) { struct kernfs_open_node *on, *new_on =3D NULL; + struct mutex *mutex =3D NULL; =20 - mutex_lock(&kernfs_open_file_mutex); + mutex =3D kernfs_open_file_mutex_lock(kn); on =3D kernfs_deref_open_node(NULL, kn); =20 if (on) { list_add_tail(&of->list, &on->files); - mutex_unlock(&kernfs_open_file_mutex); + mutex_unlock(mutex); return 0; } else { /* not there, initialize a new one */ new_on =3D kmalloc(sizeof(*new_on), GFP_KERNEL); if (!new_on) { - mutex_unlock(&kernfs_open_file_mutex); + mutex_unlock(mutex); return -ENOMEM; } atomic_set(&new_on->event, 1); @@ -591,7 +608,7 @@ static int kernfs_get_open_node(struct kernfs_node *kn, list_add_tail(&of->list, &new_on->files); rcu_assign_pointer(kn->attr.open, new_on); } - mutex_unlock(&kernfs_open_file_mutex); + mutex_unlock(mutex); =20 return 0; } @@ -613,12 +630,13 @@ static void kernfs_unlink_open_file(struct kernfs_nod= e *kn, struct kernfs_open_file *of) { struct kernfs_open_node *on; + struct mutex *mutex =3D NULL; =20 - mutex_lock(&kernfs_open_file_mutex); + mutex =3D kernfs_open_file_mutex_lock(kn); =20 on =3D kernfs_deref_open_node(NULL, kn); if (!on) { - mutex_unlock(&kernfs_open_file_mutex); + mutex_unlock(mutex); return; } =20 @@ -630,7 +648,7 @@ static void kernfs_unlink_open_file(struct kernfs_node = *kn, kfree_rcu(on, rcu_head); } =20 - mutex_unlock(&kernfs_open_file_mutex); + mutex_unlock(mutex); } =20 static int kernfs_fop_open(struct inode *inode, struct file *file) @@ -772,7 +790,7 @@ static void kernfs_release_file(struct kernfs_node *kn, * here because drain path may be called from places which can * cause circular dependency. */ - lockdep_assert_held(&kernfs_open_file_mutex); + lockdep_assert_held(kernfs_open_file_mutex_ptr(kn)); =20 if (!of->released) { /* @@ -789,11 +807,12 @@ static int kernfs_fop_release(struct inode *inode, st= ruct file *filp) { struct kernfs_node *kn =3D inode->i_private; struct kernfs_open_file *of =3D kernfs_of(filp); + struct mutex *mutex =3D NULL; =20 if (kn->flags & KERNFS_HAS_RELEASE) { - mutex_lock(&kernfs_open_file_mutex); + mutex =3D kernfs_open_file_mutex_lock(kn); kernfs_release_file(kn, of); - mutex_unlock(&kernfs_open_file_mutex); + mutex_unlock(mutex); } =20 kernfs_unlink_open_file(kn, of); @@ -808,6 +827,7 @@ void kernfs_drain_open_files(struct kernfs_node *kn) { struct kernfs_open_node *on; struct kernfs_open_file *of; + struct mutex *mutex =3D NULL; =20 if (!(kn->flags & (KERNFS_HAS_MMAP | KERNFS_HAS_RELEASE))) return; @@ -823,10 +843,10 @@ void kernfs_drain_open_files(struct kernfs_node *kn) if (!rcu_access_pointer(kn->attr.open)) return; =20 - mutex_lock(&kernfs_open_file_mutex); + mutex =3D kernfs_open_file_mutex_lock(kn); on =3D kernfs_deref_open_node(NULL, kn); if (!on) { - mutex_unlock(&kernfs_open_file_mutex); + mutex_unlock(mutex); return; } =20 @@ -840,7 +860,7 @@ void kernfs_drain_open_files(struct kernfs_node *kn) kernfs_release_file(kn, of); } =20 - mutex_unlock(&kernfs_open_file_mutex); + mutex_unlock(mutex); } =20 /* --=20 2.30.2 From nobody Wed Apr 29 12:25:12 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6E4AC433EF for ; Tue, 14 Jun 2022 02:22:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1354219AbiFNCWI (ORCPT ); Mon, 13 Jun 2022 22:22:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57252 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1353389AbiFNCP0 (ORCPT ); Mon, 13 Jun 2022 22:15:26 -0400 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 268103DA52 for ; Mon, 13 Jun 2022 19:09:20 -0700 (PDT) Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 25E1CBBT025664; Tue, 14 Jun 2022 02:09:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=N9cVDxq5TpqWNYM9+Z2Y7W5kQBF1X61g07ox915eg4E=; b=MAyvkG6pRcD36EAkPWrqZP3oCPWJrIJvL0SBXzrtNFEU+cUhWsyDjCoIeokPeNFDAC4x bvB6UOaOHb0D5pA1mbU2OjK1v/1wOeJFZ46KEn4uLEj0xC2OKL145ThSCvnvvNAxFoYY kQsV9O9cCGrS67YHLt0opaKFvuLCHDOoxp3atH62UDmDA5QbOHiAudeIyRBqyvqVVEpP YiiPVRp8NoLa9PGOy1obum2EHZU5N943rlYlblza24cnSUbNXD/4oLzSjzZCYAqQJVCf 3N8HRRFVQYkPe5GtKa62+1Dchbkah7ZSs224HKbs3XXtf8uSZQ+R5HAM1Ee1p3Y/bmhq QQ== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3gmjx9cg8t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 14 Jun 2022 02:09:03 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 25E21wTD007179; Tue, 14 Jun 2022 02:09:01 GMT Received: from nam11-co1-obe.outbound.protection.outlook.com (mail-co1nam11lp2173.outbound.protection.outlook.com [104.47.56.173]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com with ESMTP id 3gph6kg2va-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 14 Jun 2022 02:09:00 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=B2rmhGqnUjAnKjGZV2HrClXQOdYG7t5fTvxrenPsVcwi8QdV+szMwXxeH2DdAZtX61JLt68E3aO7tKF4nKrojSVcwEjKaoK1L29mDQSYvaOaWiji4JO9kc7VZj6sXcOFwU9BLBe3zV8hHIJZeV9myLll47Zbu12/h6p/f33RjFykltPCcftbhUNZpvkyqPU+3pOiQ4T/AFhN80gGX8XLjP54RSL+vmP+ZZqsHaT1ixSeCppIHsTRQk1jQPUMoYZZw/soFyWkdAv/MB1/bbEJoAD0sHwE6NLKtCC010lu+FxfNYh5aFYZMK1W2UdRIJj/3p3zqoVq5uoZ82ZHQgUTHA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=N9cVDxq5TpqWNYM9+Z2Y7W5kQBF1X61g07ox915eg4E=; b=dJsjArs2aeH7Abh6EGLF6DHzWPCJaGADR/p24ZZmMXGcAbyzadVWtharlPe8a3tCBq8oInISzfIbGtzAAfc66K7baFQdFyeSoctgMDLVw4fETmhoXMIUgLgEWB5fiQbuOCroARDuUvlM7ZIzgpByxBpbe+cKZb9MiOlWApoMFKXiikoYzkl/SAKd2gdVV1y6oFfccOsjLh2LoRGDFBeeuhssI5PtViNKkg7wd2/y4DqIGEoDVJU/ahRZB/09H0M/Ov5M3Xqtr6yBjPbF00kwM4eOYUWroTqMhS+fjMG1/zdHO3LLGn9e/jRf5rhFX0hat9kma0iZMId+yb3cxmw/IA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=N9cVDxq5TpqWNYM9+Z2Y7W5kQBF1X61g07ox915eg4E=; b=GLL1DUsUbRGpLO6NnUL0NzLAhKHWthaulUjdhxXsT77R9J6FIQhC2KdVtrOuAE7dMgdwDstkGnQL9k/xIR16rPfpYTI9enOynmn1ohqV2xipI0T1aIPkfMiI9JE6vgAWqraMAJne3W2ObMK64xtE+qSCrhrjnYbztExOu0+5C4w= Received: from CO1PR10MB4468.namprd10.prod.outlook.com (2603:10b6:303:6c::24) by MWHPR1001MB2398.namprd10.prod.outlook.com (2603:10b6:301:2f::37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5332.20; Tue, 14 Jun 2022 02:08:58 +0000 Received: from CO1PR10MB4468.namprd10.prod.outlook.com ([fe80::e9bd:ba52:c1c2:11e0]) by CO1PR10MB4468.namprd10.prod.outlook.com ([fe80::e9bd:ba52:c1c2:11e0%9]) with mapi id 15.20.5332.021; Tue, 14 Jun 2022 02:08:58 +0000 From: Imran Khan To: tj@kernel.org, gregkh@linuxfoundation.org, viro@zeniv.linux.org.uk Cc: linux-kernel@vger.kernel.org Subject: [PATCH v5 4/4] kernfs: Replace global kernfs_open_file_mutex with hashed mutexes. Date: Tue, 14 Jun 2022 12:08:40 +1000 Message-Id: <20220614020840.546013-5-imran.f.khan@oracle.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220614020840.546013-1-imran.f.khan@oracle.com> References: <20220614020840.546013-1-imran.f.khan@oracle.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SY6PR01CA0036.ausprd01.prod.outlook.com (2603:10c6:10:eb::23) To CO1PR10MB4468.namprd10.prod.outlook.com (2603:10b6:303:6c::24) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 4725fbd5-8e5c-4747-8c49-08da4daad7d5 X-MS-TrafficTypeDiagnostic: MWHPR1001MB2398:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: eTadRJbK7xhB4ypq26FxAK9GwZ2v3jnfHvGHMLaveKXC6vl+5g25O9s1JN6o7w9Zw+O3vWO/BiWudXrg54ImaTXUi2Jxqhix3aRBibdZOgf1Q0bIpHDWbjE5HDdmXaqrcnCb9Wjtcg90t63GU8Wg+yhNc2MaPE2ys2Fjx6pHKD2MIHRDgkws6fAaZaUeOtbxiXbCIWzKavXkW3DoQ8pJHgYbZgCvUqmRE9BLexUMAgGS+DX9DVb24E0ZlDQ2631E4h10jE1AUSCcU8X+YiqXOOjJei4Ygl5GiKVxPB8BwyTh703OBpeK7tThBACBzyG6+mJaMw7fZLOd6/UCEfmnpZmiCXBUfGvFkIzufNv5Iv3u+Qd/yYDFgZKpi8zAx2NqGMl346kiTlU3NWaUSNf7z+WT50VSrIJawpK6lrIRaf5uIZ6h2Q2RV8lqHhkwhD06LZjftsbGPABjpmfA0MG+8/oOCna2h0BijrOx2AvXOU9q9OCHfY3O7K1NgL/w4hlNmqtGx+WZAUfRNlKZUb4cBKvWSfcE/B5Pa7o1yyCjDWLww4U3NGVbejnAPASUqngpKRJZHeDa2DcfwLAwx/oVUCdq3vx+r5ic19Kpd5HWKgRxkBZQtYz8GpYD5gPspp6zBGg4+Cx5+cKAxFa6thn/W1W6V8oU9bvumcZua9linxdP2vCLvWgn2WLILF5VcOkdErDMzy6i/9jwMMIlhZdsjw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO1PR10MB4468.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230016)(366004)(66556008)(66946007)(1076003)(66476007)(8676002)(4326008)(186003)(5660300002)(2616005)(36756003)(52116002)(6506007)(6666004)(83380400001)(8936002)(26005)(6512007)(86362001)(103116003)(2906002)(38350700002)(6486002)(38100700002)(316002)(508600001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?jYlDP3DfT96z4Khij80gGog6f+SLZJwYtZqTB1LJLRRBf7phNJZc6Bsy6CQp?= =?us-ascii?Q?L79F2J0RZKue+JA+ZL08oAE6CpTmtMvJ9w0Bh5B56Fv+T1MsQM9qUiv79sD9?= =?us-ascii?Q?nVJtawBDFbKfGMSkYxjwSlGo/1jkBQE741bqhYzCTFPPJC/8L1fp7xG1ybj0?= =?us-ascii?Q?sTCF5wYo6+Md5jCBVNo4BzBkEnqS/AEOjloejb8up3928xOJbogABUUDsqqP?= =?us-ascii?Q?ccsJJ1i2c0t2bFt8P35wZq2wCgVN3QZxAPrG2xRPABpdEFkz7rhYDjeDfX2O?= =?us-ascii?Q?Fqwyb0ELtxKriyyIFBC/TsmhsJWDx1YPEQbcc2SVWBxMt2+8eACK2c/RfBFc?= =?us-ascii?Q?AoWeAxopl8mhNGhtrO9Bt7nl4w51RUIacpU0Qah/fTtK5scLQ0tnAxCiJ+dB?= =?us-ascii?Q?+svrljxTq7VflY8h/VTnjCE6GRwe2VlJ2tvEafCjAYFs5l9RyOjRxmtcU9Dw?= =?us-ascii?Q?n7Ewhxe0PLf2Mkp2l5XiZcFZwt0aacZMX00nC/LEKpNIQzsT1+PByPEVBCJi?= =?us-ascii?Q?RUrLp9QMR/i0iz2r70PGNb0Av1KYtJFHmmBEFyYWqpw/vP0xgFAl5lIDbsCB?= =?us-ascii?Q?D+Tufo1iHEX3MwsqAEU9zi5tsIM8ybqpUBi9UeAAw6DeoesnTqwVJMK2wWmX?= =?us-ascii?Q?LP2qm9ASt4JEdmmLA2PRxOrKnEEq5x26LMK5X3b+u/XU5TnI5sS+j0FG7cAF?= =?us-ascii?Q?e+tbVUB5Jq+6R/rErtrDrn5qF3HKXVQBwIywhNNhQ6VVpRWZ/Nikyds2zI/O?= =?us-ascii?Q?j4YBZZdaBuIeVz2C0KbLMfUIjlu4BpEBhF//G6PdhzXBlGRfigl2Oe0/dvIv?= =?us-ascii?Q?q6ul3kHbIuzSXG2D5QGRo84jSJO3fPYwk9b5fi0kZnz3ljda4jCwvOZge+Dx?= =?us-ascii?Q?IZWFWF5xoHxt1pAi2OQaFaVHt2a2XhYiMJxtbLwy0OZdkaDoPYJgHkVdda66?= =?us-ascii?Q?oBUwtQ2tQAGMxFIikYvkma+stVI8TC42I+eNj2IZ79J0rcbBxnXc1+7sB+tA?= =?us-ascii?Q?hoUig/9Crcc4c0wRoiqwZrMhdaR5IIaZkRvBDE8yasfVm2Sy3myQAhN7pgY+?= =?us-ascii?Q?6deNZeIJMrljZGoKfuNV4sOXqdn+2y9AJwXJekaHjN1J3OD3LaUzjkdWbW99?= =?us-ascii?Q?GnR5yblKS6RcyjV6WgsOteIUBQ07of4x5ozoXhuMWXpyP8JjV0pgFUx7sbjf?= =?us-ascii?Q?a/m9XjhnGEQRUdhNIJMqNWPTw+qJIexGxjrkXXQXbyFSyfDI0iN6sjjKsjpj?= =?us-ascii?Q?iFw7ouUDBbUpJxrFA1XhRwpDiEdYH2gm5duFj7NYb6rJ9YGyTcxZW0hy9yuk?= =?us-ascii?Q?8itzIzYioaH4VjvYE92+B6K9h1SPVfrrI04ncFpqPuPn9FTbxMnG/Y4ptKSL?= =?us-ascii?Q?j6kQY9ja1CNNDbgVpVxt51M6CaxHvRN5pxjUsa3Pg/vhU54Im6OjbRJ4Ym4O?= =?us-ascii?Q?ibDHaovyLtHmAgpID5ylII9CN3DabjjJhCjPZy7Z8v3/a/paAdNXMAoAgi1F?= =?us-ascii?Q?g0w36y2pD26xZkQ8MPHykAYObnjotPAIubhJZ2Hu0+doN+DZhGSqJIaV6PRz?= =?us-ascii?Q?vgstmNjNCdHx2GLyZaZhYGrjWhuvtUWHud5JcvPOB9Cm/nN2XPjBFvBXnacP?= =?us-ascii?Q?p8pX5teu2MUMqBowbog1N+oNYosjWGJRnmbLZ+gXGAFxBHpieC4dtwup5X5z?= =?us-ascii?Q?dnra0r3yYXpgYLgQpz8QQNPCst5yo3wp3MY3EqgGfRCxiOVQz1EqTc3Ogo7l?= =?us-ascii?Q?8zniLXRkC+vsnzbRbhcCLvzEjLOpiIY=3D?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4725fbd5-8e5c-4747-8c49-08da4daad7d5 X-MS-Exchange-CrossTenant-AuthSource: CO1PR10MB4468.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Jun 2022 02:08:58.8628 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 3zHCgTZKlVllN+BLGgybfuv9hI9DzUQ+qYsNQbT4XOfbem2c5kBd+hlaf67rGtGMOStcq83W/AlvLqEu0u3Zfw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR1001MB2398 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.517,18.0.874 definitions=2022-06-14_01:2022-06-13,2022-06-13 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxscore=0 mlxlogscore=999 malwarescore=0 phishscore=0 spamscore=0 suspectscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2204290000 definitions=main-2206140007 X-Proofpoint-ORIG-GUID: akbrqYTuX3GBa3-z9UEEixC4_VbDObei X-Proofpoint-GUID: akbrqYTuX3GBa3-z9UEEixC4_VbDObei Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" In current kernfs design a single mutex, kernfs_open_file_mutex, protects the list of kernfs_open_file instances corresponding to a sysfs attribute. So even if different tasks are opening or closing different sysfs files they can contend on osq_lock of this mutex. The contention is more apparent in large scale systems with few hundred CPUs where most of the CPUs have running tasks that are opening, accessing or closing sysfs files at any point of time. Using hashed mutexes in place of a single global mutex, can significantly reduce contention around global mutex and hence can provide better scalability. Moreover as these hashed mutexes are not part of kernfs_node objects we will not see any singnificant change in memory utilization of kernfs based file systems like sysfs, cgroupfs etc. Modify interface introduced in previous patch to make use of hashed mutexes. Use kernfs_node address as hashing key. Signed-off-by: Imran Khan Acked-by: Tejun Heo --- fs/kernfs/file.c | 17 ++--------- fs/kernfs/kernfs-internal.h | 4 +++ fs/kernfs/mount.c | 19 +++++++++++++ include/linux/kernfs.h | 57 +++++++++++++++++++++++++++++++++++++ 4 files changed, 83 insertions(+), 14 deletions(-) diff --git a/fs/kernfs/file.c b/fs/kernfs/file.c index 205b5c71f30cb..d821c20c59eaf 100644 --- a/fs/kernfs/file.c +++ b/fs/kernfs/file.c @@ -18,19 +18,6 @@ =20 #include "kernfs-internal.h" =20 -/* - * There's one kernfs_open_file for each open file and one kernfs_open_node - * for each kernfs_node with one or more open files. - * - * kernfs_node->attr.open points to kernfs_open_node. attr.open is - * RCU protected. - * - * filp->private_data points to seq_file whose ->private points to - * kernfs_open_file. kernfs_open_files are chained at - * kernfs_open_node->files, which is protected by kernfs_open_file_mutex. - */ -static DEFINE_MUTEX(kernfs_open_file_mutex); - struct kernfs_open_node { struct rcu_head rcu_head; atomic_t event; @@ -51,7 +38,9 @@ static LLIST_HEAD(kernfs_notify_list); =20 static inline struct mutex *kernfs_open_file_mutex_ptr(struct kernfs_node = *kn) { - return &kernfs_open_file_mutex; + int idx =3D hash_ptr(kn, NR_KERNFS_LOCK_BITS); + + return &kernfs_locks->open_file_mutex[idx]; } =20 static inline struct mutex *kernfs_open_file_mutex_lock(struct kernfs_node= *kn) diff --git a/fs/kernfs/kernfs-internal.h b/fs/kernfs/kernfs-internal.h index eeaa779b929c7..3ae214d02d441 100644 --- a/fs/kernfs/kernfs-internal.h +++ b/fs/kernfs/kernfs-internal.h @@ -164,4 +164,8 @@ void kernfs_drain_open_files(struct kernfs_node *kn); */ extern const struct inode_operations kernfs_symlink_iops; =20 +/* + * kernfs locks + */ +extern struct kernfs_global_locks *kernfs_locks; #endif /* __KERNFS_INTERNAL_H */ diff --git a/fs/kernfs/mount.c b/fs/kernfs/mount.c index cfa79715fc1a7..d0859f72d2d64 100644 --- a/fs/kernfs/mount.c +++ b/fs/kernfs/mount.c @@ -20,6 +20,7 @@ #include "kernfs-internal.h" =20 struct kmem_cache *kernfs_node_cache, *kernfs_iattrs_cache; +struct kernfs_global_locks *kernfs_locks; =20 static int kernfs_sop_show_options(struct seq_file *sf, struct dentry *den= try) { @@ -387,6 +388,22 @@ void kernfs_kill_sb(struct super_block *sb) kfree(info); } =20 +static void __init kernfs_mutex_init(void) +{ + int count; + + for (count =3D 0; count < NR_KERNFS_LOCKS; count++) + mutex_init(&kernfs_locks->open_file_mutex[count]); +} + +static void __init kernfs_lock_init(void) +{ + kernfs_locks =3D kmalloc(sizeof(struct kernfs_global_locks), GFP_KERNEL); + WARN_ON(!kernfs_locks); + + kernfs_mutex_init(); +} + void __init kernfs_init(void) { kernfs_node_cache =3D kmem_cache_create("kernfs_node_cache", @@ -397,4 +414,6 @@ void __init kernfs_init(void) kernfs_iattrs_cache =3D kmem_cache_create("kernfs_iattrs_cache", sizeof(struct kernfs_iattrs), 0, SLAB_PANIC, NULL); + + kernfs_lock_init(); } diff --git a/include/linux/kernfs.h b/include/linux/kernfs.h index 2dd9c8df0f4f6..13e703f615f79 100644 --- a/include/linux/kernfs.h +++ b/include/linux/kernfs.h @@ -18,6 +18,7 @@ #include #include #include +#include =20 struct file; struct dentry; @@ -34,6 +35,62 @@ struct kernfs_fs_context; struct kernfs_open_node; struct kernfs_iattrs; =20 +/* + * NR_KERNFS_LOCK_BITS determines size (NR_KERNFS_LOCKS) of hash + * table of locks. + * Having a small hash table would impact scalability, since + * more and more kernfs_node objects will end up using same lock + * and having a very large hash table would waste memory. + * + * At the moment size of hash table of locks is being set based on + * the number of CPUs as follows: + * + * NR_CPU NR_KERNFS_LOCK_BITS NR_KERNFS_LOCKS + * 1 1 2 + * 2-3 2 4 + * 4-7 4 16 + * 8-15 6 64 + * 16-31 8 256 + * 32 and more 10 1024 + * + * The above relation between NR_CPU and number of locks is based + * on some internal experimentation which involved booting qemu + * with different values of smp, performing some sysfs operations + * on all CPUs and observing how increase in number of locks impacts + * completion time of these sysfs operations on each CPU. + */ +#ifdef CONFIG_SMP +#define NR_KERNFS_LOCK_BITS (2 * (ilog2(NR_CPUS < 32 ? NR_CPUS : 32))) +#else +#define NR_KERNFS_LOCK_BITS 1 +#endif + +#define NR_KERNFS_LOCKS (1 << NR_KERNFS_LOCK_BITS) + +/* + * There's one kernfs_open_file for each open file and one kernfs_open_node + * for each kernfs_node with one or more open files. + * + * filp->private_data points to seq_file whose ->private points to + * kernfs_open_file. + * + * kernfs_open_files are chained at kernfs_open_node->files, which is + * protected by kernfs_global_locks.open_file_mutex[i]. + * + * To reduce possible contention in sysfs access, arising due to single + * locks, use an array of locks (e.g. open_file_mutex) and use kernfs_node + * object address as hash keys to get the index of these locks. + * + * Hashed mutexes are safe to use here because operations using these don't + * rely on global exclusion. + * + * In future we intend to replace other global locks with hashed ones as w= ell. + * kernfs_global_locks acts as a holder for all such hash tables. + */ +struct kernfs_global_locks { + struct mutex open_file_mutex[NR_KERNFS_LOCKS]; +}; + enum kernfs_node_type { KERNFS_DIR =3D 0x0001, KERNFS_FILE =3D 0x0002, --=20 2.30.2