From nobody Sat Nov 23 22:28:18 2024 Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5311A18C36 for ; Sat, 9 Nov 2024 01:28:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731115737; cv=none; b=EyEx9+v4NgSo/J9MtIJZ5xzQd2xFLB+IiaAJ2zBhSTDKXeEV0yLktS7x2YsRq3uflOE2oZoNDM3T0VtE84p5JwnJ/SkNiwQTMLfVYoswKKFIP1QTntDDL8NVvvmBToJLSDPx+HFTHyvBabz60NKwJmICx+/xjE/9bmE5uRqkLkQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731115737; c=relaxed/simple; bh=U4/39m1wSnM6NEKJmtt/qupJOrq3/D+YkHzzTDlhxIY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=sb0guesAexEv2GpKz1f9RHCdj3lNVW9MlKJWT6+Uy+5JyKVX6TbQ2MdCwXxXephnUYv6POKDRBCq1HdZp/+UCaLix7xlmizTmwCnwRTyNKYCQZiQvxOqrk5AvL2SPCvVcgeAwU336ACmJbti7/UZTI1xVglXKndQusFpzHUIKEo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=osandov.com; spf=none smtp.mailfrom=osandov.com; dkim=pass (2048-bit key) header.d=osandov-com.20230601.gappssmtp.com header.i=@osandov-com.20230601.gappssmtp.com header.b=GiyvBhg4; arc=none smtp.client-ip=209.85.214.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=osandov.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=osandov.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=osandov-com.20230601.gappssmtp.com header.i=@osandov-com.20230601.gappssmtp.com header.b="GiyvBhg4" Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-20c83c2e967so2052865ad.1 for ; Fri, 08 Nov 2024 17:28:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osandov-com.20230601.gappssmtp.com; s=20230601; t=1731115736; x=1731720536; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1JQhJjVLe/b9J3jK6ApFcepDaYbpXUd3aMShB4aDox4=; b=GiyvBhg4DWVWTHZf2VGtIgqWYyCAKimbPYxH7LjPXINyntoKf1QIWMv/Dvvg45aKtX QJhk1UrRTsnPk5nzJUrcKMzfhStyl3yoqOhvZiP1wrBKZVV7RXhEAW55N09WA662DYNy M5m7YL0f9oF2ju7fYgKaCMU44N1T0TBezqqwMWPYQtYqOfENfmQck544h66v2p2bOD26 fht5MBIf9/qzauxor3SKhAJAxZlzLDxe7S4ONw3+ctQkmjgvQ3bRtkEEVAufUN/YG4QQ Cg/1dJOqjF0Pv5vqcX0wM0Svyfxv0r8StvPHk/BlALPMRYOyWx1Bbo+Aw8IZxqvUcKRJ uobQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731115736; x=1731720536; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1JQhJjVLe/b9J3jK6ApFcepDaYbpXUd3aMShB4aDox4=; b=IJ+i8qDKl/cg2EM4KjQ8zdf9IrJTLrd5JGgPj1r9oyMAkMmuFpEp/3YNyzyuDyhfPK VeebYI1eDZS6TH1DqSAQBb4YrZHWiVFATbZlMOlLckzyyhYAx5w+odLbt1/h7a8t7LOy T7e59GEuW+BHCm5rlOA4hlEIoAlBkF2QmAsCr1bldo8Z+rk65s8qgRdvsOdVTgciWd1a OMvHZnfGmU0sLDi+WxoiGHUGzzciq2DHqKTCqyn8TaEoWV3f96X2jrIKkbPSesxPYpoC P56Lx1o1KFdhQkTkmcz1SukgIioUfSdqowqvjXux2zEjmU+lFiHoMK7TIF7OdtlseKIX N4Uw== X-Forwarded-Encrypted: i=1; AJvYcCXZfA4xuWrAT1AFrPnIRhhtF3V1Mg7lZNkyuiMOK0GWkGtBR53BhpiFU2nvguO6ZP2Q2fTeBvmKNVZAPSI=@vger.kernel.org X-Gm-Message-State: AOJu0YxnIWDJbbcRTSqXHjTjxTbHo3zrc9anqKjJKn5fKt++xk7CFLes +1/+BXEOUHE7/Nt/jiSiYlqO2/+Tvgs4gB34qJs1efDNZ7+oLTayuroVqHfUXaY= X-Google-Smtp-Source: AGHT+IFKg+q7iegnZHiSf+idYwfgc2LKuVrDKq4p8WHgcGx0frkz7U7cZpi6uSNtCUvntKMxtTgqog== X-Received: by 2002:a17:902:d4cd:b0:20c:da9a:d5b9 with SMTP id d9443c01a7336-211835107c0mr29325685ad.5.1731115735628; Fri, 08 Nov 2024 17:28:55 -0800 (PST) Received: from telecaster.hsd1.wa.comcast.net ([2601:602:8980:9170::5633]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21177e6c96fsm37493355ad.255.2024.11.08.17.28.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Nov 2024 17:28:54 -0800 (PST) From: Omar Sandoval To: linux-fsdevel@vger.kernel.org, Al Viro Cc: kernel-team@fb.com, linux-kernel@vger.kernel.org Subject: [PATCH 1/4] proc/kcore: mark proc entry as permanent Date: Fri, 8 Nov 2024 17:28:39 -0800 Message-ID: <60873e6afcfda3f08d0456f19e4733612afcf134.1731115587.git.osandov@fb.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Omar Sandoval drgn reads from /proc/kcore to debug the running kernel. For many drgn scripts, /proc/kcore is actually a bottleneck. use_pde() and unuse_pde() in prog_reg_read() show up hot in profiles. Since the entry for /proc/kcore can never be removed, this is useless overhead that can be trivially avoided by marking the entry as permanent. In my benchmark, this reduces the time per read by about 20 nanoseconds, from 235 nanoseconds per read to 215. Link: https://github.com/osandov/drgn/issues/106 Signed-off-by: Omar Sandoval --- fs/proc/kcore.c | 1 + 1 file changed, 1 insertion(+) diff --git a/fs/proc/kcore.c b/fs/proc/kcore.c index 51446c59388f..770e4e57f445 100644 --- a/fs/proc/kcore.c +++ b/fs/proc/kcore.c @@ -662,6 +662,7 @@ static int release_kcore(struct inode *inode, struct fi= le *file) } =20 static const struct proc_ops kcore_proc_ops =3D { + .proc_flags =3D PROC_ENTRY_PERMANENT, .proc_read_iter =3D read_kcore_iter, .proc_open =3D open_kcore, .proc_release =3D release_kcore, --=20 2.47.0 From nobody Sat Nov 23 22:28:18 2024 Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A8ED12110E for ; Sat, 9 Nov 2024 01:28:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731115739; cv=none; b=UoGkMADVJDH/8wH9bBXu9pvAFOW9i3E2ecMSz9MChXSmjeYfCz5fHbNnyZPwRyExucCpb7y5OBc3eNpp8iBnQ5X+pCAyFzD54I5iD5HgP42HfSTgKGmdv4iotv970AAqkInDXt4CpHDbPKjM9W54zwNd87fmQElm7jEvbBmbuX4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731115739; c=relaxed/simple; bh=XrcsrSAOKXBBBQwVA9Tuimkn9w8T10KOuIdPu7i4L1Q=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ISxepNemx3cnVHOxMEwzm5lY2WgaIZnj0FrhVxjiPI66UzJCcxsI5T99608dolLCU7oAXlHilhbfTpnbcFjGly1dfjiv50Ma4pNJB/7IKpdpDcjZGZLZQVvd/P/pDhzLVHOu7TVmqsxzFE5dQodGX6nU8nmFXoX7MbuX33l0f5U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=osandov.com; spf=none smtp.mailfrom=osandov.com; dkim=pass (2048-bit key) header.d=osandov-com.20230601.gappssmtp.com header.i=@osandov-com.20230601.gappssmtp.com header.b=KT3cFvwj; arc=none smtp.client-ip=209.85.214.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=osandov.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=osandov.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=osandov-com.20230601.gappssmtp.com header.i=@osandov-com.20230601.gappssmtp.com header.b="KT3cFvwj" Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-20c804409b0so7695ad.2 for ; Fri, 08 Nov 2024 17:28:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osandov-com.20230601.gappssmtp.com; s=20230601; t=1731115737; x=1731720537; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bsQOXM5eEzPefp4DDDW0FE1HiiPiLYyB+AE/vJxP/T4=; b=KT3cFvwjStx4TIdQ7YP5oRlY8JjUadpER7nSGZOhd8xcEWdi+KCCgxGgtCUWNXiogP vj2ryIqJKmjJQcyf2Ks6+Ya8DHNpkA0YBn44VDVoGfjRqykpLM/C6eQ01387nAA4xcNu wKGTWx2+BW7KrxKv7l0ZSwKedHfqWzAPafhmtItm9PaR+itNgXaBYjN9vSPCDyqHFhWU SIXCLcBYCTiD8A3gtHp7iL9zijbxGZVD9cnIhAxFF8ueU+Xv4/B1XNi+UR3ndFF7qL2D uYcDkyg5aHaKBgMpu0KE3YLjHwht5TUo4toLbL8pyvGaa0k4FVf/ss0xFrlvDpPKibly hJ5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731115737; x=1731720537; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bsQOXM5eEzPefp4DDDW0FE1HiiPiLYyB+AE/vJxP/T4=; b=gsTHqptSJbzBRtBUVpexRTlhLmZ4vn8bOyfnfTlCQwjgEcMLosaLVrRfQ2WE5rdj2A /zfP0FYdHEgWz13rTC9/IlG0qWFEyGC7E18Q/AvPjmpc07E3K+U2UcMVVZg1ugfxXEGM TWrto4tYEmMkam7a8kH3FqrqbJMGWwTyeqJlQ5EZ9+/R8bxzHiOInOKh05A1FiiFJGfa vN/L2qg6TdaH+C9BxN4P734jX298sdoXXCDAVkhxj4DFK1+LdSpE9g2p8mD5uCScihKM Zt1snD/P4QtGzGZekn795rZYOmm3q/azvEEWur4YBGQVdhBUMqJqMn65/L7vufEOLZVQ Y+Yg== X-Forwarded-Encrypted: i=1; AJvYcCXYWUm/oaIk9sSRq3oHpkxpHlJoi2+Cf++bqbxhuaAdHec/sBtYDiuU4ISrdsCGqywv/rdnxqMEMB8MP9A=@vger.kernel.org X-Gm-Message-State: AOJu0YzRDWgYQz4tlfGRgXzUyT3rT0v9jiK4u4tV7ZeT+NSlKYQTJv2U 03cLwHYvjxZWq7VLLxnSHQ4rnfyHjuD25Yu4LjVrknIp8lXPurpo1ixdbD+HOX8= X-Google-Smtp-Source: AGHT+IEMcxzKuKijFu4ssmt1kgbjUcwDcCVAU1kIvePW6gaP9G0DE3o/yXMPTKHGS67Lr03Hm8FB/A== X-Received: by 2002:a17:902:ecd0:b0:20c:ee32:7595 with SMTP id d9443c01a7336-211834de6d9mr27903515ad.2.1731115736792; Fri, 08 Nov 2024 17:28:56 -0800 (PST) Received: from telecaster.hsd1.wa.comcast.net ([2601:602:8980:9170::5633]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21177e6c96fsm37493355ad.255.2024.11.08.17.28.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Nov 2024 17:28:55 -0800 (PST) From: Omar Sandoval To: linux-fsdevel@vger.kernel.org, Al Viro Cc: kernel-team@fb.com, linux-kernel@vger.kernel.org Subject: [PATCH 2/4] proc/kcore: don't walk list on every read Date: Fri, 8 Nov 2024 17:28:40 -0800 Message-ID: <8d945558b9c9efe74103a34b7780f1cd90d9ce7f.1731115587.git.osandov@fb.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Omar Sandoval We maintain a list of memory ranges for /proc/kcore, which usually has 10-20 entries. Currently, every single read from /proc/kcore walks the entire list in order to count the number of entries and compute some offsets. These values only change when the list of memory ranges changes, which is very rare (only when memory is hot(un)plugged). We can cache the values when the list is populated to avoid these redundant walks. In my benchmark, this reduces the time per read by another 20 nanoseconds on top of the previous change, from 215 nanoseconds per read to 195. Link: https://github.com/osandov/drgn/issues/106 Signed-off-by: Omar Sandoval --- fs/proc/kcore.c | 70 ++++++++++++++++++++++++------------------------- 1 file changed, 35 insertions(+), 35 deletions(-) diff --git a/fs/proc/kcore.c b/fs/proc/kcore.c index 770e4e57f445..082718f5c02f 100644 --- a/fs/proc/kcore.c +++ b/fs/proc/kcore.c @@ -65,6 +65,10 @@ static inline void kc_unxlate_dev_mem_ptr(phys_addr_t ph= ys, void *virt) #endif =20 static LIST_HEAD(kclist_head); +static int kcore_nphdr; +static size_t kcore_phdrs_len; +static size_t kcore_notes_len; +static size_t kcore_data_offset; static DECLARE_RWSEM(kclist_lock); static int kcore_need_update =3D 1; =20 @@ -101,33 +105,32 @@ void __init kclist_add(struct kcore_list *new, void *= addr, size_t size, list_add_tail(&new->list, &kclist_head); } =20 -static size_t get_kcore_size(int *nphdr, size_t *phdrs_len, size_t *notes_= len, - size_t *data_offset) +static void update_kcore_size(void) { size_t try, size; struct kcore_list *m; =20 - *nphdr =3D 1; /* PT_NOTE */ + kcore_nphdr =3D 1; /* PT_NOTE */ size =3D 0; =20 list_for_each_entry(m, &kclist_head, list) { try =3D kc_vaddr_to_offset((size_t)m->addr + m->size); if (try > size) size =3D try; - *nphdr =3D *nphdr + 1; + kcore_nphdr++; } =20 - *phdrs_len =3D *nphdr * sizeof(struct elf_phdr); - *notes_len =3D (4 * sizeof(struct elf_note) + - 3 * ALIGN(sizeof(CORE_STR), 4) + - VMCOREINFO_NOTE_NAME_BYTES + - ALIGN(sizeof(struct elf_prstatus), 4) + - ALIGN(sizeof(struct elf_prpsinfo), 4) + - ALIGN(arch_task_struct_size, 4) + - ALIGN(vmcoreinfo_size, 4)); - *data_offset =3D PAGE_ALIGN(sizeof(struct elfhdr) + *phdrs_len + - *notes_len); - return *data_offset + size; + kcore_phdrs_len =3D kcore_nphdr * sizeof(struct elf_phdr); + kcore_notes_len =3D (4 * sizeof(struct elf_note) + + 3 * ALIGN(sizeof(CORE_STR), 4) + + VMCOREINFO_NOTE_NAME_BYTES + + ALIGN(sizeof(struct elf_prstatus), 4) + + ALIGN(sizeof(struct elf_prpsinfo), 4) + + ALIGN(arch_task_struct_size, 4) + + ALIGN(vmcoreinfo_size, 4)); + kcore_data_offset =3D PAGE_ALIGN(sizeof(struct elfhdr) + kcore_phdrs_len + + kcore_notes_len); + proc_root_kcore->size =3D kcore_data_offset + size; } =20 #ifdef CONFIG_HIGHMEM @@ -270,8 +273,6 @@ static int kcore_update_ram(void) { LIST_HEAD(list); LIST_HEAD(garbage); - int nphdr; - size_t phdrs_len, notes_len, data_offset; struct kcore_list *tmp, *pos; int ret =3D 0; =20 @@ -293,8 +294,7 @@ static int kcore_update_ram(void) } list_splice_tail(&list, &kclist_head); =20 - proc_root_kcore->size =3D get_kcore_size(&nphdr, &phdrs_len, ¬es_len, - &data_offset); + update_kcore_size(); =20 out: up_write(&kclist_lock); @@ -326,12 +326,10 @@ static ssize_t read_kcore_iter(struct kiocb *iocb, st= ruct iov_iter *iter) struct file *file =3D iocb->ki_filp; char *buf =3D file->private_data; loff_t *fpos =3D &iocb->ki_pos; - size_t phdrs_offset, notes_offset, data_offset; + size_t phdrs_offset, notes_offset; size_t page_offline_frozen =3D 1; - size_t phdrs_len, notes_len; struct kcore_list *m; size_t tsz; - int nphdr; unsigned long start; size_t buflen =3D iov_iter_count(iter); size_t orig_buflen =3D buflen; @@ -344,9 +342,8 @@ static ssize_t read_kcore_iter(struct kiocb *iocb, stru= ct iov_iter *iter) */ page_offline_freeze(); =20 - get_kcore_size(&nphdr, &phdrs_len, ¬es_len, &data_offset); phdrs_offset =3D sizeof(struct elfhdr); - notes_offset =3D phdrs_offset + phdrs_len; + notes_offset =3D phdrs_offset + kcore_phdrs_len; =20 /* ELF file header. */ if (buflen && *fpos < sizeof(struct elfhdr)) { @@ -368,7 +365,7 @@ static ssize_t read_kcore_iter(struct kiocb *iocb, stru= ct iov_iter *iter) .e_flags =3D ELF_CORE_EFLAGS, .e_ehsize =3D sizeof(struct elfhdr), .e_phentsize =3D sizeof(struct elf_phdr), - .e_phnum =3D nphdr, + .e_phnum =3D kcore_nphdr, }; =20 tsz =3D min_t(size_t, buflen, sizeof(struct elfhdr) - *fpos); @@ -382,10 +379,10 @@ static ssize_t read_kcore_iter(struct kiocb *iocb, st= ruct iov_iter *iter) } =20 /* ELF program headers. */ - if (buflen && *fpos < phdrs_offset + phdrs_len) { + if (buflen && *fpos < phdrs_offset + kcore_phdrs_len) { struct elf_phdr *phdrs, *phdr; =20 - phdrs =3D kzalloc(phdrs_len, GFP_KERNEL); + phdrs =3D kzalloc(kcore_phdrs_len, GFP_KERNEL); if (!phdrs) { ret =3D -ENOMEM; goto out; @@ -393,13 +390,14 @@ static ssize_t read_kcore_iter(struct kiocb *iocb, st= ruct iov_iter *iter) =20 phdrs[0].p_type =3D PT_NOTE; phdrs[0].p_offset =3D notes_offset; - phdrs[0].p_filesz =3D notes_len; + phdrs[0].p_filesz =3D kcore_notes_len; =20 phdr =3D &phdrs[1]; list_for_each_entry(m, &kclist_head, list) { phdr->p_type =3D PT_LOAD; phdr->p_flags =3D PF_R | PF_W | PF_X; - phdr->p_offset =3D kc_vaddr_to_offset(m->addr) + data_offset; + phdr->p_offset =3D kc_vaddr_to_offset(m->addr) + + kcore_data_offset; phdr->p_vaddr =3D (size_t)m->addr; if (m->type =3D=3D KCORE_RAM) phdr->p_paddr =3D __pa(m->addr); @@ -412,7 +410,8 @@ static ssize_t read_kcore_iter(struct kiocb *iocb, stru= ct iov_iter *iter) phdr++; } =20 - tsz =3D min_t(size_t, buflen, phdrs_offset + phdrs_len - *fpos); + tsz =3D min_t(size_t, buflen, + phdrs_offset + kcore_phdrs_len - *fpos); if (copy_to_iter((char *)phdrs + *fpos - phdrs_offset, tsz, iter) !=3D tsz) { kfree(phdrs); @@ -426,7 +425,7 @@ static ssize_t read_kcore_iter(struct kiocb *iocb, stru= ct iov_iter *iter) } =20 /* ELF note segment. */ - if (buflen && *fpos < notes_offset + notes_len) { + if (buflen && *fpos < notes_offset + kcore_notes_len) { struct elf_prstatus prstatus =3D {}; struct elf_prpsinfo prpsinfo =3D { .pr_sname =3D 'R', @@ -438,7 +437,7 @@ static ssize_t read_kcore_iter(struct kiocb *iocb, stru= ct iov_iter *iter) strscpy(prpsinfo.pr_psargs, saved_command_line, sizeof(prpsinfo.pr_psargs)); =20 - notes =3D kzalloc(notes_len, GFP_KERNEL); + notes =3D kzalloc(kcore_notes_len, GFP_KERNEL); if (!notes) { ret =3D -ENOMEM; goto out; @@ -459,9 +458,10 @@ static ssize_t read_kcore_iter(struct kiocb *iocb, str= uct iov_iter *iter) */ append_kcore_note(notes, &i, VMCOREINFO_NOTE_NAME, 0, vmcoreinfo_data, - min(vmcoreinfo_size, notes_len - i)); + min(vmcoreinfo_size, kcore_notes_len - i)); =20 - tsz =3D min_t(size_t, buflen, notes_offset + notes_len - *fpos); + tsz =3D min_t(size_t, buflen, + notes_offset + kcore_notes_len - *fpos); if (copy_to_iter(notes + *fpos - notes_offset, tsz, iter) !=3D tsz) { kfree(notes); ret =3D -EFAULT; @@ -477,7 +477,7 @@ static ssize_t read_kcore_iter(struct kiocb *iocb, stru= ct iov_iter *iter) * Check to see if our file offset matches with any of * the addresses in the elf_phdr on our list. */ - start =3D kc_offset_to_vaddr(*fpos - data_offset); + start =3D kc_offset_to_vaddr(*fpos - kcore_data_offset); if ((tsz =3D (PAGE_SIZE - (start & ~PAGE_MASK))) > buflen) tsz =3D buflen; =20 --=20 2.47.0 From nobody Sat Nov 23 22:28:18 2024 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9F5AB40BF2 for ; Sat, 9 Nov 2024 01:28:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731115740; cv=none; b=e69JFuMUD3ahS1EsrMiPzr98kfWiRrBlDLCbXQ84P0LVuK9jRH3xCKvGPUtMyqJeHvKU9LW+FKUGrFa7X8QzgOBJpcjmG/FQvfw4bB2nvK4v3tT2ORCCa8f0eFWwSHmFPzuyPLmA1VzzUIwB797rEq9qW6M76v1z/NeMLtpsyZU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731115740; c=relaxed/simple; bh=2V+LMyhP7QCx7vipIrK/PNDo/NUYimbVbH9Zga/QRnw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lGNnLONdqzDyQKZszGSJjfHz7Z2PEUtI5M1S0bsCvfSqpQNVHpfSlE5wh+mqTP/LA8IyzMyRu02MRhB6SxLFfX5Zpvxr8QMV9pB/hW9WtJ27LfQamQkyzzKXlUgFhmYNCBAkqOywqsQPPAsmt8oTThAikHmZlYwV63cqWrMb00A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=osandov.com; spf=none smtp.mailfrom=osandov.com; dkim=pass (2048-bit key) header.d=osandov-com.20230601.gappssmtp.com header.i=@osandov-com.20230601.gappssmtp.com header.b=SJM1fV2i; arc=none smtp.client-ip=209.85.214.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=osandov.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=osandov.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=osandov-com.20230601.gappssmtp.com header.i=@osandov-com.20230601.gappssmtp.com header.b="SJM1fV2i" Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-20cb8954ec9so3701035ad.3 for ; Fri, 08 Nov 2024 17:28:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osandov-com.20230601.gappssmtp.com; s=20230601; t=1731115738; x=1731720538; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xOqgPPtziEx8SGLaOYb14oaOw/i0JcmNn0bH4qe21tg=; b=SJM1fV2i8umSzCb58koP7j7K+NELgsaCQQQH/MkptEuEx7pohw8Zvrn2gBl5q+9kTN 740gtHf9rrk60l0atKcx41Hw9dEeaOoTz3lTLcdJEA8JCmrh9he0hQ5P62EvXd3IM0Lm U2oKGJFSbo6qHB6N3d1UZzZK+VgnjvOtpRTuqAdvwSoZvEyGaXzytFDCeh682XHxCNsq qgBWC3nPkRI6z6DkDwZyNDrUxyalAuw3qwC0xDRHlVt8F+LhnxPJ6g4G/gcolcMJeGTH /WB74lgTewunqEfl7neoRJTCejvuudZRffnm/JTSoLDOxdD8L6nLlKLaFD3INJOW0BaW rEmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731115738; x=1731720538; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xOqgPPtziEx8SGLaOYb14oaOw/i0JcmNn0bH4qe21tg=; b=SlkVr0s46si9JEolS64iMNol0ouKHNbp08Kz3BPW1W36rsIUpMp5s7bNJ4ti+18jel puXJZgueP/Zkj4IUe9/+CyZEUwkOJ74N12UZNg3o6Bi6I0cc0Miy4trRf2BUD0T3Lf3J jDrZ0h4LD7dkHwgSMYtQecWnJ3MLZLBYMmgG2OV5rnNIvwR65fs/yH4pK8kXE0eeGpPA 7kR225WGveXWVR0rCZHVD1Kvwjc375pghB7SXC+4LPLuUK/vqXoKx08UTpHhsMrPnYyH tPnNJh+7B1xnRIaOSnsEgcPVWgwfcSv4EyXe8TeRQNNRXQ62yjVCKj9NAF5BmDwdHmY9 O5Jg== X-Forwarded-Encrypted: i=1; AJvYcCVL9ltOeoowrfVe65ZAK7KMNaSsvu5PvL9L+z0llFfxeV1OfOYOx7eEI9O1+C+VlivaR79wFxRkFmuhKWo=@vger.kernel.org X-Gm-Message-State: AOJu0YxswNF39WhSGhkc/RJTmrFox+3d+O6CcP53zq2vXuhbkh7sJkYx RPnQA6OPa4wDKqQpyNauXi6x2Xc/GRky8ESZR9H14R2X9Y7XiubcPGeGNXIjooc= X-Google-Smtp-Source: AGHT+IFOd7MaBqO9qvnNSmqC8iCKDohmW0xsMff0WgR9ONvpPAG2Els+t+TmPhLbjIw8o7WypoTk7g== X-Received: by 2002:a17:902:f68a:b0:20c:e169:eb6a with SMTP id d9443c01a7336-2118357e1b5mr27424505ad.10.1731115737917; Fri, 08 Nov 2024 17:28:57 -0800 (PST) Received: from telecaster.hsd1.wa.comcast.net ([2601:602:8980:9170::5633]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21177e6c96fsm37493355ad.255.2024.11.08.17.28.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Nov 2024 17:28:57 -0800 (PST) From: Omar Sandoval To: linux-fsdevel@vger.kernel.org, Al Viro Cc: kernel-team@fb.com, linux-kernel@vger.kernel.org Subject: [PATCH 3/4] proc/kcore: use percpu_rw_semaphore for kclist_lock Date: Fri, 8 Nov 2024 17:28:41 -0800 Message-ID: <83a3b235b4bcc3b8aef7c533e0657f4d7d5d35ae.1731115587.git.osandov@fb.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Omar Sandoval The list of memory ranges for /proc/kcore is protected by a rw_semaphore. We lock it for reading on every read from /proc/kcore. This is very heavy, especially since it is rarely locked for writing. Since we want to strongly favor read lock performance, convert it to a percpu_rw_semaphore. I also experimented with percpu_ref and SRCU, but this change was the simplest and the fastest. In my benchmark, this reduces the time per read by yet another 20 nanoseconds on top of the previous two changes, from 195 nanoseconds per read to 175. Link: https://github.com/osandov/drgn/issues/106 Signed-off-by: Omar Sandoval --- fs/proc/kcore.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/fs/proc/kcore.c b/fs/proc/kcore.c index 082718f5c02f..f0d56d000816 100644 --- a/fs/proc/kcore.c +++ b/fs/proc/kcore.c @@ -69,7 +69,7 @@ static int kcore_nphdr; static size_t kcore_phdrs_len; static size_t kcore_notes_len; static size_t kcore_data_offset; -static DECLARE_RWSEM(kclist_lock); +DEFINE_STATIC_PERCPU_RWSEM(kclist_lock); static int kcore_need_update =3D 1; =20 /* @@ -276,7 +276,7 @@ static int kcore_update_ram(void) struct kcore_list *tmp, *pos; int ret =3D 0; =20 - down_write(&kclist_lock); + percpu_down_write(&kclist_lock); if (!xchg(&kcore_need_update, 0)) goto out; =20 @@ -297,7 +297,7 @@ static int kcore_update_ram(void) update_kcore_size(); =20 out: - up_write(&kclist_lock); + percpu_up_write(&kclist_lock); list_for_each_entry_safe(pos, tmp, &garbage, list) { list_del(&pos->list); kfree(pos); @@ -335,7 +335,7 @@ static ssize_t read_kcore_iter(struct kiocb *iocb, stru= ct iov_iter *iter) size_t orig_buflen =3D buflen; int ret =3D 0; =20 - down_read(&kclist_lock); + percpu_down_read(&kclist_lock); /* * Don't race against drivers that set PageOffline() and expect no * further page access. @@ -625,7 +625,7 @@ static ssize_t read_kcore_iter(struct kiocb *iocb, stru= ct iov_iter *iter) =20 out: page_offline_thaw(); - up_read(&kclist_lock); + percpu_up_read(&kclist_lock); if (ret) return ret; return orig_buflen - buflen; --=20 2.47.0 From nobody Sat Nov 23 22:28:18 2024 Received: from mail-pj1-f48.google.com (mail-pj1-f48.google.com [209.85.216.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B8EC154FAD for ; Sat, 9 Nov 2024 01:28:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731115741; cv=none; b=WAPX5Ltdx98y9iApbn1KZhllKiUNF/d31ISafPJeGFFczeMshal4N7H3QA9Pps2+O4RUYs1G+e5IqoZ8tk6H72b4olq0ch8seXg+1iDEKbem44nON+SzN5s1lIFzorwapxw3lR4kAG7jrjGkw/mhd/2Pd/i+6FPBllgfq9/WZKc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731115741; c=relaxed/simple; bh=uZOIUaRs+7H+jn62BpsP+uX28ASu2b/m0Dd0veYqTPs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BAxSZZJrRwdA4UMaoH7fz3uCHL5mNbi10DcilgsFdtw3xVNinOFOsq7LNV3nDa232bTKh5ZkuN5Ay2hl0Cf7M2n11vB7EBQq+cSPuvHdNWibzJFT8I6YNkIVQsUllV4W5UJVHOjlWko78XZ9L0U743y3qhitXc5ngv+RrHpVKPU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=osandov.com; spf=none smtp.mailfrom=osandov.com; dkim=pass (2048-bit key) header.d=osandov-com.20230601.gappssmtp.com header.i=@osandov-com.20230601.gappssmtp.com header.b=uWa5rgwV; arc=none smtp.client-ip=209.85.216.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=osandov.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=osandov.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=osandov-com.20230601.gappssmtp.com header.i=@osandov-com.20230601.gappssmtp.com header.b="uWa5rgwV" Received: by mail-pj1-f48.google.com with SMTP id 98e67ed59e1d1-2e2ab5bbc01so10395a91.2 for ; Fri, 08 Nov 2024 17:28:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osandov-com.20230601.gappssmtp.com; s=20230601; t=1731115739; x=1731720539; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=e4yNjGAg8qf3fKwWurXSx/s8j5dgkc97k/3nCeXjiZI=; b=uWa5rgwV4reeQFAdaDLk35MfwdFUPJ+wVO6Da+WNWAYyb5XPToSp87v5GfTcvvf+Iw YFNI/j6fwJbqa/71fHRWmClD5tPNynamdv5tsaHarZ0XVm/3hAReAFLb+pmxJgwDH4Nr dhHT6U/ex2MYmEiI7rpFj5u51hD3Eh0HAmk0iTB9bToJpH/KHfo80EW/YX/GEa90LTmE LbFZr9j6kSl4aF0y0ztLQ29KXgVP0wI1FabeW4z3P/5H+ILw1bqFD7Fuqdi4fma8WfcV +TeTaxTHhC3PmAWqA95HbMzaX/LPjfsUFpXzvt1KEq0z/F4JSjKUydICAKw/PssWD9qN eqrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731115739; x=1731720539; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=e4yNjGAg8qf3fKwWurXSx/s8j5dgkc97k/3nCeXjiZI=; b=w/gmpQa347sWcPUNgreIbWNhqbEMM3YaMuc8XjmZ6IOPr40S9eDiCDmKhBiWKR9WYp ign21l7XDGWTRA4brmNQgUDz3UVwSVnv1HKLeOuTMDZUmhTV76RqC/aArprZrgNCkL6N kaUjZhhZ24CRwj0ZAhuDElgnFhROYa7s/ULblIls26mSxZm5sTsqv4WYBKglNj8GxYMP ONnsRd+e/cPDdMGDlExXP8IaAOiTuzmfOIv+M2YzKsvN7qNzljNxhuWbfAHurTzsHhCv V2K62IND/zZCQvZQT6FEw4W097Pq2TCipNNZefYRqlGWkuGI5keCG5uSYdFsLCUiOEMI gCXQ== X-Forwarded-Encrypted: i=1; AJvYcCUlGGb97STYLQQwYIr2DvP1F5pUmRt22FtVdjSjD8FpwSmEgm7MM6+YtgzbUcWgk+J3gDyEVv1ghw9GnBU=@vger.kernel.org X-Gm-Message-State: AOJu0Yx/c2/FMgwa1zOqeHL1v0sS1mAMTuqg/PUrF8C7aeIcyjSSWa0c Vo9i1+lgM7n0UOTqPlG2/jHhKX1VcOa++oz2dJW5YegxNG+Wh3KT+qd6jKALTam/EYrMwBXjWoh S X-Google-Smtp-Source: AGHT+IGxDPbGoufHSHqpkSKM+XO+452K3765m0FRuWx/5mfOxgW3YLlOJkt1wHWgGfUa1caBsRR/ng== X-Received: by 2002:a17:902:dac9:b0:20b:80e6:bce6 with SMTP id d9443c01a7336-211834f8c81mr28605025ad.4.1731115738993; Fri, 08 Nov 2024 17:28:58 -0800 (PST) Received: from telecaster.hsd1.wa.comcast.net ([2601:602:8980:9170::5633]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21177e6c96fsm37493355ad.255.2024.11.08.17.28.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Nov 2024 17:28:58 -0800 (PST) From: Omar Sandoval To: linux-fsdevel@vger.kernel.org, Al Viro Cc: kernel-team@fb.com, linux-kernel@vger.kernel.org Subject: [PATCH 4/4] MAINTAINERS: add me as /proc/kcore maintainer Date: Fri, 8 Nov 2024 17:28:42 -0800 Message-ID: X-Mailer: git-send-email 2.47.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Omar Sandoval Christian volunteered me for this a while back given that drgn is the main user of /proc/kcore and I've touched it several times over the years. Link: https://lore.kernel.org/all/20231125-kurhotel-zuwege-10cce62a50fd@bra= uner/ Signed-off-by: Omar Sandoval --- MAINTAINERS | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index bdae0faf000c..89645de6faba 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -12242,6 +12242,13 @@ F: Documentation/kbuild/kconfig* F: scripts/Kconfig.include F: scripts/kconfig/ =20 +KCORE +M: Omar Sandoval +L: linux-debuggers@vger.kernel.org +S: Maintained +F: fs/proc/kcore.c +F: include/linux/kcore.h + KCOV R: Dmitry Vyukov R: Andrey Konovalov --=20 2.47.0