From nobody Fri Nov 14 06:45:33 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=crudebyte.com ARC-Seal: i=1; a=rsa-sha256; t=1585262040; cv=none; d=zohomail.com; s=zohoarc; b=Yip3/+elNcY6KZvFPbgOdiOsYxcCIkX+YdWyeJUjIGkeUW/Yf9XlWtdT9PcZiZstBY7C7kT4Nxt9JSwFY2Lm6hGepONnSVZ8Py7/Lef7anU6zJAGUJo4zYb1oYjUvFEz2oyzGxG0TzlidR5Fv9eiQsUZLVt4dT9l9eRaFWOtfqc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1585262040; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=0NiuoPE9FpFqaIkbmn2TcQVwJNLByVhPkawPEjeTIMs=; b=U9wSpG27QYWXFPUAv+zJO2OjDDIkXe2qCVlvQItyzvXHscgpMT6hrkerL1r36T6JcmbX2hSAk0CU6EwmaYzN9fWEOnHg7MrJv3wx/idHD56bk+SyKXIy2GPBZFhP2NkG7KkmrzeC/VcYfpaXY95LxUCg+i1asPfBe4bmSTo9DAo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1585262040945661.2172958384939; Thu, 26 Mar 2020 15:34:00 -0700 (PDT) Received: from localhost ([::1]:33626 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jHb4Z-0000y6-I1 for importer@patchew.org; Thu, 26 Mar 2020 18:33:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58223) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from <8ee6bb18b00ccb92ca9ecdeb4ff320503ed6a730@lizzy.crudebyte.com>) id 1jHb37-0008Ul-Ud for qemu-devel@nongnu.org; Thu, 26 Mar 2020 18:32:31 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from <8ee6bb18b00ccb92ca9ecdeb4ff320503ed6a730@lizzy.crudebyte.com>) id 1jHb35-0000aC-7U for qemu-devel@nongnu.org; Thu, 26 Mar 2020 18:32:28 -0400 Received: from lizzy.crudebyte.com ([91.194.90.13]:34939) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from <8ee6bb18b00ccb92ca9ecdeb4ff320503ed6a730@lizzy.crudebyte.com>) id 1jHb34-0008EU-W5 for qemu-devel@nongnu.org; Thu, 26 Mar 2020 18:32:27 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=crudebyte.com; s=lizzy; h=Cc:To:Subject:Date:From:References:In-Reply-To: Message-Id:Content-Type:Content-Transfer-Encoding:MIME-Version:Content-ID: Content-Description; bh=0NiuoPE9FpFqaIkbmn2TcQVwJNLByVhPkawPEjeTIMs=; b=JyGl3 g4H+waXVS0MRnKRpTOCXQ0CzRsaogdVw8QN8uk4Oaz8FQJ7K+gh2tfu5BbB/mR6fmQ2ieHGxuuWRR FtckUAq8RzTgvRaLJWglEvz+vLqT7tIUuGnPkr94AJNm0XOyKrA6VjU3o2HjncoDCml7lcNv7pWXC R9lZRjVG6Za3HHIYdwdczjFk5DTQKZ1nsI2LHZLMd2MAQJNXlhvcf2HEmDF5+DY62QKJoEt2qUkKU 6yqRjKtU++Fe+s9v/X25o91qLevKnz0YSxI3r15+VULbG+7j83hZd5pNmUBqvwGJcIFC+LaeHCAOh 7U3pZM8tZfG+guRuhwk/Azq4m/0ig==; Message-Id: <8ee6bb18b00ccb92ca9ecdeb4ff320503ed6a730.1585258105.git.qemu_oss@crudebyte.com> In-Reply-To: References: From: Christian Schoenebeck Date: Thu, 26 Mar 2020 22:25:18 +0100 Subject: [PATCH v5 5/6] 9pfs: T_readdir latency optimization To: qemu-devel@nongnu.org Cc: Greg Kurz X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 91.194.90.13 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @crudebyte.com) Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Make top half really top half and bottom half really bottom half: Each T_readdir request handling is hopping between threads (main I/O thread and background I/O driver threads) several times for every individual directory entry, which sums up to huge latencies for handling just a single T_readdir request. Instead of doing that, collect now all required directory entries (including all potentially required stat buffers for each entry) in one rush on a background I/O thread from fs driver by calling the previously added function v9fs_co_readdir_many() instead of v9fs_co_readdir(), then assemble the entire resulting network response message for the readdir request on main I/O thread. The fs driver is still aborting the directory entry retrieval loop (on the background I/O thread inside of v9fs_co_readdir_many()) as soon as it would exceed the client's requested maximum R_readdir response size. So this will not introduce a performance penalty on another end. Signed-off-by: Christian Schoenebeck --- hw/9pfs/9p.c | 122 +++++++++++++++++++++++---------------------------- 1 file changed, 55 insertions(+), 67 deletions(-) diff --git a/hw/9pfs/9p.c b/hw/9pfs/9p.c index bd8a7cbfac..5246d96a08 100644 --- a/hw/9pfs/9p.c +++ b/hw/9pfs/9p.c @@ -971,30 +971,6 @@ static int coroutine_fn fid_to_qid(V9fsPDU *pdu, V9fsF= idState *fidp, return 0; } =20 -static int coroutine_fn dirent_to_qid(V9fsPDU *pdu, V9fsFidState *fidp, - struct dirent *dent, V9fsQID *qidp) -{ - struct stat stbuf; - V9fsPath path; - int err; - - v9fs_path_init(&path); - - err =3D v9fs_co_name_to_path(pdu, &fidp->path, dent->d_name, &path); - if (err < 0) { - goto out; - } - err =3D v9fs_co_lstat(pdu, &path, &stbuf); - if (err < 0) { - goto out; - } - err =3D stat_to_qid(pdu, &stbuf, qidp); - -out: - v9fs_path_free(&path); - return err; -} - V9fsPDU *pdu_alloc(V9fsState *s) { V9fsPDU *pdu =3D NULL; @@ -2337,6 +2313,18 @@ size_t v9fs_readdir_response_size(V9fsString *name) return 24 + v9fs_string_size(name); } =20 +static void v9fs_free_dirents(struct V9fsDirEnt *e) +{ + struct V9fsDirEnt *next =3D NULL; + + for (; e; e =3D next) { + next =3D e->next; + g_free(e->dent); + g_free(e->st); + g_free(e); + } +} + static int coroutine_fn v9fs_do_readdir(V9fsPDU *pdu, V9fsFidState *fidp, int32_t maxsize) { @@ -2345,54 +2333,53 @@ static int coroutine_fn v9fs_do_readdir(V9fsPDU *pd= u, V9fsFidState *fidp, V9fsString name; int len, err =3D 0; int32_t count =3D 0; - off_t saved_dir_pos; struct dirent *dent; + struct stat *st; + struct V9fsDirEnt *entries =3D NULL; =20 - /* save the directory position */ - saved_dir_pos =3D v9fs_co_telldir(pdu, fidp); - if (saved_dir_pos < 0) { - return saved_dir_pos; - } - - while (1) { - v9fs_readdir_lock(&fidp->fs.dir); + /* + * inode remapping requires the device id, which in turn might be + * different for different directory entries, so if inode remapping is + * enabled we have to make a full stat for each directory entry + */ + const bool dostat =3D pdu->s->ctx.export_flags & V9FS_REMAP_INODES; =20 - err =3D v9fs_co_readdir(pdu, fidp, &dent); - if (err || !dent) { - break; - } - v9fs_string_init(&name); - v9fs_string_sprintf(&name, "%s", dent->d_name); - if ((count + v9fs_readdir_response_size(&name)) > maxsize) { - v9fs_readdir_unlock(&fidp->fs.dir); + /* + * Fetch all required directory entries altogether on a background IO + * thread from fs driver. We don't want to do that for each entry + * individually, because hopping between threads (this main IO thread + * and background IO driver thread) would sum up to huge latencies. + */ + count =3D v9fs_co_readdir_many(pdu, fidp, &entries, maxsize, dostat); + if (count < 0) { + err =3D count; + count =3D 0; + goto out; + } + count =3D 0; =20 - /* Ran out of buffer. Set dir back to old position and return = */ - v9fs_co_seekdir(pdu, fidp, saved_dir_pos); - v9fs_string_free(&name); - return count; - } + for (struct V9fsDirEnt *e =3D entries; e; e =3D e->next) { + dent =3D e->dent; =20 if (pdu->s->ctx.export_flags & V9FS_REMAP_INODES) { - /* - * dirent_to_qid() implies expensive stat call for each entry, - * we must do that here though since inode remapping requires - * the device id, which in turn might be different for - * different entries; we cannot make any assumption to avoid - * that here. - */ - err =3D dirent_to_qid(pdu, fidp, dent, &qid); + st =3D e->st; + /* e->st should never be NULL, but just to be sure */ + if (!st) { + err =3D -1; + break; + } + + /* remap inode */ + err =3D stat_to_qid(pdu, st, &qid); if (err < 0) { - v9fs_readdir_unlock(&fidp->fs.dir); - v9fs_co_seekdir(pdu, fidp, saved_dir_pos); - v9fs_string_free(&name); - return err; + break; } } else { /* * Fill up just the path field of qid because the client uses * only that. To fill the entire qid structure we will have * to stat each dirent found, which is expensive. For the - * latter reason we don't call dirent_to_qid() here. Only draw= back + * latter reason we don't call stat_to_qid() here. Only drawba= ck * is that no multi-device export detection of stat_to_qid() * would be done and provided as error to the user here. But * user would get that error anyway when accessing those @@ -2405,25 +2392,26 @@ static int coroutine_fn v9fs_do_readdir(V9fsPDU *pd= u, V9fsFidState *fidp, qid.version =3D 0; } =20 + v9fs_string_init(&name); + v9fs_string_sprintf(&name, "%s", dent->d_name); + /* 11 =3D 7 + 4 (7 =3D start offset, 4 =3D space for storing count= ) */ len =3D pdu_marshal(pdu, 11 + count, "Qqbs", &qid, dent->d_off, dent->d_type, &name); =20 - v9fs_readdir_unlock(&fidp->fs.dir); + v9fs_string_free(&name); =20 if (len < 0) { - v9fs_co_seekdir(pdu, fidp, saved_dir_pos); - v9fs_string_free(&name); - return len; + err =3D len; + break; } + count +=3D len; - v9fs_string_free(&name); - saved_dir_pos =3D dent->d_off; } =20 - v9fs_readdir_unlock(&fidp->fs.dir); - +out: + v9fs_free_dirents(entries); if (err < 0) { return err; } --=20 2.20.1