From nobody Mon Feb 9 03:14:32 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=crudebyte.com ARC-Seal: i=1; a=rsa-sha256; t=1597236661; cv=none; d=zohomail.com; s=zohoarc; b=eWR6SNuGRy1jRpPM2wUr/tB8FECNAspybIUG1IuzybdUq0IIls7cM79iA9mzv9r6bWXXfFeZqg2Rvh+e0mbVKbdScAeFijFakmyV/tGVGTkV764J65J7vfKwLPpS4PHjawKwnOAs+pLVyXRj0EsjlkP538VwxrajVqWtYSIPAQQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1597236661; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=GNy1vJScIjnAUJYX0AIEa090ls86NMVH10hGMsFeWbM=; b=Ebp7c0A3/ZjZs1cbCvPPL3AyEfyAD12L15j3GrsIvR3KfFsxRxZa6ymoRvQuDDMtQSts4dBinGGmg6FYJXJiwn80bt6lcfrlfCtK5sCQsyRM7Iugemm1ZCIpRJVX+NHSZMmyj/1aeIJ7+HD0HqdOC2DlOxHMgtV5VdbwL8DiglU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1597236661934997.6392816268641; Wed, 12 Aug 2020 05:51:01 -0700 (PDT) Received: from localhost ([::1]:45686 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k5qDb-0006fj-6b for importer@patchew.org; Wed, 12 Aug 2020 08:50:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35782) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <0c4356ba7dafc8ecb5877a42fc0d68d45ccf5951@lizzy.crudebyte.com>) id 1k5q9n-0001FH-QL for qemu-devel@nongnu.org; Wed, 12 Aug 2020 08:47:03 -0400 Received: from lizzy.crudebyte.com ([91.194.90.13]:39267) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <0c4356ba7dafc8ecb5877a42fc0d68d45ccf5951@lizzy.crudebyte.com>) id 1k5q9m-0005yi-4e for qemu-devel@nongnu.org; Wed, 12 Aug 2020 08:47:03 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=crudebyte.com; s=lizzy; h=Cc:To:Subject:Date:From:References:In-Reply-To: Message-Id:Content-Type:Content-Transfer-Encoding:MIME-Version:Content-ID: Content-Description; bh=GNy1vJScIjnAUJYX0AIEa090ls86NMVH10hGMsFeWbM=; b=eEGFs NTNVXUXSYhHiV5v5ReeEriH5dNc2X2+4RcjC/HE8ymqJ/xO2e6gAGLO6Tn71TrtLca/I8oRuNUSl7 Fw2BFpI6CCKo+vDwXmmiPR+HZRI0Zi8iAAcmMuCoCNYxlAZ42f0hCZdUZVG1/yKesFlmpJxtg+dtp QP7ImQNkEZ3/m38mXPPK9Y1VI2RPg0eWfYeRXQUMlaJlTovtMeKpdGu6fwhj6N5Uf1mhUk3wu87f+ bh9Em6+fCvrs2fo8ovuRQ1R20XedryKLTtsWUHcDHeFIK48y/NGWtyS1tFN1yXJAhmNNLxeqbiZXb 0hcM71ggTTzwYZWEoGZ9gc/gsY2IA==; Message-Id: <0c4356ba7dafc8ecb5877a42fc0d68d45ccf5951.1597226797.git.qemu_oss@crudebyte.com> In-Reply-To: References: From: Christian Schoenebeck Date: Wed, 29 Jul 2020 10:13:05 +0200 Subject: [PULL 5/7] 9pfs: T_readdir latency optimization To: qemu-devel@nongnu.org, Peter Maydell Cc: Greg Kurz Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: none client-ip=91.194.90.13; envelope-from=0c4356ba7dafc8ecb5877a42fc0d68d45ccf5951@lizzy.crudebyte.com; helo=lizzy.crudebyte.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/12 08:16:56 X-ACL-Warn: Detected OS = Linux 3.11 and newer X-Spam_score_int: 13 X-Spam_score: 1.3 X-Spam_bar: + X-Spam_report: (1.3 / 5.0 requ) BAYES_00=-1.9, DATE_IN_PAST_96_XX=3.405, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @crudebyte.com) Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Make top half really top half and bottom half really bottom half: Each T_readdir request handling is hopping between threads (main I/O thread and background I/O driver threads) several times for every individual directory entry, which sums up to huge latencies for handling just a single T_readdir request. Instead of doing that, collect now all required directory entries (including all potentially required stat buffers for each entry) in one rush on a background I/O thread from fs driver by calling the previously added function v9fs_co_readdir_many() instead of v9fs_co_readdir(), then assemble the entire resulting network response message for the readdir request on main I/O thread. The fs driver is still aborting the directory entry retrieval loop (on the background I/O thread inside of v9fs_co_readdir_many()) as soon as it would exceed the client's requested maximum R_readdir response size. So this will not introduce a performance penalty on another end. Also: No longer seek initial directory position in v9fs_readdir(), as this is now handled (more consistently) by v9fs_co_readdir_many() instead. Signed-off-by: Christian Schoenebeck Message-Id: Signed-off-by: Christian Schoenebeck --- hw/9pfs/9p.c | 132 ++++++++++++++++++++++----------------------------- 1 file changed, 58 insertions(+), 74 deletions(-) diff --git a/hw/9pfs/9p.c b/hw/9pfs/9p.c index 7a228c4828..cc4094b971 100644 --- a/hw/9pfs/9p.c +++ b/hw/9pfs/9p.c @@ -972,30 +972,6 @@ static int coroutine_fn fid_to_qid(V9fsPDU *pdu, V9fsF= idState *fidp, return 0; } =20 -static int coroutine_fn dirent_to_qid(V9fsPDU *pdu, V9fsFidState *fidp, - struct dirent *dent, V9fsQID *qidp) -{ - struct stat stbuf; - V9fsPath path; - int err; - - v9fs_path_init(&path); - - err =3D v9fs_co_name_to_path(pdu, &fidp->path, dent->d_name, &path); - if (err < 0) { - goto out; - } - err =3D v9fs_co_lstat(pdu, &path, &stbuf); - if (err < 0) { - goto out; - } - err =3D stat_to_qid(pdu, &stbuf, qidp); - -out: - v9fs_path_free(&path); - return err; -} - V9fsPDU *pdu_alloc(V9fsState *s) { V9fsPDU *pdu =3D NULL; @@ -2328,62 +2304,74 @@ size_t v9fs_readdir_response_size(V9fsString *name) return 24 + v9fs_string_size(name); } =20 +static void v9fs_free_dirents(struct V9fsDirEnt *e) +{ + struct V9fsDirEnt *next =3D NULL; + + for (; e; e =3D next) { + next =3D e->next; + g_free(e->dent); + g_free(e->st); + g_free(e); + } +} + static int coroutine_fn v9fs_do_readdir(V9fsPDU *pdu, V9fsFidState *fidp, - int32_t max_count) + off_t offset, int32_t max_count) { size_t size; V9fsQID qid; V9fsString name; int len, err =3D 0; int32_t count =3D 0; - off_t saved_dir_pos; struct dirent *dent; + struct stat *st; + struct V9fsDirEnt *entries =3D NULL; =20 - /* save the directory position */ - saved_dir_pos =3D v9fs_co_telldir(pdu, fidp); - if (saved_dir_pos < 0) { - return saved_dir_pos; - } - - while (1) { - v9fs_readdir_lock(&fidp->fs.dir); + /* + * inode remapping requires the device id, which in turn might be + * different for different directory entries, so if inode remapping is + * enabled we have to make a full stat for each directory entry + */ + const bool dostat =3D pdu->s->ctx.export_flags & V9FS_REMAP_INODES; =20 - err =3D v9fs_co_readdir(pdu, fidp, &dent); - if (err || !dent) { - break; - } - v9fs_string_init(&name); - v9fs_string_sprintf(&name, "%s", dent->d_name); - if ((count + v9fs_readdir_response_size(&name)) > max_count) { - v9fs_readdir_unlock(&fidp->fs.dir); + /* + * Fetch all required directory entries altogether on a background IO + * thread from fs driver. We don't want to do that for each entry + * individually, because hopping between threads (this main IO thread + * and background IO driver thread) would sum up to huge latencies. + */ + count =3D v9fs_co_readdir_many(pdu, fidp, &entries, offset, max_count, + dostat); + if (count < 0) { + err =3D count; + count =3D 0; + goto out; + } + count =3D 0; =20 - /* Ran out of buffer. Set dir back to old position and return = */ - v9fs_co_seekdir(pdu, fidp, saved_dir_pos); - v9fs_string_free(&name); - return count; - } + for (struct V9fsDirEnt *e =3D entries; e; e =3D e->next) { + dent =3D e->dent; =20 if (pdu->s->ctx.export_flags & V9FS_REMAP_INODES) { - /* - * dirent_to_qid() implies expensive stat call for each entry, - * we must do that here though since inode remapping requires - * the device id, which in turn might be different for - * different entries; we cannot make any assumption to avoid - * that here. - */ - err =3D dirent_to_qid(pdu, fidp, dent, &qid); + st =3D e->st; + /* e->st should never be NULL, but just to be sure */ + if (!st) { + err =3D -1; + break; + } + + /* remap inode */ + err =3D stat_to_qid(pdu, st, &qid); if (err < 0) { - v9fs_readdir_unlock(&fidp->fs.dir); - v9fs_co_seekdir(pdu, fidp, saved_dir_pos); - v9fs_string_free(&name); - return err; + break; } } else { /* * Fill up just the path field of qid because the client uses * only that. To fill the entire qid structure we will have * to stat each dirent found, which is expensive. For the - * latter reason we don't call dirent_to_qid() here. Only draw= back + * latter reason we don't call stat_to_qid() here. Only drawba= ck * is that no multi-device export detection of stat_to_qid() * would be done and provided as error to the user here. But * user would get that error anyway when accessing those @@ -2396,25 +2384,26 @@ static int coroutine_fn v9fs_do_readdir(V9fsPDU *pd= u, V9fsFidState *fidp, qid.version =3D 0; } =20 + v9fs_string_init(&name); + v9fs_string_sprintf(&name, "%s", dent->d_name); + /* 11 =3D 7 + 4 (7 =3D start offset, 4 =3D space for storing count= ) */ len =3D pdu_marshal(pdu, 11 + count, "Qqbs", &qid, dent->d_off, dent->d_type, &name); =20 - v9fs_readdir_unlock(&fidp->fs.dir); + v9fs_string_free(&name); =20 if (len < 0) { - v9fs_co_seekdir(pdu, fidp, saved_dir_pos); - v9fs_string_free(&name); - return len; + err =3D len; + break; } + count +=3D len; - v9fs_string_free(&name); - saved_dir_pos =3D dent->d_off; } =20 - v9fs_readdir_unlock(&fidp->fs.dir); - +out: + v9fs_free_dirents(entries); if (err < 0) { return err; } @@ -2457,12 +2446,7 @@ static void coroutine_fn v9fs_readdir(void *opaque) retval =3D -EINVAL; goto out; } - if (initial_offset =3D=3D 0) { - v9fs_co_rewinddir(pdu, fidp); - } else { - v9fs_co_seekdir(pdu, fidp, initial_offset); - } - count =3D v9fs_do_readdir(pdu, fidp, max_count); + count =3D v9fs_do_readdir(pdu, fidp, (off_t) initial_offset, max_count= ); if (count < 0) { retval =3D count; goto out; --=20 2.20.1