From nobody Sat Nov 15 14:14:02 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=crudebyte.com ARC-Seal: i=1; a=rsa-sha256; t=1595169788; cv=none; d=zohomail.com; s=zohoarc; b=bdhFYc3ivYLudYjV66HIWRUAzr1nt79p17kxI5r7PSgvcMxZndId+5uFCxYjP54YPxCHJ+Oli5hMAnZHXFpMlQNAMnFLJEA7T2Vhl3KQauE3MuEM8JR5+QYdK7hZbIM3JVhYoPvDGCwzoNmYSH3KVda6Ipnp7N1crH/dN6z24qc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1595169788; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=ol0hJmQYo1jT2opLuJdHI9w0b1u+9bnIU7qAzpwfRD0=; b=ZwjilTXfWOxChYlVKe29/dPaQtonNtBHFaE+LmGRDepuFLfy4QRg85LLbQy4G5lT1r5PXm3zI6C6nyHMmURMcW9RMABiYaPja2xHBFpHqN/qJzlC5bW889xekQrZ7gBrHQY9At3x1XtH+ValS6rhn423kdLUcWNxNreiAymAQR4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1595169788910618.2318206718747; Sun, 19 Jul 2020 07:43:08 -0700 (PDT) Received: from localhost ([::1]:36124 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jxAWx-0006IZ-NL for importer@patchew.org; Sun, 19 Jul 2020 10:43:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40456) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jxAWN-0005tL-Dd for qemu-devel@nongnu.org; Sun, 19 Jul 2020 10:42:31 -0400 Received: from lizzy.crudebyte.com ([91.194.90.13]:35281) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jxAWL-0005Uj-Lr for qemu-devel@nongnu.org; Sun, 19 Jul 2020 10:42:31 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=crudebyte.com; s=lizzy; h=Cc:To:Subject:Date:From:References:In-Reply-To: Message-Id:Content-Type:Content-Transfer-Encoding:MIME-Version:Content-ID: Content-Description; bh=ol0hJmQYo1jT2opLuJdHI9w0b1u+9bnIU7qAzpwfRD0=; b=XJ8R3 aaoDaAVZYmtRFWhi7ujwYKytNnoxSKQwRrhbx7WOspBN1ktqV0xduLgsHCMECJEnAxfx6yDZl4Oo6 HXv3vm8JXeU8v9iVhmxq/+A5nXQQPvI/bl1R978zIwwnjSaQoYXKMU7FbM27yo7QYzHVb+nPuIqtn pSb2h/Bfk66IT2XKF8FN1Ug0pypdMSSZYCTP65XLa8HdoibITbWggnjIWRfRufDJZ1UdmDwYHHy64 g0ljm8yr/k7TU6Xv6t7OHKzYtt/zCmb+V07s3m5y7oOM+vzLfZ9vik/W9XrxKiOxo0euuZlNG6erj 3BG2m7Rl95n4TjMic+lGFaE6/h15A==; Message-Id: In-Reply-To: References: From: Christian Schoenebeck Date: Sun, 19 Jul 2020 15:11:13 +0200 Subject: [PATCH v7 4/6] 9pfs: T_readdir latency optimization To: qemu-devel@nongnu.org Cc: Greg Kurz Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: none client-ip=91.194.90.13; envelope-from=cf88b6dfe31a4fd5f58386a8ad1589619b9a1ebc@lizzy.crudebyte.com; helo=lizzy.crudebyte.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/19 10:12:15 X-ACL-Warn: Detected OS = Linux 3.11 and newer X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @crudebyte.com) Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Make top half really top half and bottom half really bottom half: Each T_readdir request handling is hopping between threads (main I/O thread and background I/O driver threads) several times for every individual directory entry, which sums up to huge latencies for handling just a single T_readdir request. Instead of doing that, collect now all required directory entries (including all potentially required stat buffers for each entry) in one rush on a background I/O thread from fs driver by calling the previously added function v9fs_co_readdir_many() instead of v9fs_co_readdir(), then assemble the entire resulting network response message for the readdir request on main I/O thread. The fs driver is still aborting the directory entry retrieval loop (on the background I/O thread inside of v9fs_co_readdir_many()) as soon as it would exceed the client's requested maximum R_readdir response size. So this will not introduce a performance penalty on another end. Also: No longer seek initial directory position in v9fs_readdir(), as this is now handled (more consistently) by v9fs_co_readdir_many() instead. Signed-off-by: Christian Schoenebeck --- hw/9pfs/9p.c | 132 ++++++++++++++++++++++----------------------------- 1 file changed, 58 insertions(+), 74 deletions(-) diff --git a/hw/9pfs/9p.c b/hw/9pfs/9p.c index 7a228c4828..cc4094b971 100644 --- a/hw/9pfs/9p.c +++ b/hw/9pfs/9p.c @@ -972,30 +972,6 @@ static int coroutine_fn fid_to_qid(V9fsPDU *pdu, V9fsF= idState *fidp, return 0; } =20 -static int coroutine_fn dirent_to_qid(V9fsPDU *pdu, V9fsFidState *fidp, - struct dirent *dent, V9fsQID *qidp) -{ - struct stat stbuf; - V9fsPath path; - int err; - - v9fs_path_init(&path); - - err =3D v9fs_co_name_to_path(pdu, &fidp->path, dent->d_name, &path); - if (err < 0) { - goto out; - } - err =3D v9fs_co_lstat(pdu, &path, &stbuf); - if (err < 0) { - goto out; - } - err =3D stat_to_qid(pdu, &stbuf, qidp); - -out: - v9fs_path_free(&path); - return err; -} - V9fsPDU *pdu_alloc(V9fsState *s) { V9fsPDU *pdu =3D NULL; @@ -2328,62 +2304,74 @@ size_t v9fs_readdir_response_size(V9fsString *name) return 24 + v9fs_string_size(name); } =20 +static void v9fs_free_dirents(struct V9fsDirEnt *e) +{ + struct V9fsDirEnt *next =3D NULL; + + for (; e; e =3D next) { + next =3D e->next; + g_free(e->dent); + g_free(e->st); + g_free(e); + } +} + static int coroutine_fn v9fs_do_readdir(V9fsPDU *pdu, V9fsFidState *fidp, - int32_t max_count) + off_t offset, int32_t max_count) { size_t size; V9fsQID qid; V9fsString name; int len, err =3D 0; int32_t count =3D 0; - off_t saved_dir_pos; struct dirent *dent; + struct stat *st; + struct V9fsDirEnt *entries =3D NULL; =20 - /* save the directory position */ - saved_dir_pos =3D v9fs_co_telldir(pdu, fidp); - if (saved_dir_pos < 0) { - return saved_dir_pos; - } - - while (1) { - v9fs_readdir_lock(&fidp->fs.dir); + /* + * inode remapping requires the device id, which in turn might be + * different for different directory entries, so if inode remapping is + * enabled we have to make a full stat for each directory entry + */ + const bool dostat =3D pdu->s->ctx.export_flags & V9FS_REMAP_INODES; =20 - err =3D v9fs_co_readdir(pdu, fidp, &dent); - if (err || !dent) { - break; - } - v9fs_string_init(&name); - v9fs_string_sprintf(&name, "%s", dent->d_name); - if ((count + v9fs_readdir_response_size(&name)) > max_count) { - v9fs_readdir_unlock(&fidp->fs.dir); + /* + * Fetch all required directory entries altogether on a background IO + * thread from fs driver. We don't want to do that for each entry + * individually, because hopping between threads (this main IO thread + * and background IO driver thread) would sum up to huge latencies. + */ + count =3D v9fs_co_readdir_many(pdu, fidp, &entries, offset, max_count, + dostat); + if (count < 0) { + err =3D count; + count =3D 0; + goto out; + } + count =3D 0; =20 - /* Ran out of buffer. Set dir back to old position and return = */ - v9fs_co_seekdir(pdu, fidp, saved_dir_pos); - v9fs_string_free(&name); - return count; - } + for (struct V9fsDirEnt *e =3D entries; e; e =3D e->next) { + dent =3D e->dent; =20 if (pdu->s->ctx.export_flags & V9FS_REMAP_INODES) { - /* - * dirent_to_qid() implies expensive stat call for each entry, - * we must do that here though since inode remapping requires - * the device id, which in turn might be different for - * different entries; we cannot make any assumption to avoid - * that here. - */ - err =3D dirent_to_qid(pdu, fidp, dent, &qid); + st =3D e->st; + /* e->st should never be NULL, but just to be sure */ + if (!st) { + err =3D -1; + break; + } + + /* remap inode */ + err =3D stat_to_qid(pdu, st, &qid); if (err < 0) { - v9fs_readdir_unlock(&fidp->fs.dir); - v9fs_co_seekdir(pdu, fidp, saved_dir_pos); - v9fs_string_free(&name); - return err; + break; } } else { /* * Fill up just the path field of qid because the client uses * only that. To fill the entire qid structure we will have * to stat each dirent found, which is expensive. For the - * latter reason we don't call dirent_to_qid() here. Only draw= back + * latter reason we don't call stat_to_qid() here. Only drawba= ck * is that no multi-device export detection of stat_to_qid() * would be done and provided as error to the user here. But * user would get that error anyway when accessing those @@ -2396,25 +2384,26 @@ static int coroutine_fn v9fs_do_readdir(V9fsPDU *pd= u, V9fsFidState *fidp, qid.version =3D 0; } =20 + v9fs_string_init(&name); + v9fs_string_sprintf(&name, "%s", dent->d_name); + /* 11 =3D 7 + 4 (7 =3D start offset, 4 =3D space for storing count= ) */ len =3D pdu_marshal(pdu, 11 + count, "Qqbs", &qid, dent->d_off, dent->d_type, &name); =20 - v9fs_readdir_unlock(&fidp->fs.dir); + v9fs_string_free(&name); =20 if (len < 0) { - v9fs_co_seekdir(pdu, fidp, saved_dir_pos); - v9fs_string_free(&name); - return len; + err =3D len; + break; } + count +=3D len; - v9fs_string_free(&name); - saved_dir_pos =3D dent->d_off; } =20 - v9fs_readdir_unlock(&fidp->fs.dir); - +out: + v9fs_free_dirents(entries); if (err < 0) { return err; } @@ -2457,12 +2446,7 @@ static void coroutine_fn v9fs_readdir(void *opaque) retval =3D -EINVAL; goto out; } - if (initial_offset =3D=3D 0) { - v9fs_co_rewinddir(pdu, fidp); - } else { - v9fs_co_seekdir(pdu, fidp, initial_offset); - } - count =3D v9fs_do_readdir(pdu, fidp, max_count); + count =3D v9fs_do_readdir(pdu, fidp, (off_t) initial_offset, max_count= ); if (count < 0) { retval =3D count; goto out; --=20 2.20.1