From nobody Sat May 4 05:22:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=crudebyte.com ARC-Seal: i=1; a=rsa-sha256; t=1576682049; cv=none; d=zohomail.com; s=zohoarc; b=WTR+5dg4yJ744YlMgQXzIkEJkL49rKy/QhIUOg26kW18J/Z9Yw5aAMvLrPnviAymbvOTMndrE32Mrwik4Q0mbq6EXikxBaFAcLYxqI67kUg3wtfprr0FmKho0SCOlGuVSlZn2g7Sy/spoudRQ2kDgQfvhx2PiGq7GbokmRFKQLg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1576682049; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=f75LCKXuW/82lh/Zq9Mk5ElCuMF9Qpgl+UzPHjR/EyQ=; b=IiGdGRGP/5VZ6/XKOZsg7kB/EhSnvL1oAvWPLuPJsg8huvGFKBnkZQroP1N2LkL+FTBZfvdheEfVRs4wuc5xcH5XFUdjfG2PmtrlAbyPNyvhIUcbBIjwz0J6ImjeT5qM1ANGBuxn45QFS0h+iOnYnHDng5hVHpAVOUIL23XYnao= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1576682049814250.72536184207797; Wed, 18 Dec 2019 07:14:09 -0800 (PST) Received: from localhost ([::1]:55528 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ihb1c-0001am-JK for importer@patchew.org; Wed, 18 Dec 2019 10:14:08 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:34098) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from <11a52f72d4a301518ad3fa3ec08080cf1d02e3d1@lizzy.crudebyte.com>) id 1ihb0F-0000HS-Hi for qemu-devel@nongnu.org; Wed, 18 Dec 2019 10:12:44 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from <11a52f72d4a301518ad3fa3ec08080cf1d02e3d1@lizzy.crudebyte.com>) id 1ihb0D-0001Dp-M1 for qemu-devel@nongnu.org; Wed, 18 Dec 2019 10:12:42 -0500 Received: from lizzy.crudebyte.com ([91.194.90.13]:42473) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from <11a52f72d4a301518ad3fa3ec08080cf1d02e3d1@lizzy.crudebyte.com>) id 1ihb0C-0004hl-TR for qemu-devel@nongnu.org; Wed, 18 Dec 2019 10:12:41 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=crudebyte.com; s=lizzy; h=Subject:Date:Cc:To:From:References:In-Reply-To: Message-Id:Content-Type:Content-Transfer-Encoding:MIME-Version:Content-ID: Content-Description; bh=f75LCKXuW/82lh/Zq9Mk5ElCuMF9Qpgl+UzPHjR/EyQ=; b=P6rDk qBb8yW5GUiapZCxfjcgiI1kJIN880N8SYp4GX9LSjmYhgZQZEPGxSEwTLeMUxGk1qBTYSDR9oGmeO mt5v0WqBxSHY6zdQp0e2T7559x4l06flD0pKaJieA8NdjZ8pVFGqNLPNkR4M4iEj1OIznD55gaExh v75ZMEbRGLUGej7qtihQXQ8kABAe+xCGwycnefHeJMhjQ4bA4oiTS40QF/HyhKeLXDko46Sk3VNUR 09H02VAlZYC3oGr776tPUvPISEldZ9P6fuygYEgdttipRz8pd8bWTkvAsUs3VxUSKh7AjJGAEA343 7o7dRmIBwI4lWhPxruYmwBF2I5egA==; Message-Id: <11a52f72d4a301518ad3fa3ec08080cf1d02e3d1.1576678644.git.qemu_oss@crudebyte.com> In-Reply-To: References: From: Christian Schoenebeck To: qemu-devel@nongnu.org Cc: Greg Kurz Date: Wed, 18 Dec 2019 14:06:30 +0100 Subject: [PATCH v2 1/9] tests/virtio-9p: add terminating null in v9fs_string_read() X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 91.194.90.13 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @crudebyte.com) Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The 9p protocol sends strings in general without null termination over the wire. However for future use of this functions it is beneficial for the delivered string to be null terminated though for being able to use the string with standard C functions which often rely on strings being null terminated. Signed-off-by: Christian Schoenebeck Reviewed-by: Greg Kurz --- tests/virtio-9p-test.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tests/virtio-9p-test.c b/tests/virtio-9p-test.c index e7b58e3a0c..06263edb53 100644 --- a/tests/virtio-9p-test.c +++ b/tests/virtio-9p-test.c @@ -130,8 +130,9 @@ static void v9fs_string_read(P9Req *req, uint16_t *len,= char **string) *len =3D local_len; } if (string) { - *string =3D g_malloc(local_len); + *string =3D g_malloc(local_len + 1); v9fs_memread(req, *string, local_len); + (*string)[local_len] =3D 0; } else { v9fs_memskip(req, local_len); } --=20 2.20.1 From nobody Sat May 4 05:22:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=crudebyte.com ARC-Seal: i=1; a=rsa-sha256; t=1576682118; cv=none; d=zohomail.com; s=zohoarc; b=BfjioWCIl8TaSZmyCqRk2gAK3dDlWH1l9CXHa+Hl72HG8GhN7T2AQeJLrHuDW74VScA3i5LLE6i/bI26UCgsHvSkpLRiZKMGsVbHCB3Cep+kiC1776rdVxf3MrGJ2bFuAYLYPR+X2Ju4K36+mSVgmFIgJ6ovNOAJC4Mjix6u4ss= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1576682118; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=KBohi6m8+kT0YLSCXjzWFKtZubkOcD4kA+IhcBSo+mw=; b=SZW0vdORXs1wBjI5UcTAx/hxJyx7tw3S501xNQtmNL9or+9n7hCLyxCD5fCufk8LI8rVU8jm98UqikbNy5s7MNUbglsFkiUbH8wVPTZz8jBAW3LDWDOTz8xvLIskIwWnSyekZT/mfkZ7/O52zX03TZTlcPrvZAKYnrsP/0Gb0UI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 157668211866896.60225656905266; Wed, 18 Dec 2019 07:15:18 -0800 (PST) Received: from localhost ([::1]:55564 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ihb2j-0003Es-5c for importer@patchew.org; Wed, 18 Dec 2019 10:15:17 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:41668) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from <4866c87cb19f3191b977fc7fbe2857abf1da80b6@lizzy.crudebyte.com>) id 1ihb1F-0001ku-2q for qemu-devel@nongnu.org; Wed, 18 Dec 2019 10:13:46 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from <4866c87cb19f3191b977fc7fbe2857abf1da80b6@lizzy.crudebyte.com>) id 1ihb1D-0006Sx-RI for qemu-devel@nongnu.org; Wed, 18 Dec 2019 10:13:44 -0500 Received: from lizzy.crudebyte.com ([91.194.90.13]:54257) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from <4866c87cb19f3191b977fc7fbe2857abf1da80b6@lizzy.crudebyte.com>) id 1ihb1D-0001Ee-DX for qemu-devel@nongnu.org; Wed, 18 Dec 2019 10:13:43 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=crudebyte.com; s=lizzy; h=Subject:Date:Cc:To:From:References:In-Reply-To: Message-Id:Content-Type:Content-Transfer-Encoding:MIME-Version:Content-ID: Content-Description; bh=KBohi6m8+kT0YLSCXjzWFKtZubkOcD4kA+IhcBSo+mw=; b=ah2Cf LyXsVtfD5kaavf/HCnjbAyxDT4DplQ1n7xL6ZK4e4x7ElOm15ExiWi8uhv2LgdmB1XxgrvNGAK5R8 knYy/sJlL48B0U9ZusyF3TzsDq/TZnQP7IevfJP9QCIJNBLGz1S08tbKpDBdZd2TGjFSq/TYz2wIo pYYOCN9QCMh27TwrcfPQEQFik+HxqjW9p4lgamz8xIJE/bEPLbWUMI3nrRpVP8UMEcjnskZlWY1Mo b3iPde73l/ePUs5ftWtVeUpk7rA0tEyeAtKvju9Y9d72SEbPsIycAzyuSxbSr2EOssf3DxRcBcWhA 6yrDdtB9LGmOn3ztqM+d3Arx/bwsw==; Message-Id: <4866c87cb19f3191b977fc7fbe2857abf1da80b6.1576678644.git.qemu_oss@crudebyte.com> In-Reply-To: References: From: Christian Schoenebeck To: qemu-devel@nongnu.org Cc: Greg Kurz Date: Wed, 18 Dec 2019 14:17:59 +0100 Subject: [PATCH v2 2/9] 9pfs: validate count sent by client with T_readdir X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 91.194.90.13 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @crudebyte.com) Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" A good 9p client sends T_readdir with "count" parameter that's sufficiently smaller than client's initially negotiated msize (maximum message size). We perform a check for that though to avoid the server to be interrupted with a "Failed to encode VirtFS reply type 41" error message by bad clients. Note: we should probably also check for a minimum size of msize during T_version to avoid issues and/or too complicated count/size checks later on in other requests of that client. T_version should submit an msize that's at least as large as the largest request's header size. Signed-off-by: Christian Schoenebeck --- hw/9pfs/9p.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/hw/9pfs/9p.c b/hw/9pfs/9p.c index 520177f40c..30e33b6573 100644 --- a/hw/9pfs/9p.c +++ b/hw/9pfs/9p.c @@ -2414,6 +2414,7 @@ static void coroutine_fn v9fs_readdir(void *opaque) int32_t count; uint32_t max_count; V9fsPDU *pdu =3D opaque; + V9fsState *s =3D pdu->s; =20 retval =3D pdu_unmarshal(pdu, offset, "dqd", &fid, &initial_offset, &max_count); @@ -2422,6 +2423,13 @@ static void coroutine_fn v9fs_readdir(void *opaque) } trace_v9fs_readdir(pdu->tag, pdu->id, fid, initial_offset, max_count); =20 + if (max_count > s->msize - P9_IOHDRSZ) { + max_count =3D s->msize - P9_IOHDRSZ; + warn_report_once( + "9p: bad client: T_readdir with count > msize - P9_IOHDRSZ" + ); + } + fidp =3D get_fid(pdu, fid); if (fidp =3D=3D NULL) { retval =3D -EINVAL; --=20 2.20.1 From nobody Sat May 4 05:22:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=crudebyte.com ARC-Seal: i=1; a=rsa-sha256; t=1576682240; cv=none; d=zohomail.com; s=zohoarc; b=SYcax8yVnR9pGZ7LXCY6Rx0MxFALV6Cdo7XAExEx2M2ZAuuiRvRDSgstogQETeH1+Ue08ANsic1AixYjcWQUGQo6DbMsmXqYcn9ZjbuZzTgsEHQ7jVxC/s4F6UTeD9qh3pMxI8PQUIlMtR6hVYT19+nvN/BEcNniwT4xZYViSIo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1576682240; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=hLrT/cei1Ph+HSrT13ZTQqozjrT7hfRJvI7gt4DQDjo=; b=IVuUHX+s+cTaLRjaJaYExq2G42PPBW2EJr4Y3vpaz8jSziVCOxwEbgexkvXmJVqLOL6Ac43FS8ymO2F69AXldVx8NSajbLpejH9hE43MtL3/ZLEqRXE+9gzWav1HgNNx+TIw1WTM9d7thOT56QuuB2LaOvw9z7JGZLBHuhqV/Io= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1576682240127753.9609654983728; Wed, 18 Dec 2019 07:17:20 -0800 (PST) Received: from localhost ([::1]:55614 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ihb4g-0005oY-TW for importer@patchew.org; Wed, 18 Dec 2019 10:17:18 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:51702) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from <91905ff817851a76d3becaf27208007dfdbf22e1@lizzy.crudebyte.com>) id 1ihb3D-0004Lo-Nq for qemu-devel@nongnu.org; Wed, 18 Dec 2019 10:15:48 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from <91905ff817851a76d3becaf27208007dfdbf22e1@lizzy.crudebyte.com>) id 1ihb3C-0006aN-EG for qemu-devel@nongnu.org; Wed, 18 Dec 2019 10:15:47 -0500 Received: from lizzy.crudebyte.com ([91.194.90.13]:45427) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from <91905ff817851a76d3becaf27208007dfdbf22e1@lizzy.crudebyte.com>) id 1ihb3C-0001qy-5G for qemu-devel@nongnu.org; Wed, 18 Dec 2019 10:15:46 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=crudebyte.com; s=lizzy; h=Subject:Date:Cc:To:From:References:In-Reply-To: Message-Id:Content-Type:Content-Transfer-Encoding:MIME-Version:Content-ID: Content-Description; bh=hLrT/cei1Ph+HSrT13ZTQqozjrT7hfRJvI7gt4DQDjo=; b=M3kdj 1lzwa7L/qwDHY4tJNp69RXnlkjcRy4PyWOAZsAAQShfis4OFwzrrzmPz8auUT1SnnWo0goHg9EXqR RoH6VHzcIF0n4K87bN0QK5Vbkko4+lsVpjotpjxZQZY4wMAbjXGu/7SoCGn1su81h6561zh5gcmKI AWApp01u4oQTgXjkEElCLfHsyO6xliRQBUi9Xx6i59Y4UVU1uIlaR/cMnEP+Y9WW33f2/mBQJt+If eHMzGtE8r4LHEkTThIDQvBNMPKY1nRMm47TVovuO7usvzq3DMxrbiinjV41JEgdas9D8jqfaV6+qq n+QraHt5RAMof1D9eBUVRU0JA5iUQ==; Message-Id: <91905ff817851a76d3becaf27208007dfdbf22e1.1576678644.git.qemu_oss@crudebyte.com> In-Reply-To: References: From: Christian Schoenebeck To: qemu-devel@nongnu.org Cc: Greg Kurz Date: Wed, 18 Dec 2019 14:23:48 +0100 Subject: [PATCH v2 3/9] hw/9pfs/9p-synth: added directory for readdir test X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 91.194.90.13 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @crudebyte.com) Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This will provide the following virtual files by the 9pfs synth driver: - /ReadDirDir/ReadDirFile99 - /ReadDirDir/ReadDirFile98 ... - /ReadDirDir/ReadDirFile1 - /ReadDirDir/ReadDirFile0 This virtual directory and its virtual 100 files will be used by the upcoming 9pfs readdir tests. Signed-off-by: Christian Schoenebeck Reviewed-by: Greg Kurz --- hw/9pfs/9p-synth.c | 19 +++++++++++++++++++ hw/9pfs/9p-synth.h | 5 +++++ 2 files changed, 24 insertions(+) diff --git a/hw/9pfs/9p-synth.c b/hw/9pfs/9p-synth.c index 54239c9bbf..7eb210ffa8 100644 --- a/hw/9pfs/9p-synth.c +++ b/hw/9pfs/9p-synth.c @@ -578,6 +578,25 @@ static int synth_init(FsContext *ctx, Error **errp) NULL, v9fs_synth_qtest_flush_write, ctx); assert(!ret); + + /* Directory for READDIR test */ + { + V9fsSynthNode *dir =3D NULL; + ret =3D qemu_v9fs_synth_mkdir( + NULL, 0700, QTEST_V9FS_SYNTH_READDIR_DIR, &dir + ); + assert(!ret); + for (i =3D 0; i < QTEST_V9FS_SYNTH_READDIR_NFILES; ++i) { + char *name =3D g_strdup_printf( + QTEST_V9FS_SYNTH_READDIR_FILE, i + ); + ret =3D qemu_v9fs_synth_add_file( + dir, 0, name, NULL, NULL, ctx + ); + assert(!ret); + g_free(name); + } + } } =20 return 0; diff --git a/hw/9pfs/9p-synth.h b/hw/9pfs/9p-synth.h index af7a993a1e..036d7e4a5b 100644 --- a/hw/9pfs/9p-synth.h +++ b/hw/9pfs/9p-synth.h @@ -55,6 +55,11 @@ int qemu_v9fs_synth_add_file(V9fsSynthNode *parent, int = mode, #define QTEST_V9FS_SYNTH_LOPEN_FILE "LOPEN" #define QTEST_V9FS_SYNTH_WRITE_FILE "WRITE" =20 +/* for READDIR test */ +#define QTEST_V9FS_SYNTH_READDIR_DIR "ReadDirDir" +#define QTEST_V9FS_SYNTH_READDIR_FILE "ReadDirFile%d" +#define QTEST_V9FS_SYNTH_READDIR_NFILES 100 + /* Any write to the "FLUSH" file is handled one byte at a time by the * backend. If the byte is zero, the backend returns success (ie, 1), * otherwise it forces the server to try again forever. Thus allowing --=20 2.20.1 From nobody Sat May 4 05:22:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=crudebyte.com ARC-Seal: i=1; a=rsa-sha256; t=1576681759; cv=none; d=zohomail.com; s=zohoarc; b=CJjJ4efResEy2v+iiuLCMgPgl+HWUhqDnyg1xd/VbTVfi+Gw42EChcOYMO351hGkhwBU0xc87CyPFPu1fvLHSSCuhuCp+cSgwSQXkQuh2e9x0Bh5rNKqeyz/4iye/Zn7HIIe12sw2otj2pcgXw9MB3eZdTN/3CKGmnx96a9h1Io= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1576681759; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=qIKNwOBYoiZIKrTToOWeUyN8KcypATIyiBDvg/u2X54=; b=jRbkhxMnZhQM53I4nc9FAMRPQcsHZAV6f6x+QrYroBE5gQAqw4gOf2D5imY+VLlb3CM7+N8R6pnieZ0EUlvsagS3RjGhuYv95NAxgguBeV+G1VfBYgvfc+Ctnwd7CpSfOktKFtvUnR/KbWNBWFR+6VPQAQExowi0gaYF2CWXCuA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 157668175966816.157044646216377; Wed, 18 Dec 2019 07:09:19 -0800 (PST) Received: from localhost ([::1]:55436 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ihawv-0003vB-4D for importer@patchew.org; Wed, 18 Dec 2019 10:09:17 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:60021) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ihawF-0003Tf-Nw for qemu-devel@nongnu.org; Wed, 18 Dec 2019 10:08:37 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ihawE-0003SZ-4y for qemu-devel@nongnu.org; Wed, 18 Dec 2019 10:08:35 -0500 Received: from lizzy.crudebyte.com ([91.194.90.13]:52141) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ihawD-0006FQ-KN for qemu-devel@nongnu.org; Wed, 18 Dec 2019 10:08:34 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=crudebyte.com; s=lizzy; h=Subject:Date:Cc:To:From:References:In-Reply-To: Message-Id:Content-Type:Content-Transfer-Encoding:MIME-Version:Content-ID: Content-Description; bh=qIKNwOBYoiZIKrTToOWeUyN8KcypATIyiBDvg/u2X54=; b=LxsxY t1CG7a8ljFxs3XKf9WXE0AHpMn3T9oQsPUpzVppgMFPLI5ZJb/JHhjbvUEKl+YBx85fD5BodTKWdE 8OxlK2XjynbpY/be5f7Q71xTqv8Sjg8/21f1/XZvx0MbttADP04QhOsvmNuDX7+uxuXCDaY4FNLvY GUjURJRQs7J5sLxt1CNVv5hcnuS4Hs5n9jVJd2BQjxVeb6ux6ZELgq/NQp09m++chFWclzkciZSJW XdNL1Nk6/JFr6VPh0deYdW4H/ipEUonIcfKWEX7ulFRM3zPBqUu2kHuTJWg6eWhRNAHxCqPIjWjas 6enTWuGGpJs9vlHrizkjBlb9XTkiw==; Message-Id: In-Reply-To: References: From: Christian Schoenebeck To: qemu-devel@nongnu.org Cc: Greg Kurz Date: Wed, 18 Dec 2019 14:30:43 +0100 Subject: [PATCH v2 4/9] tests/virtio-9p: added readdir test X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 91.194.90.13 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @crudebyte.com) Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This first readdir test simply checks the amount of directory entries returned by 9pfs server, according to the created amount of virtual files on 9pfs synth driver side. Signed-off-by: Christian Schoenebeck --- tests/virtio-9p-test.c | 124 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 124 insertions(+) diff --git a/tests/virtio-9p-test.c b/tests/virtio-9p-test.c index 06263edb53..48c0eca292 100644 --- a/tests/virtio-9p-test.c +++ b/tests/virtio-9p-test.c @@ -68,6 +68,11 @@ static void v9fs_memread(P9Req *req, void *addr, size_t = len) req->r_off +=3D len; } =20 +static void v9fs_uint8_read(P9Req *req, uint8_t *val) +{ + v9fs_memread(req, val, 1); +} + static void v9fs_uint16_write(P9Req *req, uint16_t val) { uint16_t le_val =3D cpu_to_le16(val); @@ -101,6 +106,12 @@ static void v9fs_uint32_read(P9Req *req, uint32_t *val) le32_to_cpus(val); } =20 +static void v9fs_uint64_read(P9Req *req, uint64_t *val) +{ + v9fs_memread(req, val, 8); + le64_to_cpus(val); +} + /* len[2] string[len] */ static uint16_t v9fs_string_size(const char *string) { @@ -191,6 +202,7 @@ static const char *rmessage_name(uint8_t id) id =3D=3D P9_RLOPEN ? "RLOPEN" : id =3D=3D P9_RWRITE ? "RWRITE" : id =3D=3D P9_RFLUSH ? "RFLUSH" : + id =3D=3D P9_RREADDIR ? "READDIR" : ""; } =20 @@ -348,6 +360,79 @@ static void v9fs_rwalk(P9Req *req, uint16_t *nwqid, v9= fs_qid **wqid) v9fs_req_free(req); } =20 +/* size[4] Treaddir tag[2] fid[4] offset[8] count[4] */ +static P9Req *v9fs_treaddir(QVirtio9P *v9p, uint32_t fid, uint64_t offset, + uint32_t count, uint16_t tag) +{ + P9Req *req; + + req =3D v9fs_req_init(v9p, 4 + 8 + 4, P9_TREADDIR, tag); + v9fs_uint32_write(req, fid); + v9fs_uint64_write(req, offset); + v9fs_uint32_write(req, count); + v9fs_req_send(req); + return req; +} + +struct v9fs_dirent { + v9fs_qid qid; + uint64_t offset; + uint8_t type; + char *name; + struct v9fs_dirent *next; +}; + +/* size[4] Rreaddir tag[2] count[4] data[count] */ +static void v9fs_rreaddir(P9Req *req, uint32_t *count, uint32_t *nentries, + struct v9fs_dirent **entries) +{ + uint32_t sz; + struct v9fs_dirent *e =3D NULL; + uint16_t slen; + uint32_t n =3D 0; + + v9fs_req_recv(req, P9_RREADDIR); + v9fs_uint32_read(req, &sz); + + if (count) { + *count =3D sz; + } + + for (int32_t togo =3D (int32_t)sz; + togo >=3D 13 + 8 + 1 + 2; + togo -=3D 13 + 8 + 1 + 2 + slen, ++n) + { + if (!e) { + e =3D g_malloc(sizeof(struct v9fs_dirent)); + if (entries) + *entries =3D e; + } else { + e =3D e->next =3D g_malloc(sizeof(struct v9fs_dirent)); + } + e->next =3D NULL; + /* qid[13] offset[8] type[1] name[s] */ + v9fs_memread(req, &e->qid, 13); + v9fs_uint64_read(req, &e->offset); + v9fs_uint8_read(req, &e->type); + v9fs_string_read(req, &slen, &e->name); + } + + if (nentries) { + *nentries =3D n; + } +} + +static void v9fs_free_dirents(struct v9fs_dirent *e) +{ + struct v9fs_dirent *next =3D NULL; + + for (; e; e =3D next) { + next =3D e->next; + g_free(e->name); + g_free(e); + } +} + /* size[4] Tlopen tag[2] fid[4] flags[4] */ static P9Req *v9fs_tlopen(QVirtio9P *v9p, uint32_t fid, uint32_t flags, uint16_t tag) @@ -480,6 +565,44 @@ static void fs_walk(void *obj, void *data, QGuestAlloc= ator *t_alloc) g_free(wqid); } =20 +static void fs_readdir(void *obj, void *data, QGuestAllocator *t_alloc) +{ + QVirtio9P *v9p =3D obj; + alloc =3D t_alloc; + char *const wnames[] =3D { g_strdup(QTEST_V9FS_SYNTH_READDIR_DIR) }; + uint16_t nqid; + v9fs_qid qid; + uint32_t count, nentries; + struct v9fs_dirent *entries =3D NULL; + P9Req *req; + + fs_attach(v9p, NULL, t_alloc); + req =3D v9fs_twalk(v9p, 0, 1, 1, wnames, 0); + v9fs_req_wait_for_reply(req, NULL); + v9fs_rwalk(req, &nqid, NULL); + g_assert_cmpint(nqid, =3D=3D, 1); + + req =3D v9fs_tlopen(v9p, 1, O_DIRECTORY, 0); + v9fs_req_wait_for_reply(req, NULL); + v9fs_rlopen(req, &qid, NULL); + + req =3D v9fs_treaddir(v9p, 1, 0, P9_MAX_SIZE - P9_IOHDRSZ, 0); + v9fs_req_wait_for_reply(req, NULL); + v9fs_rreaddir(req, &count, &nentries, &entries); + + /* + * Assuming msize (P9_MAX_SIZE) is large enough so we can retrieve all + * dir entries with only one readdir request. + */ + g_assert_cmpint( + nentries, =3D=3D, + QTEST_V9FS_SYNTH_READDIR_NFILES + 2 /* "." and ".." */ + ); + + v9fs_free_dirents(entries); + g_free(wnames[0]); +} + static void fs_walk_no_slash(void *obj, void *data, QGuestAllocator *t_all= oc) { QVirtio9P *v9p =3D obj; @@ -658,6 +781,7 @@ static void register_virtio_9p_test(void) NULL); qos_add_test("fs/flush/ignored", "virtio-9p", fs_flush_ignored, NULL); + qos_add_test("fs/readdir/basic", "virtio-9p", fs_readdir, NULL); } =20 libqos_init(register_virtio_9p_test); --=20 2.20.1 From nobody Sat May 4 05:22:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=crudebyte.com ARC-Seal: i=1; a=rsa-sha256; t=1576682298; cv=none; d=zohomail.com; s=zohoarc; b=KXm+H9+PK9v5FWqY9kOLdJebKFgwgXlPosH1gMn1XWaxvTPUfhIm3vdKkZlMGC4GiPub7ga3m9gHnDCHGzU0N13M1RwjqePj1WxeRbSQhJTiPD5Fs/bEwHodKq8aQIJ7EPd5C1+3JdIdNBB2Uxc85+FeUYcg0bhe9P34HWAPVDY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1576682298; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=dS1iuR3dCQd46YTSHKj8EZ1+/iu1EX7LIzJnLFpbqfQ=; b=ZC2Gq5Hif1gxOuiU+Dn4yOyZRE6rDYUNxSKyT4+Yc7U3Ur9IhLsgiHFqodqqZ64Xqr+vdRKFbSNM+LHyUzu7HvrNgiim3sCq426shSr7hK2B1h8FbwwZ19fSpjCIRmUanfGtR+VrRh/MeAueEu0emOUe/M8tGxr4y271SEt94TQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1576682298310817.6768238935206; Wed, 18 Dec 2019 07:18:18 -0800 (PST) Received: from localhost ([::1]:55632 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ihb5d-0006kl-7d for importer@patchew.org; Wed, 18 Dec 2019 10:18:17 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:59275) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from <2a40095340d9536575af8fd214a7ac647b4603c9@lizzy.crudebyte.com>) id 1ihb4C-0005Xz-OK for qemu-devel@nongnu.org; Wed, 18 Dec 2019 10:16:49 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from <2a40095340d9536575af8fd214a7ac647b4603c9@lizzy.crudebyte.com>) id 1ihb4B-000436-J6 for qemu-devel@nongnu.org; Wed, 18 Dec 2019 10:16:48 -0500 Received: from lizzy.crudebyte.com ([91.194.90.13]:52497) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from <2a40095340d9536575af8fd214a7ac647b4603c9@lizzy.crudebyte.com>) id 1ihb4B-0006bb-A1 for qemu-devel@nongnu.org; Wed, 18 Dec 2019 10:16:47 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=crudebyte.com; s=lizzy; h=Subject:Date:Cc:To:From:References:In-Reply-To: Message-Id:Content-Type:Content-Transfer-Encoding:MIME-Version:Content-ID: Content-Description; bh=dS1iuR3dCQd46YTSHKj8EZ1+/iu1EX7LIzJnLFpbqfQ=; b=SPxy7 F0MDCIgP8kqguZzShP3LZfUxYXYPE6reJmoNy9o9Qq/fd2gGYvlCUmhtNqHd3qTm41SBtV5Pd+j4m cZydf6x8sk5TrDrStOJRalB8yecl0oqXbCvG/aO2zFgLdT855b0tRpw5aHCYRKlOc52OywdEOqgqL T6U6Ehu4qGLJD/3xKaxazZRYtDX584PJPfnsr1I/WHxtDGl/j+h6BR1pFcZ9YgxBAny2KqNONOtgf SOJ//dJ17qGGFBWA1HuGFc3wBWKLUgFzNKx1VJIuMpKEurDaGBhj3cP2M4FhYUfIdOLjk6GBb3Srf 4JKw3N3jXaouYm8wgQ8EKlUHysIzg==; Message-Id: <2a40095340d9536575af8fd214a7ac647b4603c9.1576678644.git.qemu_oss@crudebyte.com> In-Reply-To: References: From: Christian Schoenebeck To: qemu-devel@nongnu.org Cc: Greg Kurz Date: Wed, 18 Dec 2019 14:35:56 +0100 Subject: [PATCH v2 5/9] tests/virtio-9p: check file names of R_readdir response X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 91.194.90.13 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @crudebyte.com) Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Additionally to the already existing check for expected amount of directory entries returned by R_readdir response, also check whether all directory entries have the expected file names (as created on 9pfs synth driver side), ignoring their precise order in result list though. Signed-off-by: Christian Schoenebeck --- tests/virtio-9p-test.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/tests/virtio-9p-test.c b/tests/virtio-9p-test.c index 48c0eca292..cb5c9fb420 100644 --- a/tests/virtio-9p-test.c +++ b/tests/virtio-9p-test.c @@ -565,6 +565,16 @@ static void fs_walk(void *obj, void *data, QGuestAlloc= ator *t_alloc) g_free(wqid); } =20 +static bool fs_dirents_contain_name(struct v9fs_dirent *e, const char* nam= e) +{ + for (; e; e =3D e->next) { + if (!strcmp(e->name, name)) { + return true; + } + } + return false; +} + static void fs_readdir(void *obj, void *data, QGuestAllocator *t_alloc) { QVirtio9P *v9p =3D obj; @@ -599,6 +609,18 @@ static void fs_readdir(void *obj, void *data, QGuestAl= locator *t_alloc) QTEST_V9FS_SYNTH_READDIR_NFILES + 2 /* "." and ".." */ ); =20 + /* + * Check all file names exist in returned entries, ignore their order + * though. + */ + g_assert_cmpint(fs_dirents_contain_name(entries, "."), =3D=3D, true); + g_assert_cmpint(fs_dirents_contain_name(entries, ".."), =3D=3D, true); + for (int i =3D 0; i < QTEST_V9FS_SYNTH_READDIR_NFILES; ++i) { + char *name =3D g_strdup_printf(QTEST_V9FS_SYNTH_READDIR_FILE, i); + g_assert_cmpint(fs_dirents_contain_name(entries, name), =3D=3D, tr= ue); + g_free(name); + } + v9fs_free_dirents(entries); g_free(wnames[0]); } --=20 2.20.1 From nobody Sat May 4 05:22:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=crudebyte.com ARC-Seal: i=1; a=rsa-sha256; t=1576682183; cv=none; d=zohomail.com; s=zohoarc; b=EwE8kPPf7Qj24fD6uBSo9pGL21jSY1Kkvv1Js8Pcqmc00CMd0ug4mZpXgHutMRtofrvZpzHPLpMnP0uuQLtQ+0+nbS/qv0j6XPrLj1f10QgQPT6P78j1P/KkY9xEIpaWOWcue4Fywl8mqv9OmO+AVh7qdakrTALFZUHTZ/Dc/SQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1576682183; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=u7Pi8DHq80FsSvKzTS9rTP58g706j7ijrYyMVBFOJ30=; b=crwAV3OJgVR0NvTwgXHlGA0B6rEjHsC+/9w7GynjnYs4pkZyS0hFw5xX86HbIg5ERTCAAczVRze4lQITvaXMx+9BfzUIUSgVY0ef1b4BtxOrsam/qHP/FPRbk0CrFfELDuWBUXnzQsBXyUGsTTpdhyEpWUwt8wiVZJqrHLbGId8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1576682183938201.79018283878634; Wed, 18 Dec 2019 07:16:23 -0800 (PST) Received: from localhost ([::1]:55592 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ihb3m-0004SS-Dc for importer@patchew.org; Wed, 18 Dec 2019 10:16:22 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46210) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from <7c0b113e491530d31731b20cba51ffb00a4435c4@lizzy.crudebyte.com>) id 1ihb2F-00037q-Jk for qemu-devel@nongnu.org; Wed, 18 Dec 2019 10:14:48 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from <7c0b113e491530d31731b20cba51ffb00a4435c4@lizzy.crudebyte.com>) id 1ihb2D-0001pq-4s for qemu-devel@nongnu.org; Wed, 18 Dec 2019 10:14:46 -0500 Received: from lizzy.crudebyte.com ([91.194.90.13]:59215) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from <7c0b113e491530d31731b20cba51ffb00a4435c4@lizzy.crudebyte.com>) id 1ihb2C-0006UA-Pi for qemu-devel@nongnu.org; Wed, 18 Dec 2019 10:14:45 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=crudebyte.com; s=lizzy; h=Subject:Date:Cc:To:From:References:In-Reply-To: Message-Id:Content-Type:Content-Transfer-Encoding:MIME-Version:Content-ID: Content-Description; bh=u7Pi8DHq80FsSvKzTS9rTP58g706j7ijrYyMVBFOJ30=; b=KHMIw 4j4HYF0Jtxbb7wsBGMun0Bnab2as5VIWJ13C7oESKhlb+Y11Tir7Tc9FYEahHTshxueqJSfc3tccF zQknFyiFe3pK1tuy/QP2zkN62Bgkh0AXwbmH6kGfNQBgli4FA5xZEixJnSb7EflVOIkV1R81xn5Mn MrFFYzccMS+6KduMDtl9QOogJhreHuoicA+8LvF6aUd0m3AXx5P5gGp0nDT4DT0PvTuYr+5KGkNxe +4hbPKikNyVJafFFo9QZEHGTxClVzsv11vyDk6FlbECi0eg5mdwusNI8rthcGU/YZlXu7f2Tzm6eY /2uJ3XPPTUON5J5lRUoJYyJQ8ChFA==; Message-Id: <7c0b113e491530d31731b20cba51ffb00a4435c4.1576678644.git.qemu_oss@crudebyte.com> In-Reply-To: References: From: Christian Schoenebeck To: qemu-devel@nongnu.org Cc: Greg Kurz Date: Wed, 18 Dec 2019 14:43:38 +0100 Subject: [PATCH v2 6/9] 9pfs: readdir benchmark X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 91.194.90.13 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @crudebyte.com) Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This patch is not intended to be merged. It just provides a temporary benchmark foundation for coneniently A/B comparison of the subsequent 9p readdir optimization patches: * hw/9pfs/9p-synth: increase amount of simulated files for readdir test to 2000 files. * tests/virtio-9p: measure wall time that elapsed between sending T_readdir request and arrival of R_readdir response and print out that measured duration, as well as amount of directory entries received, and the amount of bytes of the response message. * tests/virtio-9p: increased msize to 256kiB to allow retrieving all 2000 files (simulated by 9pfs synth driver) with only one T_readdir request. Running this benchmark is fairly quick & simple and does not require any guest OS installation or other prerequisites: cd build make && make tests/qos-test export QTEST_QEMU_BINARY=3Dx86_64-softmmu/qemu-system-x86_64 tests/qos-test -p $(tests/qos-test -l | grep readdir/basic) Since this benchmark uses the 9pfs synth driver, the host machine's I/O hardware (SSDs/HDDs) is not relevant for the benchmark result, because the synth backend's readdir implementation returns immediately (without any blocking I/O that would incur with a real-life fs driver) and just returns already prepared, simulated directory entries directly from RAM. So this benchmark focuses on the efficiency of the 9pfs controller code (or top half) for readdir request handling. Signed-off-by: Christian Schoenebeck --- hw/9pfs/9p-synth.h | 2 +- tests/virtio-9p-test.c | 37 ++++++++++++++++++++++++++++++++++++- 2 files changed, 37 insertions(+), 2 deletions(-) diff --git a/hw/9pfs/9p-synth.h b/hw/9pfs/9p-synth.h index 036d7e4a5b..7d6cedcdac 100644 --- a/hw/9pfs/9p-synth.h +++ b/hw/9pfs/9p-synth.h @@ -58,7 +58,7 @@ int qemu_v9fs_synth_add_file(V9fsSynthNode *parent, int m= ode, /* for READDIR test */ #define QTEST_V9FS_SYNTH_READDIR_DIR "ReadDirDir" #define QTEST_V9FS_SYNTH_READDIR_FILE "ReadDirFile%d" -#define QTEST_V9FS_SYNTH_READDIR_NFILES 100 +#define QTEST_V9FS_SYNTH_READDIR_NFILES 2000 =20 /* Any write to the "FLUSH" file is handled one byte at a time by the * backend. If the byte is zero, the backend returns success (ie, 1), diff --git a/tests/virtio-9p-test.c b/tests/virtio-9p-test.c index cb5c9fb420..7feeca06ff 100644 --- a/tests/virtio-9p-test.c +++ b/tests/virtio-9p-test.c @@ -15,6 +15,18 @@ #include "libqos/virtio-9p.h" #include "libqos/qgraph.h" =20 +/* + * to benchmark the real time (not CPU time) that elapsed between start of + * a request and arrival of its response + */ +static double wall_time(void) +{ + struct timeval t; + struct timezone tz; + gettimeofday(&t, &tz); + return t.tv_sec + t.tv_usec * 0.000001; +} + #define QVIRTIO_9P_TIMEOUT_US (10 * 1000 * 1000) static QGuestAllocator *alloc; =20 @@ -36,7 +48,7 @@ static void pci_config(void *obj, void *data, QGuestAlloc= ator *t_alloc) g_free(tag); } =20 -#define P9_MAX_SIZE 4096 /* Max size of a T-message or R-message */ +#define P9_MAX_SIZE (256 * 1024) /* Max size of a T-message or R-message */ =20 typedef struct { QTestState *qts; @@ -596,9 +608,32 @@ static void fs_readdir(void *obj, void *data, QGuestAl= locator *t_alloc) v9fs_req_wait_for_reply(req, NULL); v9fs_rlopen(req, &qid, NULL); =20 + const double start =3D wall_time(); + req =3D v9fs_treaddir(v9p, 1, 0, P9_MAX_SIZE - P9_IOHDRSZ, 0); + const double treaddir =3D wall_time(); v9fs_req_wait_for_reply(req, NULL); + const double waitforreply =3D wall_time(); v9fs_rreaddir(req, &count, &nentries, &entries); + const double end =3D wall_time(); + + printf("\nTime client spent on sending T_readdir: %fs\n\n", + treaddir - start); + + printf("Time client spent for waiting for reply from server: %fs " + "[MOST IMPORTANT]\n", waitforreply - start); + printf("(This is the most important value, because it reflects the tim= e\n" + "the 9p server required to process and return the result of the= \n" + "T_readdir request.)\n\n"); + + printf("Total client time: %fs\n", end - start); + printf("(NOTE: this time is not relevant; this huge time comes from\n" + "inefficient qtest_memread() calls. So you can discard this\n" + "value as a problem of this test client implementation while\n" + "processing the received server T_readdir reply.)\n\n"); + + printf("Details of response message data: R_readddir nentries=3D%d " + "rbytes=3D%d\n", nentries, count); =20 /* * Assuming msize (P9_MAX_SIZE) is large enough so we can retrieve all --=20 2.20.1 From nobody Sat May 4 05:22:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=crudebyte.com ARC-Seal: i=1; a=rsa-sha256; t=1576681833; cv=none; d=zohomail.com; s=zohoarc; b=GI1yGxZwWGhxIToCROReXRGyjx2z43DG4Bq8gelESKtx5/N7bgv3uT2QdbZ3Q7kv1UrZLFgMMkScUeS9wooHlcmgml3GJcpsmNJ9yYw2UMUnnc7pd08/90vjfeQksPEjY8+jVNzM/7BPvkuoH75m9SWa1+JNv1jLDq5UU7HB2sw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1576681833; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=PrslkhuesGKKMdjjO00pl3IBF8eqlXsPArsnSI4/mCw=; b=ID6lKckZ3aROmF5YWlBPvEwCSLu/K9f3RgJHOEqmrILVI/VUG6XGfrdMs4ax5v+OxO0CW1Ou2o4BEwe8wjsHZTPm6+iYmbufa/yFMafEbespWA7dTYhxLZqr+VERBfK5875Tie/EtkDf1IUAM1ck8tA5yttkBHlMiJw5G710uw4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1576681833914400.76925346825294; Wed, 18 Dec 2019 07:10:33 -0800 (PST) Received: from localhost ([::1]:55454 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ihay8-0005DW-Ij for importer@patchew.org; Wed, 18 Dec 2019 10:10:32 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:40235) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ihaxE-0004Np-KN for qemu-devel@nongnu.org; Wed, 18 Dec 2019 10:09:37 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ihaxD-0001i2-EW for qemu-devel@nongnu.org; Wed, 18 Dec 2019 10:09:36 -0500 Received: from lizzy.crudebyte.com ([91.194.90.13]:51709) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ihaxD-0003Tt-4t for qemu-devel@nongnu.org; Wed, 18 Dec 2019 10:09:35 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=crudebyte.com; s=lizzy; h=Subject:Date:Cc:To:From:References:In-Reply-To: Message-Id:Content-Type:Content-Transfer-Encoding:MIME-Version:Content-ID: Content-Description; bh=PrslkhuesGKKMdjjO00pl3IBF8eqlXsPArsnSI4/mCw=; b=mDM26 sRXzAy2xvhAaMbMUM5gV8ceKbIdRhCoHD0MH6QkSudJr9BERfZ2B775BFDHrFY17jiju3uGmfzSLr rq0r2jSogCql06HU2VMGxD7usjf63pLjWbB2iVp9E4V6h+HqcrBKpOYfH93orGlPe/z8Ejs6yVgaU YcFEFYEJfGAUW3vV+y5I1aV6AE9sKdkgBSJ8wsEPFyQfIpcP2Q2BUXlPu4f2zjslOXECT1HvpbzdR qZcmXfed3ZVTM1UVRlFTa1gUNhrM62FA3p57RmjlO5PnBpg2SV0v1itadMoK2jGx3896xkPCFTyMG 0xXJDU+evbJ7E/H2tJW1lOtmDu6xw==; Message-Id: In-Reply-To: References: From: Christian Schoenebeck To: qemu-devel@nongnu.org Cc: Greg Kurz Date: Wed, 18 Dec 2019 14:52:03 +0100 Subject: [PATCH v2 7/9] hw/9pfs/9p-synth: avoid n-square issue in synth_readdir() X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 91.194.90.13 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @crudebyte.com) Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This patch is just a temporary benchmark hack, not intended to be merged! 9pfs synth driver's readdir() implementation has a severe n-square performance problem. This patch is a quick and dirty hack to prevent that performance problem from tainting the readdir() benchmark results. In its current form, this patch is not useful for anything else than for an isolated readdir benchmark. Signed-off-by: Christian Schoenebeck --- hw/9pfs/9p-synth.c | 29 ++++++++++++++++++++++++++--- 1 file changed, 26 insertions(+), 3 deletions(-) diff --git a/hw/9pfs/9p-synth.c b/hw/9pfs/9p-synth.c index 7eb210ffa8..54dc30f37b 100644 --- a/hw/9pfs/9p-synth.c +++ b/hw/9pfs/9p-synth.c @@ -225,7 +225,8 @@ static void synth_direntry(V9fsSynthNode *node, } =20 static struct dirent *synth_get_dentry(V9fsSynthNode *dir, - struct dirent *entry, off_t of= f) + struct dirent *entry, off_t off, + V9fsSynthNode **hack) { int i =3D 0; V9fsSynthNode *node; @@ -243,16 +244,38 @@ static struct dirent *synth_get_dentry(V9fsSynthNode = *dir, /* end of directory */ return NULL; } + *hack =3D node; synth_direntry(node, entry, off); return entry; } =20 static struct dirent *synth_readdir(FsContext *ctx, V9fsFidOpenState *fs) { - struct dirent *entry; + struct dirent *entry =3D NULL; V9fsSynthOpenState *synth_open =3D fs->private; V9fsSynthNode *node =3D synth_open->node; - entry =3D synth_get_dentry(node, &synth_open->dent, synth_open->offset= ); + + /* + * HACK: This is just intended for benchmark, to avoid severe n-square + * performance problem of synth driver's readdir implementation here w= hich + * would otherwise unncessarily taint the benchmark results. By simply + * caching (globally) the previous node (of the previous synth_readdir= () + * call) we can simply proceed to next node in chained list efficientl= y. + * + * not a good idea for any production code ;-) + */ + static struct V9fsSynthNode *cachedNode; + + if (!cachedNode) { + entry =3D synth_get_dentry(node, &synth_open->dent, synth_open->of= fset, + &cachedNode); + } else { + cachedNode =3D cachedNode->sibling.le_next; + if (cachedNode) { + entry =3D &synth_open->dent; + synth_direntry(cachedNode, entry, synth_open->offset + 1); + } + } if (entry) { synth_open->offset++; } --=20 2.20.1 From nobody Sat May 4 05:22:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=crudebyte.com ARC-Seal: i=1; a=rsa-sha256; t=1576681992; cv=none; d=zohomail.com; s=zohoarc; b=KcqvrHZuCljiXH1jVo++tQhmx3mdVYupHT5zAhWTOvDdSous9frJTXSQXz0qLcqhTiho3yNaYlkcoJvbB2lMniPQdCq4dGjo+jPa+TaWDZt0p1H5mA9d6D2XwG21vUOgKFirpzdsZIMe66A2POcSRLFAD9eai9dJVUxX/eHf/XQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1576681992; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=hMLy1vTDceH5fUdn6fiu3I0Nc0HN1OhTeonc8ybektc=; b=P/fn0xRph0vUoharvkJXAGkmaGy0bni6BMBik5os9yWKzXyl4zGLRcoHE4Jj4u7WF7XBx238LXHhzypNa5LWvblGItIokjUCCIG9ErBsh/TtSQ7s2mbOOTaAlGmTA6GA/GRQZ4amfLJ1ThHD6RyMb9NQ499P6oKW7ghz5g9uxmA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1576681992626977.1654853195037; Wed, 18 Dec 2019 07:13:12 -0800 (PST) Received: from localhost ([::1]:55522 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ihb0h-0000Ph-1U for importer@patchew.org; Wed, 18 Dec 2019 10:13:11 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:55059) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from <4e387d39f47b7184fc0e48eb5471164299558755@lizzy.crudebyte.com>) id 1ihazF-0007PX-P5 for qemu-devel@nongnu.org; Wed, 18 Dec 2019 10:11:43 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from <4e387d39f47b7184fc0e48eb5471164299558755@lizzy.crudebyte.com>) id 1ihazD-0004gG-F9 for qemu-devel@nongnu.org; Wed, 18 Dec 2019 10:11:41 -0500 Received: from lizzy.crudebyte.com ([91.194.90.13]:43007) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from <4e387d39f47b7184fc0e48eb5471164299558755@lizzy.crudebyte.com>) id 1ihazD-0007kI-0U for qemu-devel@nongnu.org; Wed, 18 Dec 2019 10:11:39 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=crudebyte.com; s=lizzy; h=Subject:Date:Cc:To:From:References:In-Reply-To: Message-Id:Content-Type:Content-Transfer-Encoding:MIME-Version:Content-ID: Content-Description; bh=hMLy1vTDceH5fUdn6fiu3I0Nc0HN1OhTeonc8ybektc=; b=dBO+n bpA7gNnJbAFu06GbAWuyB3enNilzm7HE+V6JVb2g4AAEICrJxuqSdGmKPzCCAghfR/W0s7sYFKPEH hUStGn0KW0yjELLAa4CQ1E4VH7pFc5ieHWJwWvuLPgfKGRf9a0HIuyOL2JSZWwXszqoJmr5BWYKpy neMccGrgrllHA1y1TLE1S9snl+eiOcEEnq/WMjG4YpxDf0Dwh6sZ5agA26aNgpkoe9pd4145Ph7Ob jJ4Wmqe3+wDzmqAGvTd+3oBHupKlwpd+viEOX3a7NfIjE1aRR7tLauy1tz60s/UazBt+3n5yQDkMv dDK0B5F0hcSk6iPt4RTQWaXe/SlvA==; Message-Id: <4e387d39f47b7184fc0e48eb5471164299558755.1576678644.git.qemu_oss@crudebyte.com> In-Reply-To: References: From: Christian Schoenebeck To: qemu-devel@nongnu.org Cc: Greg Kurz Date: Wed, 18 Dec 2019 15:00:56 +0100 Subject: [PATCH v2 8/9] 9pfs: T_readdir latency optimization X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 91.194.90.13 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @crudebyte.com) Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Make top half really top half and bottom half really bottom half: Each T_readdir request handling is hopping between threads (main I/O thread and background I/O driver threads) several times for every individual directory entry, which sums up to huge latencies for handling just a single T_readdir request. Instead of doing that, collect now all required directory entries (including all potentially required stat buffers for each entry) in one rush on a background I/O thread from fs driver, then assemble the entire resulting network response message for the readdir request on main I/O thread. The fs driver is still aborting the directory entry retrieval loop (on the background I/O thread) as soon as it would exceed the client's requested maximum R_readdir response size. So we should not have any performance penalty by doing this. Signed-off-by: Christian Schoenebeck --- hw/9pfs/9p.c | 124 +++++++++++++++----------------- hw/9pfs/9p.h | 23 ++++++ hw/9pfs/codir.c | 183 +++++++++++++++++++++++++++++++++++++++++++++--- hw/9pfs/coth.h | 3 + 4 files changed, 254 insertions(+), 79 deletions(-) diff --git a/hw/9pfs/9p.c b/hw/9pfs/9p.c index 30e33b6573..a7d36b6350 100644 --- a/hw/9pfs/9p.c +++ b/hw/9pfs/9p.c @@ -971,30 +971,6 @@ static int coroutine_fn fid_to_qid(V9fsPDU *pdu, V9fsF= idState *fidp, return 0; } =20 -static int coroutine_fn dirent_to_qid(V9fsPDU *pdu, V9fsFidState *fidp, - struct dirent *dent, V9fsQID *qidp) -{ - struct stat stbuf; - V9fsPath path; - int err; - - v9fs_path_init(&path); - - err =3D v9fs_co_name_to_path(pdu, &fidp->path, dent->d_name, &path); - if (err < 0) { - goto out; - } - err =3D v9fs_co_lstat(pdu, &path, &stbuf); - if (err < 0) { - goto out; - } - err =3D stat_to_qid(pdu, &stbuf, qidp); - -out: - v9fs_path_free(&path); - return err; -} - V9fsPDU *pdu_alloc(V9fsState *s) { V9fsPDU *pdu =3D NULL; @@ -2302,7 +2278,7 @@ out_nofid: pdu_complete(pdu, err); } =20 -static size_t v9fs_readdir_data_size(V9fsString *name) +size_t v9fs_readdir_response_size(V9fsString *name) { /* * Size of each dirent on the wire: size of qid (13) + size of offset = (8) @@ -2311,6 +2287,18 @@ static size_t v9fs_readdir_data_size(V9fsString *nam= e) return 24 + v9fs_string_size(name); } =20 +static void v9fs_free_dirents(struct V9fsDirEnt *e) +{ + struct V9fsDirEnt *next =3D NULL; + + for (; e; e =3D next) { + next =3D e->next; + g_free(e->dent); + g_free(e->st); + g_free(e); + } +} + static int coroutine_fn v9fs_do_readdir(V9fsPDU *pdu, V9fsFidState *fidp, int32_t max_count) { @@ -2319,54 +2307,53 @@ static int coroutine_fn v9fs_do_readdir(V9fsPDU *pd= u, V9fsFidState *fidp, V9fsString name; int len, err =3D 0; int32_t count =3D 0; - off_t saved_dir_pos; struct dirent *dent; + struct stat *st; + struct V9fsDirEnt *entries =3D NULL; =20 - /* save the directory position */ - saved_dir_pos =3D v9fs_co_telldir(pdu, fidp); - if (saved_dir_pos < 0) { - return saved_dir_pos; - } - - while (1) { - v9fs_readdir_lock(&fidp->fs.dir); + /* + * inode remapping requires the device id, which in turn might be + * different for different directory entries, so if inode remapping is + * enabled we have to make a full stat for each directory entry + */ + const bool dostat =3D pdu->s->ctx.export_flags & V9FS_REMAP_INODES; =20 - err =3D v9fs_co_readdir(pdu, fidp, &dent); - if (err || !dent) { - break; - } - v9fs_string_init(&name); - v9fs_string_sprintf(&name, "%s", dent->d_name); - if ((count + v9fs_readdir_data_size(&name)) > max_count) { - v9fs_readdir_unlock(&fidp->fs.dir); + /* + * Fetch all required directory entries altogether on a background IO + * thread from fs driver. We don't want to do that for each entry + * individually, because hopping between threads (this main IO thread + * and background IO driver thread) would sum up to huge latencies. + */ + count =3D v9fs_co_readdir_lowlat(pdu, fidp, &entries, max_count, dosta= t); + if (count < 0) { + err =3D count; + count =3D 0; + goto out; + } + count =3D 0; =20 - /* Ran out of buffer. Set dir back to old position and return = */ - v9fs_co_seekdir(pdu, fidp, saved_dir_pos); - v9fs_string_free(&name); - return count; - } + for (struct V9fsDirEnt *e =3D entries; e; e =3D e->next) { + dent =3D e->dent; =20 if (pdu->s->ctx.export_flags & V9FS_REMAP_INODES) { - /* - * dirent_to_qid() implies expensive stat call for each entry, - * we must do that here though since inode remapping requires - * the device id, which in turn might be different for - * different entries; we cannot make any assumption to avoid - * that here. - */ - err =3D dirent_to_qid(pdu, fidp, dent, &qid); + st =3D e->st; + /* e->st should never be NULL, but just to be sure */ + if (!st) { + err =3D -1; + break; + } + + /* remap inode */ + err =3D stat_to_qid(pdu, st, &qid); if (err < 0) { - v9fs_readdir_unlock(&fidp->fs.dir); - v9fs_co_seekdir(pdu, fidp, saved_dir_pos); - v9fs_string_free(&name); - return err; + break; } } else { /* * Fill up just the path field of qid because the client uses * only that. To fill the entire qid structure we will have * to stat each dirent found, which is expensive. For the - * latter reason we don't call dirent_to_qid() here. Only draw= back + * latter reason we don't call stat_to_qid() here. Only drawba= ck * is that no multi-device export detection of stat_to_qid() * would be done and provided as error to the user here. But * user would get that error anyway when accessing those @@ -2379,25 +2366,26 @@ static int coroutine_fn v9fs_do_readdir(V9fsPDU *pd= u, V9fsFidState *fidp, qid.version =3D 0; } =20 + v9fs_string_init(&name); + v9fs_string_sprintf(&name, "%s", dent->d_name); + /* 11 =3D 7 + 4 (7 =3D start offset, 4 =3D space for storing count= ) */ len =3D pdu_marshal(pdu, 11 + count, "Qqbs", &qid, dent->d_off, dent->d_type, &name); =20 - v9fs_readdir_unlock(&fidp->fs.dir); + v9fs_string_free(&name); =20 if (len < 0) { - v9fs_co_seekdir(pdu, fidp, saved_dir_pos); - v9fs_string_free(&name); - return len; + err =3D len; + break; } + count +=3D len; - v9fs_string_free(&name); - saved_dir_pos =3D dent->d_off; } =20 - v9fs_readdir_unlock(&fidp->fs.dir); - +out: + v9fs_free_dirents(entries); if (err < 0) { return err; } diff --git a/hw/9pfs/9p.h b/hw/9pfs/9p.h index 3904f82901..38712dd16d 100644 --- a/hw/9pfs/9p.h +++ b/hw/9pfs/9p.h @@ -204,6 +204,28 @@ static inline void v9fs_readdir_init(V9fsDir *dir) qemu_mutex_init(&dir->readdir_mutex); } =20 +/* + * Type for 9p fs drivers' (a.k.a. 9p backends) result of readdir requests, + * which is a chained list of directory entries. + */ +typedef struct V9fsDirEnt { + /* mandatory (must not be NULL) information for all readdir requests */ + struct dirent *dent; + /* + * optional (may be NULL): A full stat of each directory entry is just + * done if explicitly told to fs driver. + */ + struct stat *st; + /* + * instead of an array, directory entries are always returned as + * chained list, that's because the amount of entries retrieved by fs + * drivers is dependent on the individual entries' name (since response + * messages are size limited), so the final amount cannot be estimated + * before hand + */ + struct V9fsDirEnt *next; +} V9fsDirEnt; + /* * Filled by fs driver on open and other * calls. @@ -408,6 +430,7 @@ void v9fs_path_init(V9fsPath *path); void v9fs_path_free(V9fsPath *path); void v9fs_path_sprintf(V9fsPath *path, const char *fmt, ...); void v9fs_path_copy(V9fsPath *dst, const V9fsPath *src); +size_t v9fs_readdir_response_size(V9fsString *name); int v9fs_name_to_path(V9fsState *s, V9fsPath *dirpath, const char *name, V9fsPath *path); int v9fs_device_realize_common(V9fsState *s, const V9fsTransport *t, diff --git a/hw/9pfs/codir.c b/hw/9pfs/codir.c index 73f9a751e1..bd36491510 100644 --- a/hw/9pfs/codir.c +++ b/hw/9pfs/codir.c @@ -18,28 +18,189 @@ #include "qemu/main-loop.h" #include "coth.h" =20 +/* + * This is solely executed on a background IO thread. + */ +static int do_readdir(V9fsPDU *pdu, V9fsFidState *fidp, struct dirent **de= nt) +{ + int err =3D 0; + V9fsState *s =3D pdu->s; + struct dirent *entry; + + errno =3D 0; + entry =3D s->ops->readdir(&s->ctx, &fidp->fs); + if (!entry && errno) { + *dent =3D NULL; + err =3D -errno; + } else { + *dent =3D entry; + } + return err; +} + +/* + * TODO: This will be removed for performance reasons. + * Use v9fs_co_readdir_lowlat() instead. + */ int coroutine_fn v9fs_co_readdir(V9fsPDU *pdu, V9fsFidState *fidp, struct dirent **dent) { int err; - V9fsState *s =3D pdu->s; =20 if (v9fs_request_cancelled(pdu)) { return -EINTR; } - v9fs_co_run_in_worker( - { - struct dirent *entry; + v9fs_co_run_in_worker({ + err =3D do_readdir(pdu, fidp, dent); + }); + return err; +} + +/* + * This is solely executed on a background IO thread. + * + * See v9fs_co_readdir_lowlat() (as its only user) below for details. + */ +static int do_readdir_lowlat(V9fsPDU *pdu, V9fsFidState *fidp, + struct V9fsDirEnt **entries, + int32_t maxsize, bool dostat) +{ + V9fsState *s =3D pdu->s; + V9fsString name; + int len, err =3D 0; + int32_t size =3D 0; + off_t saved_dir_pos; + struct dirent *dent; + struct V9fsDirEnt *e =3D NULL; + V9fsPath path; + struct stat stbuf; + + *entries =3D NULL; + v9fs_path_init(&path); + + /* + * TODO: Here should be an error_warn_once() if lock failed. + * + * With a good 9p client we should not get into concurrency here, + * because a good client would not use the same fid for concurrent + * requests. We do the lock here for safety reasons though. However + * the client would then suffer performance issues, so better log that + * issue here. + */ + v9fs_readdir_lock(&fidp->fs.dir); + + /* save the directory position */ + saved_dir_pos =3D s->ops->telldir(&s->ctx, &fidp->fs); + if (saved_dir_pos < 0) { + err =3D saved_dir_pos; + goto out; + } + + while (true) { + /* get directory entry from fs driver */ + err =3D do_readdir(pdu, fidp, &dent); + if (err || !dent) { + break; + } + + /* + * stop this loop as soon as it would exceed the allowed maximum + * response message size for the directory entries collected so fa= r, + * because anything beyond that size would need to be discarded by + * 9p controller (main thread / top half) anyway + */ + v9fs_string_init(&name); + v9fs_string_sprintf(&name, "%s", dent->d_name); + len =3D v9fs_readdir_response_size(&name); + v9fs_string_free(&name); + if (size + len > maxsize) { + /* this is not an error case actually */ + break; + } + + /* append next node to result chain */ + if (!e) { + *entries =3D e =3D g_malloc0(sizeof(V9fsDirEnt)); + } else { + e =3D e->next =3D g_malloc0(sizeof(V9fsDirEnt)); + } + e->dent =3D g_malloc0(sizeof(struct dirent)); + memcpy(e->dent, dent, sizeof(struct dirent)); =20 - errno =3D 0; - entry =3D s->ops->readdir(&s->ctx, &fidp->fs); - if (!entry && errno) { + /* perform a full stat() for directory entry if requested by calle= r */ + if (dostat) { + err =3D s->ops->name_to_path( + &s->ctx, &fidp->path, dent->d_name, &path + ); + if (err < 0) { err =3D -errno; - } else { - *dent =3D entry; - err =3D 0; + break; } - }); + + err =3D s->ops->lstat(&s->ctx, &path, &stbuf); + if (err < 0) { + err =3D -errno; + break; + } + + e->st =3D g_malloc0(sizeof(struct stat)); + memcpy(e->st, &stbuf, sizeof(struct stat)); + } + + size +=3D len; + saved_dir_pos =3D dent->d_off; + } + + /* restore (last) saved position */ + s->ops->seekdir(&s->ctx, &fidp->fs, saved_dir_pos); + +out: + v9fs_readdir_unlock(&fidp->fs.dir); + v9fs_path_free(&path); + if (err < 0) { + return err; + } + return size; +} + +/** + * @brief Low latency variant of fs driver readdir handling. + * + * Retrieves the requested (max. amount of) directory entries from the fs + * driver. This function must only be called by the main IO thread (top ha= lf). + * Internally this function call will be dispatched to a background IO thr= ead + * (bottom half) where it is eventually executed by the fs driver. + * + * The old readdir implementation above just retrieves always one dir entry + * per call. The problem of that implementation above is that latency is + * added for (retrieval of) each directory entry, which in practice lead to + * latencies of several hundred ms for readdir of only one directory. + * + * This is avoided in this function by letting the fs driver retrieve all + * required directory entries with only call of this function and hence wi= th + * only a single fs driver request. + * + * @param pdu - the causing 9p (T_readdir) client request + * @param fidp - already opened directory where readdir shall be performed= on + * @param entries - output for directory entries (must not be NULL) + * @param maxsize - maximum result message body size (in bytes) + * @param dostat - whether a stat() should be performed and returned for + * each directory entry + * @returns resulting response message body size (in bytes) on success, + * negative error code otherwise + */ +int coroutine_fn v9fs_co_readdir_lowlat(V9fsPDU *pdu, V9fsFidState *fidp, + struct V9fsDirEnt **entries, + int32_t maxsize, bool dostat) +{ + int err =3D 0; + + if (v9fs_request_cancelled(pdu)) { + return -EINTR; + } + v9fs_co_run_in_worker({ + err =3D do_readdir_lowlat(pdu, fidp, entries, maxsize, dostat); + }); return err; } =20 diff --git a/hw/9pfs/coth.h b/hw/9pfs/coth.h index c2cdc7a9ea..1249dbe6df 100644 --- a/hw/9pfs/coth.h +++ b/hw/9pfs/coth.h @@ -49,6 +49,9 @@ void co_run_in_worker_bh(void *); int coroutine_fn v9fs_co_readlink(V9fsPDU *, V9fsPath *, V9fsString *); int coroutine_fn v9fs_co_readdir(V9fsPDU *, V9fsFidState *, struct dirent = **); +int coroutine_fn v9fs_co_readdir_lowlat(V9fsPDU *, V9fsFidState *, + struct V9fsDirEnt **, + int32_t, bool); off_t coroutine_fn v9fs_co_telldir(V9fsPDU *, V9fsFidState *); void coroutine_fn v9fs_co_seekdir(V9fsPDU *, V9fsFidState *, off_t); void coroutine_fn v9fs_co_rewinddir(V9fsPDU *, V9fsFidState *); --=20 2.20.1 From nobody Sat May 4 05:22:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=crudebyte.com ARC-Seal: i=1; a=rsa-sha256; t=1576680545; cv=none; d=zohomail.com; s=zohoarc; b=EiMYAlZD18HuGGaOEmOQ2mS0rYzQMyY/aJAlDRDLmwWtrVNo5+SLym3OsrhJchmEHkOKZ9bGVNT4YzSGOCR7iZgGY4XaEBEzo1Od5aV0aLlM0y5TRgJZ6hIQuOAekB7+LIDUl9me/VbVH0zvCXzwOm8INbDz54IzOqIhGK9pnsI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1576680545; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To; bh=iK1ghDnL3doRVpTkI0ecqERy9JQWkEzcyIFDl/0Jn9g=; b=EuDNjiN46PBsASmCFVmIZkAn7ECih9ch7IfiQLtkCsaR17xpW3HtZRJTCuDKV32a3AxORXHITT8ts0KMaUWdSzUrr4uLWTEKJv7mOq/O8tEG8lmga3mTctQcwqfl4sUJXFVYHqsx4wOWcDL8zUy07iQiLbwPBy6l58rd3niaF6g= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1576680544712175.99646900186917; Wed, 18 Dec 2019 06:49:04 -0800 (PST) Received: from localhost ([::1]:55276 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ihadJ-0006UA-6l for importer@patchew.org; Wed, 18 Dec 2019 09:49:01 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:51383) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ihacS-00062h-TQ for qemu-devel@nongnu.org; Wed, 18 Dec 2019 09:48:10 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ihacQ-00012I-N2 for qemu-devel@nongnu.org; Wed, 18 Dec 2019 09:48:08 -0500 Received: from lizzy.crudebyte.com ([91.194.90.13]:42647) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ihacQ-0005m7-5p for qemu-devel@nongnu.org; Wed, 18 Dec 2019 09:48:06 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=crudebyte.com; s=lizzy; h=Subject:Date:Cc:To:From:References:In-Reply-To: Message-Id:Content-Type:Content-Transfer-Encoding:MIME-Version:Content-ID: Content-Description; bh=iK1ghDnL3doRVpTkI0ecqERy9JQWkEzcyIFDl/0Jn9g=; b=cOTW7 say2fBIeq/ANafmE3xAkYsFxCmSjI87oStQWt5vd/cYLcwZv26edeQ9Z3UHU3jjdOH6iXWru8M3oo 3sTZy1b+9V5GVJy4el8+KU8ZBPqNSQP7WetY3S8doOnoVvPyGuRHU76GOpfS04lP+EbSshI4aVdXc 8zPuYIQRvZAxjxOOoD/UQG0qVkOevl3mLnk1VicpcW4peXT8X7hCZ+iU52c6pjfPjT8lAeuEt2vC8 mQH1TGsyzsro5d0aBc320K5UnuZoJ22ZE7w9hurxF7wPKnQbuuEpIz2pTpLEbIjbq9jmzoN2+C3k2 Pgh0LZPdoW9lYoeM/qpKLgWRZkAAA==; Message-Id: In-Reply-To: References: From: Christian Schoenebeck To: qemu-devel@nongnu.org Cc: Greg Kurz Date: Wed, 18 Dec 2019 15:10:46 +0100 Subject: [PATCH v2 9/9] hw/9pfs/9p.c: benchmark time on T_readdir request X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 91.194.90.13 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @crudebyte.com) Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This patch is not intended to be merged, it measures and prints the time the 9p server spends on handling a T_readdir request. It prints the total time it spent on handling the request, and also the time it spent on I/O (fs driver) only. Signed-off-by: Christian Schoenebeck --- hw/9pfs/9p.c | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/hw/9pfs/9p.c b/hw/9pfs/9p.c index a7d36b6350..c6f70b96ee 100644 --- a/hw/9pfs/9p.c +++ b/hw/9pfs/9p.c @@ -2299,6 +2299,15 @@ static void v9fs_free_dirents(struct V9fsDirEnt *e) } } =20 +static double wall_time(void) +{ + struct timeval t; + struct timezone tz; + gettimeofday(&t, &tz); + return t.tv_sec + t.tv_usec * 0.000001; +} + + static int coroutine_fn v9fs_do_readdir(V9fsPDU *pdu, V9fsFidState *fidp, int32_t max_count) { @@ -2318,6 +2327,8 @@ static int coroutine_fn v9fs_do_readdir(V9fsPDU *pdu,= V9fsFidState *fidp, */ const bool dostat =3D pdu->s->ctx.export_flags & V9FS_REMAP_INODES; =20 + const double start =3D wall_time(); + /* * Fetch all required directory entries altogether on a background IO * thread from fs driver. We don't want to do that for each entry @@ -2332,6 +2343,10 @@ static int coroutine_fn v9fs_do_readdir(V9fsPDU *pdu= , V9fsFidState *fidp, } count =3D 0; =20 + const double end =3D wall_time(); + printf("\n\nTime 9p server spent on synth_readdir() I/O only (synth " + "driver): %fs\n", end - start); + for (struct V9fsDirEnt *e =3D entries; e; e =3D e->next) { dent =3D e->dent; =20 @@ -2404,6 +2419,8 @@ static void coroutine_fn v9fs_readdir(void *opaque) V9fsPDU *pdu =3D opaque; V9fsState *s =3D pdu->s; =20 + const double start =3D wall_time(); + retval =3D pdu_unmarshal(pdu, offset, "dqd", &fid, &initial_offset, &max_count); if (retval < 0) { @@ -2447,6 +2464,10 @@ out: put_fid(pdu, fidp); out_nofid: pdu_complete(pdu, retval); + + const double end =3D wall_time(); + printf("Time 9p server spent on entire T_readdir request: %fs " + "[IMPORTANT]\n", end - start); } =20 static int v9fs_xattr_write(V9fsState *s, V9fsPDU *pdu, V9fsFidState *fidp, --=20 2.20.1