From nobody Sun Apr 19 04:00:11 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 52C46C433EF for ; Sat, 9 Jul 2022 02:28:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229645AbiGIC2S (ORCPT ); Fri, 8 Jul 2022 22:28:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50972 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229631AbiGIC2Q (ORCPT ); Fri, 8 Jul 2022 22:28:16 -0400 Received: from smtp236.sjtu.edu.cn (smtp236.sjtu.edu.cn [202.120.2.236]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D58D479EE8 for ; Fri, 8 Jul 2022 19:28:14 -0700 (PDT) Received: from proxy02.sjtu.edu.cn (smtp188.sjtu.edu.cn [202.120.2.188]) by smtp236.sjtu.edu.cn (Postfix) with ESMTPS id 9A2941008B388; Sat, 9 Jul 2022 10:28:05 +0800 (CST) Received: from localhost (localhost.localdomain [127.0.0.1]) by proxy02.sjtu.edu.cn (Postfix) with ESMTP id C61E2200A5E62; Sat, 9 Jul 2022 10:28:03 +0800 (CST) X-Virus-Scanned: amavisd-new at Received: from proxy02.sjtu.edu.cn ([127.0.0.1]) by localhost (proxy02.sjtu.edu.cn [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id jtiPptT7xq7X; Sat, 9 Jul 2022 10:28:03 +0800 (CST) Received: from localhost.localdomain (unknown [202.120.40.82]) (Authenticated sender: qtxuning1999@sjtu.edu.cn) by proxy02.sjtu.edu.cn (Postfix) with ESMTPSA id 3A3F72009BEAF; Sat, 9 Jul 2022 10:27:54 +0800 (CST) From: Guo Zhi To: jasowang@redhat.com, mst@redhat.com Cc: eperezma@redhat.com, virtualization@lists.linux-foundation.org, linux-kernel@vger.kernel.org, sgarzare@redhat.com, Guo Zhi Subject: [PATCH v3 1/4] virtio_test: kick vhost for a batch of descriptors Date: Sat, 9 Jul 2022 10:27:42 +0800 Message-Id: <20220709022745.21020-2-qtxuning1999@sjtu.edu.cn> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20220709022745.21020-1-qtxuning1999@sjtu.edu.cn> References: <20220709022745.21020-1-qtxuning1999@sjtu.edu.cn> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Only kick vhost when the batch finishes. Signed-off-by: Guo Zhi --- tools/virtio/virtio_test.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/tools/virtio/virtio_test.c b/tools/virtio/virtio_test.c index 23f142af5..95f78b311 100644 --- a/tools/virtio/virtio_test.c +++ b/tools/virtio/virtio_test.c @@ -208,11 +208,10 @@ static void run_test(struct vdev_info *dev, struct vq= _info *vq, } =20 ++started; - - if (unlikely(!virtqueue_kick(vq->vq))) { - r =3D -1; - break; - } + } + if (unlikely(!virtqueue_kick(vq->vq))) { + r =3D -1; + break; } =20 if (started >=3D bufs) --=20 2.17.1 From nobody Sun Apr 19 04:00:11 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7A7B6C433EF for ; Sat, 9 Jul 2022 02:28:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229631AbiGIC23 (ORCPT ); Fri, 8 Jul 2022 22:28:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51376 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229651AbiGIC21 (ORCPT ); Fri, 8 Jul 2022 22:28:27 -0400 Received: from smtp236.sjtu.edu.cn (smtp236.sjtu.edu.cn [202.120.2.236]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A5E3C255 for ; Fri, 8 Jul 2022 19:28:26 -0700 (PDT) Received: from proxy02.sjtu.edu.cn (smtp188.sjtu.edu.cn [202.120.2.188]) by smtp236.sjtu.edu.cn (Postfix) with ESMTPS id 40C141008B389; Sat, 9 Jul 2022 10:28:16 +0800 (CST) Received: from localhost (localhost.localdomain [127.0.0.1]) by proxy02.sjtu.edu.cn (Postfix) with ESMTP id 79B272009BEC5; Sat, 9 Jul 2022 10:28:10 +0800 (CST) X-Virus-Scanned: amavisd-new at Received: from proxy02.sjtu.edu.cn ([127.0.0.1]) by localhost (proxy02.sjtu.edu.cn [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 9XYm2DB9fy6l; Sat, 9 Jul 2022 10:28:10 +0800 (CST) Received: from localhost.localdomain (unknown [202.120.40.82]) (Authenticated sender: qtxuning1999@sjtu.edu.cn) by proxy02.sjtu.edu.cn (Postfix) with ESMTPSA id B88C5200C91EC; Sat, 9 Jul 2022 10:28:03 +0800 (CST) From: Guo Zhi To: jasowang@redhat.com, mst@redhat.com Cc: eperezma@redhat.com, virtualization@lists.linux-foundation.org, linux-kernel@vger.kernel.org, sgarzare@redhat.com, Guo Zhi Subject: [PATCH v3 2/4] virtio_test: use random length scatterlists to test descriptor chain Date: Sat, 9 Jul 2022 10:27:43 +0800 Message-Id: <20220709022745.21020-3-qtxuning1999@sjtu.edu.cn> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20220709022745.21020-1-qtxuning1999@sjtu.edu.cn> References: <20220709022745.21020-1-qtxuning1999@sjtu.edu.cn> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Prior implementation only use one descriptor for each io event, which does't test code of descriptor chain. More importantly, one descriptor will not use indirect feature even indirect feature is specified. Use random length scatterlists here to test descriptor chain. Signed-off-by: Guo Zhi --- v3: - drop fda270fcd virtio_test: move magic number in code as defined constant --- tools/virtio/virtio_test.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/tools/virtio/virtio_test.c b/tools/virtio/virtio_test.c index 95f78b311..1408a4a20 100644 --- a/tools/virtio/virtio_test.c +++ b/tools/virtio/virtio_test.c @@ -20,6 +20,7 @@ #include "../../drivers/vhost/test.h" =20 #define RANDOM_BATCH -1 +#define MAX_SG_FRAGS 8UL =20 /* Unused */ void *__kmalloc_fake, *__kfree_ignore_start, *__kfree_ignore_end; @@ -169,7 +170,8 @@ static void wait_for_interrupt(struct vdev_info *dev) static void run_test(struct vdev_info *dev, struct vq_info *vq, bool delayed, int batch, int reset_n, int bufs) { - struct scatterlist sl; + struct scatterlist sg[MAX_SG_FRAGS]; + int sg_size =3D 0; long started =3D 0, completed =3D 0, next_reset =3D reset_n; long completed_before, started_before; int r, test =3D 1; @@ -194,8 +196,11 @@ static void run_test(struct vdev_info *dev, struct vq_= info *vq, =20 while (started < bufs && (started - completed) < batch) { - sg_init_one(&sl, dev->buf, dev->buf_size); - r =3D virtqueue_add_outbuf(vq->vq, &sl, 1, + sg_size =3D random() % (MAX_SG_FRAGS - 1) + 1; + sg_init_table(sg, sg_size); + for (int i =3D 0; i < sg_size; ++i) + sg_set_buf(&sg[i], dev->buf + i, 0x1); + r =3D virtqueue_add_outbuf(vq->vq, sg, sg_size, dev->buf + started, GFP_ATOMIC); if (unlikely(r !=3D 0)) { --=20 2.17.1 From nobody Sun Apr 19 04:00:11 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A38F9C433EF for ; Sat, 9 Jul 2022 02:28:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229544AbiGIC2q (ORCPT ); Fri, 8 Jul 2022 22:28:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51744 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229659AbiGIC2k (ORCPT ); Fri, 8 Jul 2022 22:28:40 -0400 Received: from smtp236.sjtu.edu.cn (smtp236.sjtu.edu.cn [202.120.2.236]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6FA1818E0B for ; Fri, 8 Jul 2022 19:28:38 -0700 (PDT) Received: from proxy02.sjtu.edu.cn (smtp188.sjtu.edu.cn [202.120.2.188]) by smtp236.sjtu.edu.cn (Postfix) with ESMTPS id B1AC51008B388; Sat, 9 Jul 2022 10:28:23 +0800 (CST) Received: from localhost (localhost.localdomain [127.0.0.1]) by proxy02.sjtu.edu.cn (Postfix) with ESMTP id 6E1A32009BEB2; Sat, 9 Jul 2022 10:28:18 +0800 (CST) X-Virus-Scanned: amavisd-new at Received: from proxy02.sjtu.edu.cn ([127.0.0.1]) by localhost (proxy02.sjtu.edu.cn [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id nVtFykjv01sq; Sat, 9 Jul 2022 10:28:18 +0800 (CST) Received: from localhost.localdomain (unknown [202.120.40.82]) (Authenticated sender: qtxuning1999@sjtu.edu.cn) by proxy02.sjtu.edu.cn (Postfix) with ESMTPSA id 415902009BEAF; Sat, 9 Jul 2022 10:28:10 +0800 (CST) From: Guo Zhi To: jasowang@redhat.com, mst@redhat.com Cc: eperezma@redhat.com, virtualization@lists.linux-foundation.org, linux-kernel@vger.kernel.org, sgarzare@redhat.com, Guo Zhi Subject: [PATCH v3 3/4] virtio_test: enable indirection feature Date: Sat, 9 Jul 2022 10:27:44 +0800 Message-Id: <20220709022745.21020-4-qtxuning1999@sjtu.edu.cn> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20220709022745.21020-1-qtxuning1999@sjtu.edu.cn> References: <20220709022745.21020-1-qtxuning1999@sjtu.edu.cn> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Prior implementation don't use indirection feature because there is only one descriptor for every io event, actually prior implementation don't support indirection because vhost can't translate and find the indirect descriptors. This commit enable virtio_test malloc indirect descriptors in a indirect buffer and map this buffer to vhost, thus resolve this problem. Signed-off-by: Guo Zhi --- v3: - use C style comment - changed INDIRECT_TABLE name - add comment for __kmalloc_fake --- tools/virtio/virtio_test.c | 56 ++++++++++++++++++++++++++++++++------ 1 file changed, 47 insertions(+), 9 deletions(-) diff --git a/tools/virtio/virtio_test.c b/tools/virtio/virtio_test.c index 1408a4a20..8f3ef3a78 100644 --- a/tools/virtio/virtio_test.c +++ b/tools/virtio/virtio_test.c @@ -21,8 +21,9 @@ =20 #define RANDOM_BATCH -1 #define MAX_SG_FRAGS 8UL +#define RINGSIZE 256 +#define INDIRECT_TABLE_SIZE (RINGSIZE * sizeof(struct vring_desc) * 8) =20 -/* Unused */ void *__kmalloc_fake, *__kfree_ignore_start, *__kfree_ignore_end; =20 struct vq_info { @@ -44,6 +45,8 @@ struct vdev_info { int nvqs; void *buf; size_t buf_size; + void *indirect_table; + size_t indirect_table_size; struct vhost_memory *mem; }; =20 @@ -128,6 +131,8 @@ static void vq_info_add(struct vdev_info *dev, int num) static void vdev_info_init(struct vdev_info* dev, unsigned long long featu= res) { int r; + int nregions =3D 2; + memset(dev, 0, sizeof *dev); dev->vdev.features =3D features; INIT_LIST_HEAD(&dev->vdev.vqs); @@ -135,19 +140,25 @@ static void vdev_info_init(struct vdev_info* dev, uns= igned long long features) dev->buf_size =3D 1024; dev->buf =3D malloc(dev->buf_size); assert(dev->buf); - dev->control =3D open("/dev/vhost-test", O_RDWR); + dev->indirect_table_size =3D INDIRECT_TABLE_SIZE; + dev->indirect_table =3D malloc(dev->indirect_table_size); + assert(dev->indirect_table); + dev->control =3D open("/dev/vhost-test", O_RDWR); assert(dev->control >=3D 0); r =3D ioctl(dev->control, VHOST_SET_OWNER, NULL); assert(r >=3D 0); dev->mem =3D malloc(offsetof(struct vhost_memory, regions) + - sizeof dev->mem->regions[0]); + (sizeof(dev->mem->regions[0])) * nregions); assert(dev->mem); memset(dev->mem, 0, offsetof(struct vhost_memory, regions) + - sizeof dev->mem->regions[0]); - dev->mem->nregions =3D 1; + (sizeof(dev->mem->regions[0])) * nregions); + dev->mem->nregions =3D nregions; dev->mem->regions[0].guest_phys_addr =3D (long)dev->buf; dev->mem->regions[0].userspace_addr =3D (long)dev->buf; dev->mem->regions[0].memory_size =3D dev->buf_size; + dev->mem->regions[1].guest_phys_addr =3D (long)dev->indirect_table; + dev->mem->regions[1].userspace_addr =3D (long)dev->indirect_table; + dev->mem->regions[1].memory_size =3D dev->indirect_table_size; r =3D ioctl(dev->control, VHOST_SET_MEM_TABLE, dev->mem); assert(r >=3D 0); } @@ -167,6 +178,22 @@ static void wait_for_interrupt(struct vdev_info *dev) } } =20 +static int test_virtqueue_add_outbuf(struct virtqueue *vq, + struct scatterlist *sg, unsigned int num, + void *data, void *indirect_table) +{ + int r; + + /* Allocate an indirect, force it to allocate user addr + * exists in vhost iotlb, otherwise vhost can't access + */ + __kmalloc_fake =3D indirect_table; + r =3D virtqueue_add_outbuf(vq, sg, num, data, + GFP_ATOMIC); + __kmalloc_fake =3D NULL; + return r; +} + static void run_test(struct vdev_info *dev, struct vq_info *vq, bool delayed, int batch, int reset_n, int bufs) { @@ -178,6 +205,7 @@ static void run_test(struct vdev_info *dev, struct vq_i= nfo *vq, unsigned len; long long spurious =3D 0; const bool random_batch =3D batch =3D=3D RANDOM_BATCH; + void *indirect_table; =20 r =3D ioctl(dev->control, VHOST_TEST_RUN, &test); assert(r >=3D 0); @@ -185,10 +213,15 @@ static void run_test(struct vdev_info *dev, struct vq= _info *vq, next_reset =3D INT_MAX; } =20 + /* Don't kfree indirect_table. */ + __kfree_ignore_start =3D dev->indirect_table; + __kfree_ignore_end =3D dev->indirect_table + dev->indirect_table_size; + for (;;) { virtqueue_disable_cb(vq->vq); completed_before =3D completed; started_before =3D started; + indirect_table =3D dev->indirect_table; do { const bool reset =3D completed > next_reset; if (random_batch) @@ -200,9 +233,13 @@ static void run_test(struct vdev_info *dev, struct vq_= info *vq, sg_init_table(sg, sg_size); for (int i =3D 0; i < sg_size; ++i) sg_set_buf(&sg[i], dev->buf + i, 0x1); - r =3D virtqueue_add_outbuf(vq->vq, sg, sg_size, - dev->buf + started, - GFP_ATOMIC); + + /* use indirect_table buffer repeatedly */ + if (indirect_table + sg_size * sizeof(struct vring_desc) > + dev->indirect_table + dev->indirect_table_size) + indirect_table =3D dev->indirect_table; + r =3D test_virtqueue_add_outbuf(vq->vq, sg, sg_size, + dev->buf + started, indirect_table); if (unlikely(r !=3D 0)) { if (r =3D=3D -ENOSPC && started > started_before) @@ -213,6 +250,7 @@ static void run_test(struct vdev_info *dev, struct vq_i= nfo *vq, } =20 ++started; + indirect_table +=3D sg_size * sizeof(struct vring_desc); } if (unlikely(!virtqueue_kick(vq->vq))) { r =3D -1; @@ -401,7 +439,7 @@ int main(int argc, char **argv) =20 done: vdev_info_init(&dev, features); - vq_info_add(&dev, 256); + vq_info_add(&dev, RINGSIZE); run_test(&dev, &dev.vqs[0], delayed, batch, reset, 0x100000); return 0; } --=20 2.17.1 From nobody Sun Apr 19 04:00:11 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA679C43334 for ; Sat, 9 Jul 2022 02:28:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229681AbiGIC2y (ORCPT ); Fri, 8 Jul 2022 22:28:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52070 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229678AbiGIC2x (ORCPT ); Fri, 8 Jul 2022 22:28:53 -0400 Received: from smtp236.sjtu.edu.cn (smtp236.sjtu.edu.cn [202.120.2.236]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E8025DF3E for ; Fri, 8 Jul 2022 19:28:51 -0700 (PDT) Received: from proxy02.sjtu.edu.cn (smtp188.sjtu.edu.cn [202.120.2.188]) by smtp236.sjtu.edu.cn (Postfix) with ESMTPS id 871231008B38A; Sat, 9 Jul 2022 10:28:27 +0800 (CST) Received: from localhost (localhost.localdomain [127.0.0.1]) by proxy02.sjtu.edu.cn (Postfix) with ESMTP id 725FE2009BEA0; Sat, 9 Jul 2022 10:28:27 +0800 (CST) X-Virus-Scanned: amavisd-new at Received: from proxy02.sjtu.edu.cn ([127.0.0.1]) by localhost (proxy02.sjtu.edu.cn [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id sZRFSb9s3H5X; Sat, 9 Jul 2022 10:28:27 +0800 (CST) Received: from localhost.localdomain (unknown [202.120.40.82]) (Authenticated sender: qtxuning1999@sjtu.edu.cn) by proxy02.sjtu.edu.cn (Postfix) with ESMTPSA id 73ED92009BEAF; Sat, 9 Jul 2022 10:28:18 +0800 (CST) From: Guo Zhi To: jasowang@redhat.com, mst@redhat.com Cc: eperezma@redhat.com, virtualization@lists.linux-foundation.org, linux-kernel@vger.kernel.org, sgarzare@redhat.com, Guo Zhi Subject: [PATCH v3 4/4] virtio_test: pregenerate random numbers Date: Sat, 9 Jul 2022 10:27:45 +0800 Message-Id: <20220709022745.21020-5-qtxuning1999@sjtu.edu.cn> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20220709022745.21020-1-qtxuning1999@sjtu.edu.cn> References: <20220709022745.21020-1-qtxuning1999@sjtu.edu.cn> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" random numbers on data path is expensive, using a rand pool to pregenerate them before tests start Signed-off-by: Guo Zhi --- tools/virtio/virtio_test.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/tools/virtio/virtio_test.c b/tools/virtio/virtio_test.c index 8f3ef3a78..7cc43e5bf 100644 --- a/tools/virtio/virtio_test.c +++ b/tools/virtio/virtio_test.c @@ -23,6 +23,7 @@ #define MAX_SG_FRAGS 8UL #define RINGSIZE 256 #define INDIRECT_TABLE_SIZE (RINGSIZE * sizeof(struct vring_desc) * 8) +#define RAND_POOL_SIZE 4096 =20 void *__kmalloc_fake, *__kfree_ignore_start, *__kfree_ignore_end; =20 @@ -199,6 +200,8 @@ static void run_test(struct vdev_info *dev, struct vq_i= nfo *vq, { struct scatterlist sg[MAX_SG_FRAGS]; int sg_size =3D 0; + int rand_pool[RAND_POOL_SIZE]; + unsigned int rand_idx; long started =3D 0, completed =3D 0, next_reset =3D reset_n; long completed_before, started_before; int r, test =3D 1; @@ -213,6 +216,10 @@ static void run_test(struct vdev_info *dev, struct vq_= info *vq, next_reset =3D INT_MAX; } =20 + /* Init rand pool */ + for (rand_idx =3D 0; rand_idx < RAND_POOL_SIZE; ++rand_idx) + rand_pool[rand_idx] =3D random(); + /* Don't kfree indirect_table. */ __kfree_ignore_start =3D dev->indirect_table; __kfree_ignore_end =3D dev->indirect_table + dev->indirect_table_size; @@ -225,11 +232,13 @@ static void run_test(struct vdev_info *dev, struct vq= _info *vq, do { const bool reset =3D completed > next_reset; if (random_batch) - batch =3D (random() % vq->vring.num) + 1; + batch =3D (rand_pool[rand_idx++ % RAND_POOL_SIZE] + % vq->vring.num) + 1; =20 while (started < bufs && (started - completed) < batch) { - sg_size =3D random() % (MAX_SG_FRAGS - 1) + 1; + sg_size =3D (rand_pool[rand_idx++ % RAND_POOL_SIZE] + % (MAX_SG_FRAGS - 1)) + 1; sg_init_table(sg, sg_size); for (int i =3D 0; i < sg_size; ++i) sg_set_buf(&sg[i], dev->buf + i, 0x1); --=20 2.17.1