From nobody Thu May 2 22:43:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; dkim=fail spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1493280576623363.71015657763417; Thu, 27 Apr 2017 01:09:36 -0700 (PDT) Received: from localhost ([::1]:59301 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d3eUX-00068G-Hw for importer@patchew.org; Thu, 27 Apr 2017 04:09:33 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56341) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d3eRI-0003Fr-Qr for qemu-devel@nongnu.org; Thu, 27 Apr 2017 04:06:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d3eRC-00085I-UK for qemu-devel@nongnu.org; Thu, 27 Apr 2017 04:06:12 -0400 Received: from mail-pf0-x243.google.com ([2607:f8b0:400e:c00::243]:35547) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1d3eRC-00083k-OZ; Thu, 27 Apr 2017 04:06:06 -0400 Received: by mail-pf0-x243.google.com with SMTP id a188so7471426pfa.2; Thu, 27 Apr 2017 01:06:06 -0700 (PDT) Received: from linux.local ([157.51.104.251]) by smtp.gmail.com with ESMTPSA id 12sm3295277pgb.35.2017.04.27.01.06.01 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 27 Apr 2017 01:06:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=IlnxF6kfjqlPCG3CQGlgRbUb7F2oQkuehDnoRyUmjOE=; b=BS/zyxSJWer263ti18NVrujk66OzNrswmeckyYsA3supPSvfRlvdZshFDkmgdEJf1y KGqK80qbsG5WuzV/0p91+0g5TSA19s3uulWFC3rZy51AP5YtQsnQdVpCTos8J+isb5oH fiSQOlKy+RSY2A+aGH+IykUeR7wd44hRRUun7doZiowuTdqGHtd6Jatay9pVdGVVtZbJ daMRrLfE7/p/B7kmAbwnBuvYTnrUjZUUa7KV2LnGogIV4AQK9kD0uSpLVZkQbqmRvWBE KLa+EaDe0WNIb/jLxnfFUO1E0YEnhOvrhV5HSfBJTCZ4x7xxmVApXvASay6vz+fUNOw1 92Xg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=IlnxF6kfjqlPCG3CQGlgRbUb7F2oQkuehDnoRyUmjOE=; b=MGeHrJbMdNtu+O1w7wKKNkzOS1Wck9GIMegmBrFK49pBDwIyaNm4cI/jRsSM1Ux1MZ MymtL63iKxaIpXYOKXDK8XLyEiS0sCvd5ip/QmgmuPrRuNknN9zHrdpF4chUMd2Qah+Y jVbrGKd0p9RjI2puoMqdJ5UPhk9k0y/yenrf7cYW4WthwlEkOb4gqLa04VMvfPnfAODO MwKb7ZSAZxvCRyDnOzRFYRW2SbXqxggSZrTSqL1m67PXMafClDz+qs8P3KvBfsmDELUJ dJ05FbufM8rhYwI8JMBNFIY+FuV0NkXisEBt1runJ/UPFlbpdLSqaYHbE+/lni1olC8F 2pHQ== X-Gm-Message-State: AN3rC/7LDO05R8UJ78eM/361xwkHuXIIKFuvQnXxJurALlc4K/j1TgMS OrlThUSaj/y0BA== X-Received: by 10.84.194.37 with SMTP id g34mr5532975pld.182.1493280365984; Thu, 27 Apr 2017 01:06:05 -0700 (PDT) From: Ashijeet Acharya To: stefanha@gmail.com Date: Thu, 27 Apr 2017 13:36:31 +0530 Message-Id: <1493280397-9622-2-git-send-email-ashijeetacharya@gmail.com> X-Mailer: git-send-email 2.6.2 In-Reply-To: <1493280397-9622-1-git-send-email-ashijeetacharya@gmail.com> References: <1493280397-9622-1-git-send-email-ashijeetacharya@gmail.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c00::243 Subject: [Qemu-devel] [PATCH v2 1/7] dmg: Introduce a new struct to cache random access points X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, famz@redhat.com, qemu-block@nongnu.org, qemu-devel@nongnu.org, mreitz@redhat.com, peter@lekensteyn.nl, Ashijeet Acharya , jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We need to cache the random access point while performing partial decompression so that we can resume decompression from that point onwards in our next sequential read request. Introduce a new struct DMGReadState which will help us do this. Signed-off-by: Ashijeet Acharya --- block/dmg.h | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/block/dmg.h b/block/dmg.h index b592d6f..ee67ae1 100644 --- a/block/dmg.h +++ b/block/dmg.h @@ -31,6 +31,15 @@ #include "block/block_int.h" #include =20 +/* used to cache current position in compressed input stream */ +typedef struct DMGReadState { + uint8_t *saved_next_in; + int64_t saved_avail_in; + int32_t saved_chunk_type; + int64_t sectors_read; /* possible sectors read in each cycle */ + int32_t sector_offset_in_chunk; +} DMGReadState; + typedef struct BDRVDMGState { CoMutex lock; /* each chunk contains a certain number of sectors, @@ -51,6 +60,7 @@ typedef struct BDRVDMGState { uint8_t *compressed_chunk; uint8_t *uncompressed_chunk; z_stream zstream; + DMGReadState *drs; } BDRVDMGState; =20 extern int (*dmg_uncompress_bz2)(char *next_in, unsigned int avail_in, --=20 2.6.2 From nobody Thu May 2 22:43:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; dkim=fail spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1493280722983629.7948371356631; Thu, 27 Apr 2017 01:12:02 -0700 (PDT) Received: from localhost ([::1]:59321 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d3eWv-0008IX-LC for importer@patchew.org; Thu, 27 Apr 2017 04:12:01 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56345) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d3eRJ-0003G3-2G for qemu-devel@nongnu.org; Thu, 27 Apr 2017 04:06:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d3eRI-0008Cz-2n for qemu-devel@nongnu.org; Thu, 27 Apr 2017 04:06:13 -0400 Received: from mail-pf0-x242.google.com ([2607:f8b0:400e:c00::242]:36571) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1d3eRH-0008BX-Sp; Thu, 27 Apr 2017 04:06:12 -0400 Received: by mail-pf0-x242.google.com with SMTP id v14so7445428pfd.3; Thu, 27 Apr 2017 01:06:11 -0700 (PDT) Received: from linux.local ([157.51.104.251]) by smtp.gmail.com with ESMTPSA id 12sm3295277pgb.35.2017.04.27.01.06.07 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 27 Apr 2017 01:06:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=vKRWlpiWK/pGAzUMPnNXBO77u7wg9oKIRckfGdnlhB4=; b=VeDYM9JDkdmFavM5IjIVZ0y2nWWHjMlNf9VVKO3Gf4vjIVxaLi1Iq+k0D0M8/f2F3p lCbze536AglLXn33kaWUTO9UtW5evJY70rAommcUbA9Eb7wizEOJ45j7TYcqTTtdZ7sU m8OIeDdIecdCDAjGf1oAF3rSAxtRlrsoWt16uTVZ7dLXG7xvuLv6DScwJ/oDjN8jHxT+ 0Cqp6vCREx99L5hWNEbF9RjWWdr6UjNc2YTuJOwNm4fsiaS4RXJDOTmcbthnXhEnxsEB TiONiNDpW/F5kQZ5n4kwvutadtbJIf0yB4UVE7i7XQB8WhGIyWwc9cdKMgBCDz3nIS7p TOgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=vKRWlpiWK/pGAzUMPnNXBO77u7wg9oKIRckfGdnlhB4=; b=FD5Jd0WY3qyG35sr0L+iT/vR4wqkBM1GeGxePwSN9VvTYmNDMk2iGI9HqXIdh469KI 1ZfWPSz/afA2sRYitvylH4+fid9zd8y1Nz/eOF6RP3O2alqwCfBtzYg5HDf3GwtZncd3 EQSc54Hj16CINmLeNiviQjtKEcYIHDPJiDXjY86YRqcTvpmt/r0Z+VWues6TZaDJg8AZ vQqLHSEgKgnLuSBeIMJG7s4LsS3JRzcLPrZtcV4ij8LSU6KeENuv/bbAzAvBJG5uUMbq 3FuK2vU7fwR/usjGE3RKqvm97Whq1J3qZrdmi6KZCiuub/HXr6gXrE93Q3ichODVVcIY XVqA== X-Gm-Message-State: AN3rC/4v1H6kmRDOW1THRH7DSrWeq1tt4ibL9to5z2GPHc38krcEBKFI 6OSjAYMT8edjXA== X-Received: by 10.84.247.9 with SMTP id n9mr5303447pll.119.1493280371074; Thu, 27 Apr 2017 01:06:11 -0700 (PDT) From: Ashijeet Acharya To: stefanha@gmail.com Date: Thu, 27 Apr 2017 13:36:32 +0530 Message-Id: <1493280397-9622-3-git-send-email-ashijeetacharya@gmail.com> X-Mailer: git-send-email 2.6.2 In-Reply-To: <1493280397-9622-1-git-send-email-ashijeetacharya@gmail.com> References: <1493280397-9622-1-git-send-email-ashijeetacharya@gmail.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c00::242 Subject: [Qemu-devel] [PATCH v2 2/7] dmg: New function to help us cache random access point X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, famz@redhat.com, qemu-block@nongnu.org, qemu-devel@nongnu.org, mreitz@redhat.com, peter@lekensteyn.nl, Ashijeet Acharya , jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Introduce a new cache_access_point() function which will help us first cache a random access point inside a compressed stream and then keep updating it according to our requirement at appropriate times. Signed-off-by: Ashijeet Acharya --- block/dmg.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/block/dmg.c b/block/dmg.c index a7d25fc..c6fe8b0 100644 --- a/block/dmg.c +++ b/block/dmg.c @@ -128,6 +128,18 @@ static void update_max_chunk_size(BDRVDMGState *s, uin= t32_t chunk, } } =20 +static void cache_access_point(DMGReadState *drs, uint8_t *next_in, + int64_t avail_in, int32_t chunk, + int64_t sectors_read, int32_t sector_offse= t) +{ + drs->saved_next_in =3D next_in; + drs->saved_avail_in =3D avail_in; + drs->saved_chunk_type =3D chunk; + drs->sectors_read =3D sectors_read; + drs->sector_offset_in_chunk =3D sector_offset; + return; +} + static int64_t dmg_find_koly_offset(BdrvChild *file, Error **errp) { BlockDriverState *file_bs =3D file->bs; @@ -507,6 +519,10 @@ static int dmg_open(BlockDriverState *bs, QDict *optio= ns, int flags, goto fail; } =20 + s->drs =3D g_malloc(sizeof(DMGReadState)); + /* initialise our access point cache */ + cache_access_point(s->drs, NULL, -1, -1, -1, -1); + if (inflateInit(&s->zstream) !=3D Z_OK) { ret =3D -EINVAL; goto fail; @@ -523,6 +539,7 @@ fail: g_free(s->lengths); g_free(s->sectors); g_free(s->sectorcounts); + g_free(s->drs); qemu_vfree(s->compressed_chunk); qemu_vfree(s->uncompressed_chunk); return ret; @@ -685,6 +702,7 @@ static void dmg_close(BlockDriverState *bs) g_free(s->lengths); g_free(s->sectors); g_free(s->sectorcounts); + g_free(s->drs); qemu_vfree(s->compressed_chunk); qemu_vfree(s->uncompressed_chunk); =20 --=20 2.6.2 From nobody Thu May 2 22:43:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; dkim=fail spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1493280785663309.39364471931435; Thu, 27 Apr 2017 01:13:05 -0700 (PDT) Received: from localhost ([::1]:59324 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d3eXw-0000oW-4H for importer@patchew.org; Thu, 27 Apr 2017 04:13:04 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56430) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d3eRP-0003Kb-1S for qemu-devel@nongnu.org; Thu, 27 Apr 2017 04:06:20 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d3eRN-0008Ih-Bd for qemu-devel@nongnu.org; Thu, 27 Apr 2017 04:06:19 -0400 Received: from mail-pf0-x242.google.com ([2607:f8b0:400e:c00::242]:36596) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1d3eRN-0008IH-3i; Thu, 27 Apr 2017 04:06:17 -0400 Received: by mail-pf0-x242.google.com with SMTP id v14so7446110pfd.3; Thu, 27 Apr 2017 01:06:17 -0700 (PDT) Received: from linux.local ([157.51.104.251]) by smtp.gmail.com with ESMTPSA id 12sm3295277pgb.35.2017.04.27.01.06.12 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 27 Apr 2017 01:06:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=ygC6I0sIqx8FBJtUh03LV3FpDHCP+pBT7gRt6bQYFhw=; b=WZZXaH8GK//bHF8cgFBJKURiTpH/RWH7nwwsgwEUlxz+eTx+2IU2XXspGNcQvzAUXP MTiYu2w0JH68x+QcE4Nnp/5vf9KIzVxwLz6Rk49UdrXA/iY9x+7Xny9T7ozNcxCv7Oun nXV60sEq/qG2jX4/HrusjLd3Kzacqf78/OLYu3a7Yc0JtwwEF18U9ENN0zarfrDuPF5s n5ZfWDqBPAt3l74fEAgTMe8JgShUC7MQF0AjKd7Zha4EjiAu4/wbmWWR6aZM9bdDxjWT mbxRkOn516mh8PtsUHGKRPd6gD37d5mt2vMRea9ONZgfAQf6m9jaKA67TOe4iA9sv3Oz xApg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=ygC6I0sIqx8FBJtUh03LV3FpDHCP+pBT7gRt6bQYFhw=; b=NU8QCu+KROLQDQ8HAkNY2EtHHjuyUyBwAk+A9/lIo2l/ZAyQg/XVdi9JcGkEi9HqZu PHsShmbnRlGypqCW1uXKXH+A451NEGfg919Z/W2Rg5gnlBLEjg6YVgNZLJRfC6wtmpjb XnlpuGT44CehVacJOpXCbAbTaWdGKT/xgBQ3V1aNZPotc6sto1Rp3/CfgsKnroAmOmnx PBje7BLZ7bSPuYm7cozDlMvAEc9imd70xTR+Xhk6d+f3xMmtwE5mXiZvrPJ59NNjubmm 5Xq8ivgsHa9CbT0DPCy4wK2vedAzC4tLvY1w/KmsX9vYRUfWN6anQ+27XrXvGAI7KSV1 cCyw== X-Gm-Message-State: AN3rC/5oKRT8qt27njZhEWBt+oIVAKkncB17ibwxT+5X3g4UmJ7xx1BJ TBfRvh7XR4+p0QFD X-Received: by 10.84.214.22 with SMTP id h22mr5458846pli.96.1493280376243; Thu, 27 Apr 2017 01:06:16 -0700 (PDT) From: Ashijeet Acharya To: stefanha@gmail.com Date: Thu, 27 Apr 2017 13:36:33 +0530 Message-Id: <1493280397-9622-4-git-send-email-ashijeetacharya@gmail.com> X-Mailer: git-send-email 2.6.2 In-Reply-To: <1493280397-9622-1-git-send-email-ashijeetacharya@gmail.com> References: <1493280397-9622-1-git-send-email-ashijeetacharya@gmail.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c00::242 Subject: [Qemu-devel] [PATCH v2 3/7] dmg: Refactor and prepare dmg_read_chunk() to cache random access points X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, famz@redhat.com, qemu-block@nongnu.org, qemu-devel@nongnu.org, mreitz@redhat.com, peter@lekensteyn.nl, Ashijeet Acharya , jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Refactor dmg_read_chunk() to start making use of the new DMGReadState structure and do chunk and sector related calculations based on it. Add a new argument "DMGReadState *drs" to it. Also, rename DMG_SECTORCOUNTS_MAX to DMG_SECTOR_MAX to avoid indentaion problems at some places inside dmg_read_chunk() Signed-off-by: Ashijeet Acharya --- block/dmg.c | 159 +++++++++++++++++++++++++++++++++++---------------------= ---- 1 file changed, 94 insertions(+), 65 deletions(-) diff --git a/block/dmg.c b/block/dmg.c index c6fe8b0..32623e2 100644 --- a/block/dmg.c +++ b/block/dmg.c @@ -38,7 +38,7 @@ enum { * or truncating when converting to 32-bit types */ DMG_LENGTHS_MAX =3D 64 * 1024 * 1024, /* 64 MB */ - DMG_SECTORCOUNTS_MAX =3D DMG_LENGTHS_MAX / 512, + DMG_SECTOR_MAX =3D DMG_LENGTHS_MAX / 512, }; =20 static int dmg_probe(const uint8_t *buf, int buf_size, const char *filenam= e) @@ -260,10 +260,10 @@ static int dmg_read_mish_block(BDRVDMGState *s, DmgHe= aderState *ds, =20 /* all-zeroes sector (type 2) does not need to be "uncompressed" a= nd can * therefore be unbounded. */ - if (s->types[i] !=3D 2 && s->sectorcounts[i] > DMG_SECTORCOUNTS_MA= X) { + if (s->types[i] !=3D 2 && s->sectorcounts[i] > DMG_SECTOR_MAX) { error_report("sector count %" PRIu64 " for chunk %" PRIu32 " is larger than max (%u)", - s->sectorcounts[i], i, DMG_SECTORCOUNTS_MAX); + s->sectorcounts[i], i, DMG_SECTOR_MAX); ret =3D -EINVAL; goto fail; } @@ -578,78 +578,106 @@ static inline uint32_t search_chunk(BDRVDMGState *s,= uint64_t sector_num) return s->n_chunks; /* error */ } =20 -static inline int dmg_read_chunk(BlockDriverState *bs, uint64_t sector_num) +static inline int dmg_read_chunk(BlockDriverState *bs, uint64_t sector_num, + DMGReadState *drs) { BDRVDMGState *s =3D bs->opaque; =20 + int ret; + uint32_t sector_offset; + uint64_t sectors_read; + uint32_t chunk; + if (!is_sector_in_chunk(s, s->current_chunk, sector_num)) { - int ret; - uint32_t chunk =3D search_chunk(s, sector_num); + chunk =3D search_chunk(s, sector_num); + } else { + chunk =3D drs->saved_chunk_type; + } =20 - if (chunk >=3D s->n_chunks) { + if (chunk >=3D s->n_chunks) { + return -1; + } + + /* reset our access point cache if we had a change in current chunk */ + if (chunk !=3D drs->saved_chunk_type) { + cache_access_point(drs, NULL, -1, -1, -1, -1); + } + + sector_offset =3D sector_num - s->sectors[chunk]; + + if ((s->sectorcounts[chunk] - sector_offset) > DMG_SECTOR_MAX) { + sectors_read =3D DMG_SECTOR_MAX; + } else { + sectors_read =3D s->sectorcounts[chunk] - sector_offset; + } + + /* truncate sectors read if it exceeds the 2MB buffer of qemu-img + * convert */ + if ((sector_num % DMG_SECTOR_MAX) + sectors_read > DMG_SECTOR_MAX) { + sectors_read =3D DMG_SECTOR_MAX - (sector_num % DMG_SECTOR_MAX); + } + + s->current_chunk =3D s->n_chunks; + + switch (s->types[chunk]) { /* block entry type */ + case 0x80000005: { /* zlib compressed */ + /* we need to buffer, because only the chunk as whole can be + * inflated. */ + ret =3D bdrv_pread(bs->file, s->offsets[chunk], + s->compressed_chunk, s->lengths[chunk]); + if (ret !=3D s->lengths[chunk]) { return -1; } =20 - s->current_chunk =3D s->n_chunks; - switch (s->types[chunk]) { /* block entry type */ - case 0x80000005: { /* zlib compressed */ - /* we need to buffer, because only the chunk as whole can be - * inflated. */ - ret =3D bdrv_pread(bs->file, s->offsets[chunk], - s->compressed_chunk, s->lengths[chunk]); - if (ret !=3D s->lengths[chunk]) { - return -1; - } - - s->zstream.next_in =3D s->compressed_chunk; - s->zstream.avail_in =3D s->lengths[chunk]; - s->zstream.next_out =3D s->uncompressed_chunk; - s->zstream.avail_out =3D 512 * s->sectorcounts[chunk]; - ret =3D inflateReset(&s->zstream); - if (ret !=3D Z_OK) { - return -1; - } - ret =3D inflate(&s->zstream, Z_FINISH); - if (ret !=3D Z_STREAM_END || - s->zstream.total_out !=3D 512 * s->sectorcounts[chunk]) { - return -1; - } - break; } - case 0x80000006: /* bzip2 compressed */ - if (!dmg_uncompress_bz2) { - break; - } - /* we need to buffer, because only the chunk as whole can be - * inflated. */ - ret =3D bdrv_pread(bs->file, s->offsets[chunk], - s->compressed_chunk, s->lengths[chunk]); - if (ret !=3D s->lengths[chunk]) { - return -1; - } - - ret =3D dmg_uncompress_bz2((char *)s->compressed_chunk, - (unsigned int) s->lengths[chunk], - (char *)s->uncompressed_chunk, - (unsigned int) - (512 * s->sectorcounts[chunk])); - if (ret < 0) { - return ret; - } - break; - case 1: /* copy */ - ret =3D bdrv_pread(bs->file, s->offsets[chunk], - s->uncompressed_chunk, s->lengths[chunk]); - if (ret !=3D s->lengths[chunk]) { - return -1; - } - break; - case 2: /* zero */ - /* see dmg_read, it is treated specially. No buffer needs to be - * pre-filled, the zeroes can be set directly. */ + s->zstream.next_in =3D s->compressed_chunk; + s->zstream.avail_in =3D s->lengths[chunk]; + s->zstream.next_out =3D s->uncompressed_chunk; + s->zstream.avail_out =3D 512 * s->sectorcounts[chunk]; + ret =3D inflateReset(&s->zstream); + if (ret !=3D Z_OK) { + return -1; + } + ret =3D inflate(&s->zstream, Z_FINISH); + if (ret !=3D Z_STREAM_END || + s->zstream.total_out !=3D 512 * s->sectorcounts[chunk]) { + return -1; + } + break; } + case 0x80000006: /* bzip2 compressed */ + if (!dmg_uncompress_bz2) { break; } - s->current_chunk =3D chunk; + /* we need to buffer, because only the chunk as whole can be + * inflated. */ + ret =3D bdrv_pread(bs->file, s->offsets[chunk], + s->compressed_chunk, s->lengths[chunk]); + if (ret !=3D s->lengths[chunk]) { + return -1; + } + + ret =3D dmg_uncompress_bz2((char *)s->compressed_chunk, + (unsigned int) s->lengths[chunk], + (char *)s->uncompressed_chunk, + (unsigned int) + (512 * s->sectorcounts[chunk])); + if (ret < 0) { + return ret; + } + break; + case 1: /* copy */ + ret =3D bdrv_pread(bs->file, s->offsets[chunk], + s->uncompressed_chunk, s->lengths[chunk]); + if (ret !=3D s->lengths[chunk]) { + return -1; + } + break; + case 2: /* zero */ + /* see dmg_read, it is treated specially. No buffer needs to be + * pre-filled, the zeroes can be set directly. */ + break; } + s->current_chunk =3D chunk; + return 0; } =20 @@ -661,6 +689,7 @@ dmg_co_preadv(BlockDriverState *bs, uint64_t offset, ui= nt64_t bytes, uint64_t sector_num =3D offset >> BDRV_SECTOR_BITS; int nb_sectors =3D bytes >> BDRV_SECTOR_BITS; int ret, i; + DMGReadState *drs =3D s->drs; =20 assert((offset & (BDRV_SECTOR_SIZE - 1)) =3D=3D 0); assert((bytes & (BDRV_SECTOR_SIZE - 1)) =3D=3D 0); @@ -671,7 +700,7 @@ dmg_co_preadv(BlockDriverState *bs, uint64_t offset, ui= nt64_t bytes, uint32_t sector_offset_in_chunk; void *data; =20 - if (dmg_read_chunk(bs, sector_num + i) !=3D 0) { + if (dmg_read_chunk(bs, sector_num + i, drs) !=3D 0) { ret =3D -EIO; goto fail; } --=20 2.6.2 From nobody Thu May 2 22:43:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; dkim=fail spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1493280474182758.5806871641846; Thu, 27 Apr 2017 01:07:54 -0700 (PDT) Received: from localhost ([::1]:59292 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d3eSu-0004mc-Mj for importer@patchew.org; Thu, 27 Apr 2017 04:07:52 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56470) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d3eRT-0003Oa-8W for qemu-devel@nongnu.org; Thu, 27 Apr 2017 04:06:24 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d3eRS-0008On-6t for qemu-devel@nongnu.org; Thu, 27 Apr 2017 04:06:23 -0400 Received: from mail-pf0-x241.google.com ([2607:f8b0:400e:c00::241]:35614) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1d3eRS-0008Nt-0G; Thu, 27 Apr 2017 04:06:22 -0400 Received: by mail-pf0-x241.google.com with SMTP id a188so7473429pfa.2; Thu, 27 Apr 2017 01:06:21 -0700 (PDT) Received: from linux.local ([157.51.104.251]) by smtp.gmail.com with ESMTPSA id 12sm3295277pgb.35.2017.04.27.01.06.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 27 Apr 2017 01:06:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=VpBxeQqhMPqGtrAXYs1LE5RxBfl20UsTTF4j9evfuAY=; b=fXrcF6WDjlv3fCktR0mJNH/r5YstBjsTqU2a5udDHw1fIHBcS3RwPtyQD5ONKlZW8S ak/5Rh40veUXqf6mXk5z2PrSsxNVBF3ILwcNOIE6P2zmm9rAQ2ALONesftToAIGny1cT pp0cXDEqv2nNhpVzBQu6yL2AlOwYuenUEWFvZKx3HCbju7JlxsliH7zhJ5YyImQOOpUw 8sMGHrhB0m9Ss2fhTCXIHMyt1aY09CGvkpkMAiKeK9MZDe2ZyGxm8ioeq8AL/zYKg0y+ K67x7JnpWK/1RV3PuVbtDAjQ9/y4beH6i8BY7qSqbwbo04Nwh9if9zriscVU+GgQV6u+ u1gA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=VpBxeQqhMPqGtrAXYs1LE5RxBfl20UsTTF4j9evfuAY=; b=V9eudIaULp6JYNsyDC+89YJbZ82KbBtVD3N04Qwz2FgFnf0Qr/rsnxwAgPGvrgwti9 RhbWFAWe4fSjryLiCnDNzpQiJo+SPvtw6J0AqHwhqrShwtsuefZW5eVqmQDcVjkU3U4k /WpRUGgrulRHR1hPYXrDjpj2yCLFTVmADDPwYfjgMh9JPn3P0EJ0w+Vt6ar5he/HNGDw XHViS2UwmxB83HfUQKHmuIqCuLgsWk2yyMdy7UfoY9w79OeIz7La0KzKPxxaeCxsW4Zl Rc7Kce3MR/K1yOeAhpFchY4AGRe1ky4UdUjK80Ziz4gfHlitT/cKsgyFi720qbHDoT22 CTJg== X-Gm-Message-State: AN3rC/5+EEuqbTr8hGGfK/go7S6QLtB3YrjqNwWPbVAB5R6IGa+8/zFl Q+wv+vZs/lU38g== X-Received: by 10.99.125.18 with SMTP id y18mr4444850pgc.32.1493280381107; Thu, 27 Apr 2017 01:06:21 -0700 (PDT) From: Ashijeet Acharya To: stefanha@gmail.com Date: Thu, 27 Apr 2017 13:36:34 +0530 Message-Id: <1493280397-9622-5-git-send-email-ashijeetacharya@gmail.com> X-Mailer: git-send-email 2.6.2 In-Reply-To: <1493280397-9622-1-git-send-email-ashijeetacharya@gmail.com> References: <1493280397-9622-1-git-send-email-ashijeetacharya@gmail.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c00::241 Subject: [Qemu-devel] [PATCH v2 4/7] dmg: Handle zlib compressed chunks X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, famz@redhat.com, qemu-block@nongnu.org, qemu-devel@nongnu.org, mreitz@redhat.com, peter@lekensteyn.nl, Ashijeet Acharya , jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Set the output buffer size to be equal to the size of number of sectors stored in @sectors_read. Start inflating to a max output buffer size of 2MB and cache our access point to aid random access later if required. Signed-off-by: Ashijeet Acharya --- block/dmg.c | 48 ++++++++++++++++++++++++++++++++++-------------- 1 file changed, 34 insertions(+), 14 deletions(-) diff --git a/block/dmg.c b/block/dmg.c index 32623e2..073e864 100644 --- a/block/dmg.c +++ b/block/dmg.c @@ -621,27 +621,47 @@ static inline int dmg_read_chunk(BlockDriverState *bs= , uint64_t sector_num, =20 switch (s->types[chunk]) { /* block entry type */ case 0x80000005: { /* zlib compressed */ - /* we need to buffer, because only the chunk as whole can be - * inflated. */ - ret =3D bdrv_pread(bs->file, s->offsets[chunk], - s->compressed_chunk, s->lengths[chunk]); - if (ret !=3D s->lengths[chunk]) { - return -1; + /* check for cached random access point */ + if (drs->saved_next_in =3D=3D NULL) { + /* we need to buffer, because only the chunk as whole can be + * inflated. */ + ret =3D bdrv_pread(bs->file, s->offsets[chunk], + s->compressed_chunk, s->lengths[chunk]); + if (ret !=3D s->lengths[chunk]) { + return -1; + } + + s->zstream.next_in =3D s->compressed_chunk; + s->zstream.avail_in =3D s->lengths[chunk]; + } else { + s->zstream.next_in =3D drs->saved_next_in; + s->zstream.avail_in =3D drs->saved_avail_in; } =20 - s->zstream.next_in =3D s->compressed_chunk; - s->zstream.avail_in =3D s->lengths[chunk]; s->zstream.next_out =3D s->uncompressed_chunk; - s->zstream.avail_out =3D 512 * s->sectorcounts[chunk]; - ret =3D inflateReset(&s->zstream); - if (ret !=3D Z_OK) { - return -1; + + s->zstream.avail_out =3D sectors_read * BDRV_SECTOR_SIZE; + + if (drs->saved_next_in =3D=3D NULL) { + ret =3D inflateReset(&s->zstream); + if (ret !=3D Z_OK) { + return -1; + } + } + /* reset total_out for each successive call */ + s->zstream.total_out =3D 0; + ret =3D inflate(&s->zstream, Z_SYNC_FLUSH); + if (ret =3D=3D Z_OK && + s->zstream.total_out =3D=3D 512 * sectors_read) { + goto update; } - ret =3D inflate(&s->zstream, Z_FINISH); if (ret !=3D Z_STREAM_END || - s->zstream.total_out !=3D 512 * s->sectorcounts[chunk]) { + s->zstream.total_out !=3D 512 * sectors_read) { return -1; } +update: + cache_access_point(drs, s->zstream.next_in, s->zstream.avail_in, + chunk, sectors_read, sector_offset); break; } case 0x80000006: /* bzip2 compressed */ if (!dmg_uncompress_bz2) { --=20 2.6.2 From nobody Thu May 2 22:43:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; dkim=fail spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1493280848051981.1261049303446; Thu, 27 Apr 2017 01:14:08 -0700 (PDT) Received: from localhost ([::1]:59329 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d3eYw-0001QA-AY for importer@patchew.org; Thu, 27 Apr 2017 04:14:06 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56520) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d3eRZ-0003UG-NW for qemu-devel@nongnu.org; Thu, 27 Apr 2017 04:06:30 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d3eRX-0008UX-GN for qemu-devel@nongnu.org; Thu, 27 Apr 2017 04:06:29 -0400 Received: from mail-pf0-x244.google.com ([2607:f8b0:400e:c00::244]:36640) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1d3eRX-0008Tk-Ai; Thu, 27 Apr 2017 04:06:27 -0400 Received: by mail-pf0-x244.google.com with SMTP id v14so7447528pfd.3; Thu, 27 Apr 2017 01:06:27 -0700 (PDT) Received: from linux.local ([157.51.104.251]) by smtp.gmail.com with ESMTPSA id 12sm3295277pgb.35.2017.04.27.01.06.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 27 Apr 2017 01:06:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=dx8DVoQIxQQ2QPeCG+pUqSdoSz3OnbHLU6+77A/H6D4=; b=fzDcEotg8hEa6QIoEVN0RvN3l/wNIOpM3AQZOFVObiVIEOOKStI+Py6Z9mr+hNYWop Qjv7/tMk1WFYXDIkx+m4wYozhsOpQRHi3017tdaEM1VD9E/fr+UZx+2vRFqEzPMeRW0e 9FechYhZ1XTkAdA60aPpfUVmGTcVa4YXws64geWX47l+vgncu29FJaWwONiw4dmyHYPV qya5eYFtRmjptbF9HNZ2FuAUGmHA4kyruwvsjrj4PomizxqVCGTz5Ap67JfHCI1c5MIr AA9lnoNXU0tTYr93jyD7s7gbZxnxBH2nCrtbKWqkyTWLjLu1lrfP+gIQKCpxqRlTSW8O TlLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=dx8DVoQIxQQ2QPeCG+pUqSdoSz3OnbHLU6+77A/H6D4=; b=iwk2nAksqbi0Ym0X3xEb/OtWHxwSZlg45D6xumIKS0DtsxjmUQlRo5xCYe5AyirEvv VhK2MAclmJ/J5OWVj7uMTIIc/qAWg4RzZUAIMgtfpRi/bRwsjzNem9yRkMBsruXKJ8eV hYJJa88+J+NfGF4ODDDAugLD7+o/pHB9q0ExBMq7nSbVSqQ6vBivgoVTCXqX1hLOVxIp O3TdP5GjTzH2onWF/0pRcsJBRI7Ojxd9wW+Yt7iLsa5FLvXKlpBU6kiwEd8+Qdx7osUe slGgKZOT4yrD8iqc0Mewy6enjGKirqaUlO3ttqM7mxJBXyM0ppYyBWfRE3Eqh8MSS1fh gh4Q== X-Gm-Message-State: AN3rC/5ntxsZF6eN+W3Xcrrs/3Q4uKBmpLZoe9/HknS8VDdChH1yUqdr A4zlW/kK8r2r5A== X-Received: by 10.98.34.5 with SMTP id i5mr4398259pfi.228.1493280386438; Thu, 27 Apr 2017 01:06:26 -0700 (PDT) From: Ashijeet Acharya To: stefanha@gmail.com Date: Thu, 27 Apr 2017 13:36:35 +0530 Message-Id: <1493280397-9622-6-git-send-email-ashijeetacharya@gmail.com> X-Mailer: git-send-email 2.6.2 In-Reply-To: <1493280397-9622-1-git-send-email-ashijeetacharya@gmail.com> References: <1493280397-9622-1-git-send-email-ashijeetacharya@gmail.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c00::244 Subject: [Qemu-devel] [PATCH v2 5/7] dmg: Handle bz2 compressed/raw/zeroed chunks X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, famz@redhat.com, qemu-block@nongnu.org, qemu-devel@nongnu.org, mreitz@redhat.com, peter@lekensteyn.nl, Ashijeet Acharya , jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" We do not need to cache the access point for these chunks but need to update our various supporting variables like chunk, sectors_read etc. to keep maintaining our code structure. Call cache_access_point() after reading chunks of these types. Signed-off-by: Ashijeet Acharya --- block/dmg.c | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/block/dmg.c b/block/dmg.c index 073e864..f9045f9 100644 --- a/block/dmg.c +++ b/block/dmg.c @@ -680,20 +680,30 @@ update: (char *)s->uncompressed_chunk, (unsigned int) (512 * s->sectorcounts[chunk])); + if (ret < 0) { return ret; } + cache_access_point(drs, NULL, -1, chunk, sectors_read, + sector_offset); break; case 1: /* copy */ - ret =3D bdrv_pread(bs->file, s->offsets[chunk], - s->uncompressed_chunk, s->lengths[chunk]); - if (ret !=3D s->lengths[chunk]) { - return -1; + if (drs->sectors_read =3D=3D -1) { + ret =3D bdrv_pread(bs->file, s->offsets[chunk], + s->uncompressed_chunk, s->lengths[chunk]); + if (ret !=3D s->lengths[chunk]) { + return -1; + } } + cache_access_point(drs, NULL, -1, chunk, sectors_read, + sector_offset); break; case 2: /* zero */ /* see dmg_read, it is treated specially. No buffer needs to be * pre-filled, the zeroes can be set directly. */ + cache_access_point(drs, NULL, -1, chunk, sectors_read, + sector_offset); + break; } s->current_chunk =3D chunk; --=20 2.6.2 From nobody Thu May 2 22:43:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; dkim=fail spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 149328059931851.24432090096229; Thu, 27 Apr 2017 01:09:59 -0700 (PDT) Received: from localhost ([::1]:59302 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d3eUv-0006QG-OP for importer@patchew.org; Thu, 27 Apr 2017 04:09:57 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56582) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d3eRd-0003X8-FD for qemu-devel@nongnu.org; Thu, 27 Apr 2017 04:06:34 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d3eRc-00007Q-IB for qemu-devel@nongnu.org; Thu, 27 Apr 2017 04:06:33 -0400 Received: from mail-pf0-x243.google.com ([2607:f8b0:400e:c00::243]:35652) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1d3eRc-00007B-CL; Thu, 27 Apr 2017 04:06:32 -0400 Received: by mail-pf0-x243.google.com with SMTP id a188so7474760pfa.2; Thu, 27 Apr 2017 01:06:32 -0700 (PDT) Received: from linux.local ([157.51.104.251]) by smtp.gmail.com with ESMTPSA id 12sm3295277pgb.35.2017.04.27.01.06.27 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 27 Apr 2017 01:06:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=4acXijkNEnMWOZC9MFvY1Q35rJXGr9zjf34TMnSb0so=; b=IWEjEdMPQSk8vW6Z9K9z6KtAJAkaI8rWJvwdETXAW4X1ZPGIfhDnSt7aiiP6ahMyRj nIphAdRBQ8UVZbOZstzS6OLpNTtFsSB6E9dK3aOKbwlsulisds9YE21O0uaIpLFNX0Wm JDzMl/RuHOzQdScueBsoWg+3dXmW+38OwK3EOgg96+Tm5Bu0MwOudZRXIetHhNBzD8fr 4LGbUZ7sMYXoz9EXCP/sCwv7Mc1VprmJxn5kYOcLZKvniVxeuQAs4kVFnV+Qh1tBzY7d HGIesVDqdf1YmbJngFIvzxRBZOiGOdZejl0v0O4TGxPAia4DME4mhegjuU+rO2dLW2Io 7fTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=4acXijkNEnMWOZC9MFvY1Q35rJXGr9zjf34TMnSb0so=; b=XN08jnRIZ5wPVDketLfVGBOXJUmvZdcq1UNCfL5Rwl1s5OsudehhZ3McAyhNW12Tl7 g+gKI0gNQGuGDBL9K5k0FLJ45ESWpBBT+j9aQyzbuh59uZgdKkcEMc51mXPwMId0OgVu x4GUu1eLpHWGyQ8Qtr1eiSBCVtz6kXdbr1nr86X5aZRAsLwIFdlZIqEMz73akVpO5TeI cim8m+ZkfrVi5yTdp0cf6lpsyZn2FTllT0uw/g2sXR0gxb+jl5GMLutPZ1VGT3WvRGHw 57hi1A49d0hkNkSLVWlUK73SFf6aTH36wDTXT6lyG+n4EHxPjDNZBtOG9KdCqDdRsKdD 0nDg== X-Gm-Message-State: AN3rC/70F+d+tQlDhHhKnzvsMzu+L7dwDJgENcwLWQjvcAOTh2ydfbkj xZQNVF3D90krmTNs X-Received: by 10.84.245.1 with SMTP id i1mr487586pll.51.1493280391604; Thu, 27 Apr 2017 01:06:31 -0700 (PDT) From: Ashijeet Acharya To: stefanha@gmail.com Date: Thu, 27 Apr 2017 13:36:36 +0530 Message-Id: <1493280397-9622-7-git-send-email-ashijeetacharya@gmail.com> X-Mailer: git-send-email 2.6.2 In-Reply-To: <1493280397-9622-1-git-send-email-ashijeetacharya@gmail.com> References: <1493280397-9622-1-git-send-email-ashijeetacharya@gmail.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c00::243 Subject: [Qemu-devel] [PATCH v2 6/7] dmg: Refactor dmg_co_preadv() to start reading multiple sectors X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, famz@redhat.com, qemu-block@nongnu.org, qemu-devel@nongnu.org, mreitz@redhat.com, peter@lekensteyn.nl, Ashijeet Acharya , jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" At the moment, dmg_co_preadv() reads one sector at a time. Make it read multiple sectors at a time depending on the number of sectors stored in "drs->sectors_read". This does not provide any significant optimization in the I/O process of DMG but is still a nicer way. Adjust the 'data' variable depending on our cached access point situation to align our read requests appropriately. Signed-off-by: Ashijeet Acharya --- block/dmg.c | 23 +++++++++++++++-------- 1 file changed, 15 insertions(+), 8 deletions(-) diff --git a/block/dmg.c b/block/dmg.c index f9045f9..8b7460c 100644 --- a/block/dmg.c +++ b/block/dmg.c @@ -718,7 +718,7 @@ dmg_co_preadv(BlockDriverState *bs, uint64_t offset, ui= nt64_t bytes, BDRVDMGState *s =3D bs->opaque; uint64_t sector_num =3D offset >> BDRV_SECTOR_BITS; int nb_sectors =3D bytes >> BDRV_SECTOR_BITS; - int ret, i; + int ret, i =3D 0; DMGReadState *drs =3D s->drs; =20 assert((offset & (BDRV_SECTOR_SIZE - 1)) =3D=3D 0); @@ -726,8 +726,7 @@ dmg_co_preadv(BlockDriverState *bs, uint64_t offset, ui= nt64_t bytes, =20 qemu_co_mutex_lock(&s->lock); =20 - for (i =3D 0; i < nb_sectors; i++) { - uint32_t sector_offset_in_chunk; + while (i < nb_sectors) { void *data; =20 if (dmg_read_chunk(bs, sector_num + i, drs) !=3D 0) { @@ -738,12 +737,20 @@ dmg_co_preadv(BlockDriverState *bs, uint64_t offset, = uint64_t bytes, * s->uncompressed_chunk may be too small to cover the large all-z= eroes * section. dmg_read_chunk is called to find s->current_chunk */ if (s->types[s->current_chunk] =3D=3D 2) { /* all zeroes block ent= ry */ - qemu_iovec_memset(qiov, i * 512, 0, 512); - continue; + qemu_iovec_memset(qiov, i * 512, 0, + 512 * drs->sectors_read); + goto increment; + } + + if (drs->saved_next_in =3D=3D NULL) { + data =3D s->uncompressed_chunk + drs->sector_offset_in_chunk *= 512; + } else { + data =3D s->uncompressed_chunk; } - sector_offset_in_chunk =3D sector_num + i - s->sectors[s->current_= chunk]; - data =3D s->uncompressed_chunk + sector_offset_in_chunk * 512; - qemu_iovec_from_buf(qiov, i * 512, data, 512); + qemu_iovec_from_buf(qiov, i * 512, data, drs->sectors_read * 512); + +increment: + i +=3D drs->sectors_read; } =20 ret =3D 0; --=20 2.6.2 From nobody Thu May 2 22:43:31 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; dkim=fail spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1493280607506259.3965607578607; Thu, 27 Apr 2017 01:10:07 -0700 (PDT) Received: from localhost ([::1]:59304 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d3eV4-0006Xr-3B for importer@patchew.org; Thu, 27 Apr 2017 04:10:06 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56647) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d3eRk-0003cp-Oo for qemu-devel@nongnu.org; Thu, 27 Apr 2017 04:06:42 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d3eRh-0000BX-Uu for qemu-devel@nongnu.org; Thu, 27 Apr 2017 04:06:40 -0400 Received: from mail-pf0-x243.google.com ([2607:f8b0:400e:c00::243]:35677) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1d3eRh-0000B9-Ol; Thu, 27 Apr 2017 04:06:37 -0400 Received: by mail-pf0-x243.google.com with SMTP id a188so7475557pfa.2; Thu, 27 Apr 2017 01:06:37 -0700 (PDT) Received: from linux.local ([157.51.104.251]) by smtp.gmail.com with ESMTPSA id 12sm3295277pgb.35.2017.04.27.01.06.32 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 27 Apr 2017 01:06:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=YhO5+srANrnRpcijNdsLgMjEPD8u/QrMGtb+IQ9P6Ng=; b=hnmfXegEdwOC3KuIDJ/i2Q0hP97dAXZFYejAVLZFSjuNNTotuDovozgMVd1uyqGPxc wmoqk6L0r9N3drOR85wPVjt6Y8AEqfF34HTkFDg3d/a2RsUBWS20fU5VdJC718e5943m ysDS2Qefu4+A5fEi9G77+DGsBnNm7J3H8S9GLa2wKBgI4yFsUVjInRM6nFjRGFq3giFQ qCP4g9XvkHGAJi4ARd+UhBfJnnRZcV5yKjgm1Zhiici6qEfz+VZ/92ec6p0rYNWSaSTo OZ8FmxdGc8KmRRdrEDQzByh0eO2zB8w0yoKIH78OFsybjFzj2IZD0bQ661BH6rEmG6hr 5i5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=YhO5+srANrnRpcijNdsLgMjEPD8u/QrMGtb+IQ9P6Ng=; b=VQIQJTtcAAYePLkGCPcZQG3blnv97cXSWxz6HYnr0XraDleTwbbNCaHhoCoGUz5sFR dbCDMxZXHiM1ZX3Mhlm3LXyILTqav7D+6aKOWYuxfbghsi/Y7blb83pec3N5Bsf+unN6 etNZn/QXLc0jBwcFp+lRqbxznajLcHMLzZ6bY0iy/NqTgTyPcErujUOd7oYiR+/i4Uoz wj5oKG90rrRCpjm0Majx+rVHx2DLiwv7Tjf5lt5U7ODWSEOExBZVTQE2fAp/pC74tcQU +edrKlbxXLB+dODfuAlkU0FtDWvfQpvqUkZJuF7bNpfmGNdw/fHCaeWvBexXREmck0gz nV5A== X-Gm-Message-State: AN3rC/7yGNK7p6ks9v9PpV9x1BCEJBbdcp0jftQtAqiaOihSSWLnNIM1 0MSGbHz/FvpLZQ== X-Received: by 10.84.217.148 with SMTP id p20mr5400139pli.164.1493280397018; Thu, 27 Apr 2017 01:06:37 -0700 (PDT) From: Ashijeet Acharya To: stefanha@gmail.com Date: Thu, 27 Apr 2017 13:36:37 +0530 Message-Id: <1493280397-9622-8-git-send-email-ashijeetacharya@gmail.com> X-Mailer: git-send-email 2.6.2 In-Reply-To: <1493280397-9622-1-git-send-email-ashijeetacharya@gmail.com> References: <1493280397-9622-1-git-send-email-ashijeetacharya@gmail.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c00::243 Subject: [Qemu-devel] [PATCH v2 7/7] dmg: Limit the output buffer size to a max of 2MB X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, famz@redhat.com, qemu-block@nongnu.org, qemu-devel@nongnu.org, mreitz@redhat.com, peter@lekensteyn.nl, Ashijeet Acharya , jsnow@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZohoMail: RDKM_2 RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The size of the output buffer is limited to a maximum of 2MB so that QEMU doesn't end up allocating huge amounts of memory while decompressing compressed input streams. 2MB is an appropriate size because "qemu-img convert" has the same I/O buffer size and the most important use case for DMG files is to be compatible with qemu-img convert. We have refactored the DMG driver to accept and process images irrespective of their chunk sizes since we now have a limit of 2MB on our output buffer size. Thus QEMU will not allocate huge amounts of memory no matter what the chunk size is. Remove the error messages to prevent denial-of-service in cases where untrusted files are being accessed by the user. Signed-off-by: Ashijeet Acharya --- block/dmg.c | 24 +----------------------- 1 file changed, 1 insertion(+), 23 deletions(-) diff --git a/block/dmg.c b/block/dmg.c index 8b7460c..ade2578 100644 --- a/block/dmg.c +++ b/block/dmg.c @@ -37,7 +37,7 @@ enum { /* Limit chunk sizes to prevent unreasonable amounts of memory being u= sed * or truncating when converting to 32-bit types */ - DMG_LENGTHS_MAX =3D 64 * 1024 * 1024, /* 64 MB */ + DMG_LENGTHS_MAX =3D 2 * 1024 * 1024, /* 2 MB */ DMG_SECTOR_MAX =3D DMG_LENGTHS_MAX / 512, }; =20 @@ -209,7 +209,6 @@ static int dmg_read_mish_block(BDRVDMGState *s, DmgHead= erState *ds, uint8_t *buffer, uint32_t count) { uint32_t type, i; - int ret; size_t new_size; uint32_t chunk_count; int64_t offset =3D 0; @@ -258,16 +257,6 @@ static int dmg_read_mish_block(BDRVDMGState *s, DmgHea= derState *ds, /* sector count */ s->sectorcounts[i] =3D buff_read_uint64(buffer, offset + 0x10); =20 - /* all-zeroes sector (type 2) does not need to be "uncompressed" a= nd can - * therefore be unbounded. */ - if (s->types[i] !=3D 2 && s->sectorcounts[i] > DMG_SECTOR_MAX) { - error_report("sector count %" PRIu64 " for chunk %" PRIu32 - " is larger than max (%u)", - s->sectorcounts[i], i, DMG_SECTOR_MAX); - ret =3D -EINVAL; - goto fail; - } - /* offset in (compressed) data fork */ s->offsets[i] =3D buff_read_uint64(buffer, offset + 0x18); s->offsets[i] +=3D in_offset; @@ -275,23 +264,12 @@ static int dmg_read_mish_block(BDRVDMGState *s, DmgHe= aderState *ds, /* length in (compressed) data fork */ s->lengths[i] =3D buff_read_uint64(buffer, offset + 0x20); =20 - if (s->lengths[i] > DMG_LENGTHS_MAX) { - error_report("length %" PRIu64 " for chunk %" PRIu32 - " is larger than max (%u)", - s->lengths[i], i, DMG_LENGTHS_MAX); - ret =3D -EINVAL; - goto fail; - } - update_max_chunk_size(s, i, &ds->max_compressed_size, &ds->max_sectors_per_chunk); offset +=3D 40; } s->n_chunks +=3D chunk_count; return 0; - -fail: - return ret; } =20 static int dmg_read_resource_fork(BlockDriverState *bs, DmgHeaderState *ds, --=20 2.6.2