From nobody Tue Nov 4 19:03:27 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1530715792657672.6191654606664; Wed, 4 Jul 2018 07:49:52 -0700 (PDT) Received: from localhost ([::1]:47566 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1faj6N-0003CW-Vk for importer@patchew.org; Wed, 04 Jul 2018 10:49:52 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:43779) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1faj3U-0001UH-4K for qemu-devel@nongnu.org; Wed, 04 Jul 2018 10:46:52 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1faj3T-0007jl-9m for qemu-devel@nongnu.org; Wed, 04 Jul 2018 10:46:52 -0400 Received: from mx1.mpynet.fi ([82.197.21.84]:61747) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1faj3P-0007gm-GE; Wed, 04 Jul 2018 10:46:47 -0400 From: Ari Sundholm To: Date: Wed, 4 Jul 2018 17:46:35 +0300 Message-ID: <1530715597-30975-2-git-send-email-ari@tuxera.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1530715597-30975-1-git-send-email-ari@tuxera.com> References: <1530715597-30975-1-git-send-email-ari@tuxera.com> MIME-Version: 1.0 X-ClientProxiedBy: tuxera-exch.ad.tuxera.com (10.20.48.11) To tuxera-exch.ad.tuxera.com (10.20.48.11) Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: none X-detected-operating-system: by eggs.gnu.org: FreeBSD 9.x [fuzzy] X-Received-From: 82.197.21.84 Subject: [Qemu-devel] [PATCH 1/3] block/blklogwrites: Change log_sector_size from int64_t to uint64_t X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , "open list:blklogwrites" , Ari Sundholm , Max Reitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This was a simple oversight when working on intermediate versions of the original patch which introduced blklogwrites. Signed-off-by: Ari Sundholm --- block/blklogwrites.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/block/blklogwrites.c b/block/blklogwrites.c index 47093fa..272e11a 100644 --- a/block/blklogwrites.c +++ b/block/blklogwrites.c @@ -79,7 +79,7 @@ static int blk_log_writes_open(BlockDriverState *bs, QDic= t *options, int flags, QemuOpts *opts; Error *local_err =3D NULL; int ret; - int64_t log_sector_size; + uint64_t log_sector_size; =20 opts =3D qemu_opts_create(&runtime_opts, NULL, 0, &error_abort); qemu_opts_absorb_qdict(opts, options, &local_err); @@ -101,11 +101,9 @@ static int blk_log_writes_open(BlockDriverState *bs, Q= Dict *options, int flags, log_sector_size =3D qemu_opt_get_size(opts, "log-sector-size", BDRV_SECTOR_SIZE); =20 - if (log_sector_size < 0 || log_sector_size > (1ull << 23) || - !is_power_of_2(log_sector_size)) - { + if (log_sector_size > (1ull << 23) || !is_power_of_2(log_sector_size))= { ret =3D -EINVAL; - error_setg(errp, "Invalid log sector size %"PRId64, log_sector_siz= e); + error_setg(errp, "Invalid log sector size %"PRIu64, log_sector_siz= e); goto fail; } =20 --=20 2.7.4 From nobody Tue Nov 4 19:03:27 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1530715755367965.3761554715408; Wed, 4 Jul 2018 07:49:15 -0700 (PDT) Received: from localhost ([::1]:47564 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1faj5m-0002p7-Jq for importer@patchew.org; Wed, 04 Jul 2018 10:49:14 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:43802) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1faj3V-0001Vn-HO for qemu-devel@nongnu.org; Wed, 04 Jul 2018 10:46:55 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1faj3T-0007kD-WC for qemu-devel@nongnu.org; Wed, 04 Jul 2018 10:46:53 -0400 Received: from mx1.mpynet.fi ([82.197.21.84]:52579) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1faj3Q-0007h6-Kg; Wed, 04 Jul 2018 10:46:48 -0400 From: Ari Sundholm To: Date: Wed, 4 Jul 2018 17:46:36 +0300 Message-ID: <1530715597-30975-3-git-send-email-ari@tuxera.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1530715597-30975-1-git-send-email-ari@tuxera.com> References: <1530715597-30975-1-git-send-email-ari@tuxera.com> MIME-Version: 1.0 X-ClientProxiedBy: tuxera-exch.ad.tuxera.com (10.20.48.11) To tuxera-exch.ad.tuxera.com (10.20.48.11) Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: none X-detected-operating-system: by eggs.gnu.org: FreeBSD 9.x [fuzzy] X-Received-From: 82.197.21.84 Subject: [Qemu-devel] [PATCH 2/3] block/blklogwrites: Add an option for appending to an old log X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , "open list:blklogwrites" , Markus Armbruster , Max Reitz , Ari Sundholm Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Suggested by Kevin Wolf. May be useful when testing multiple batches of writes or doing long-term testing involving restarts of the VM. Signed-off-by: Ari Sundholm --- block/blklogwrites.c | 147 ++++++++++++++++++++++++++++++++++++++++++++++-= ---- qapi/block-core.json | 3 +- 2 files changed, 135 insertions(+), 15 deletions(-) diff --git a/block/blklogwrites.c b/block/blklogwrites.c index 272e11a..56154e7 100644 --- a/block/blklogwrites.c +++ b/block/blklogwrites.c @@ -24,6 +24,10 @@ #define LOG_FUA_FLAG (1 << 1) #define LOG_DISCARD_FLAG (1 << 2) #define LOG_MARK_FLAG (1 << 3) +#define LOG_FLAG_MASK (LOG_FLUSH_FLAG \ + | LOG_FUA_FLAG \ + | LOG_DISCARD_FLAG \ + | LOG_MARK_FLAG) =20 #define WRITE_LOG_VERSION 1ULL #define WRITE_LOG_MAGIC 0x6a736677736872ULL @@ -58,6 +62,11 @@ static QemuOptsList runtime_opts =3D { .head =3D QTAILQ_HEAD_INITIALIZER(runtime_opts.head), .desc =3D { { + .name =3D "log-append", + .type =3D QEMU_OPT_BOOL, + .help =3D "Append to an existing log", + }, + { .name =3D "log-sector-size", .type =3D QEMU_OPT_SIZE, .help =3D "Log sector size", @@ -72,6 +81,53 @@ static inline uint32_t blk_log_writes_log2(uint32_t valu= e) return 31 - clz32(value); } =20 +static inline bool blk_log_writes_sector_size_valid(uint32_t sector_size) +{ + return sector_size < (1ull << 24) && is_power_of_2(sector_size); +} + +static uint64_t blk_log_writes_find_cur_log_sector(BdrvChild *log, + uint32_t sector_size, + uint64_t nr_entries, + Error **errp) +{ + uint64_t cur_sector =3D 1; + uint64_t cur_idx =3D 0; + uint32_t sector_bits =3D blk_log_writes_log2(sector_size); + struct log_write_entry cur_entry; + + while (cur_idx < nr_entries) { + int read_ret =3D bdrv_pread(log, cur_sector << sector_bits, &cur_e= ntry, + sizeof(cur_entry)); + if (read_ret < 0) { + error_setg_errno(errp, -read_ret, + "Failed to read log entry %"PRIu64, cur_idx); + return (uint64_t)-1ull; + } + + if (cur_entry.flags & ~cpu_to_le64(LOG_FLAG_MASK)) { + error_setg(errp, "Invalid flags 0x%"PRIx64" in log entry %"PRI= u64, + le64_to_cpu(cur_entry.flags), cur_idx); + return (uint64_t)-1ull; + } + + /* Account for the sector of the entry itself */ + ++cur_sector; + + /* + * Account for the data of the write. + * For discards, this data is not present. + */ + if (!(cur_entry.flags & cpu_to_le64(LOG_DISCARD_FLAG))) { + cur_sector +=3D le64_to_cpu(cur_entry.nr_sectors); + } + + ++cur_idx; + } + + return cur_sector; +} + static int blk_log_writes_open(BlockDriverState *bs, QDict *options, int f= lags, Error **errp) { @@ -80,6 +136,7 @@ static int blk_log_writes_open(BlockDriverState *bs, QDi= ct *options, int flags, Error *local_err =3D NULL; int ret; uint64_t log_sector_size; + bool log_append; =20 opts =3D qemu_opts_create(&runtime_opts, NULL, 0, &error_abort); qemu_opts_absorb_qdict(opts, options, &local_err); @@ -98,20 +155,6 @@ static int blk_log_writes_open(BlockDriverState *bs, QD= ict *options, int flags, goto fail; } =20 - log_sector_size =3D qemu_opt_get_size(opts, "log-sector-size", - BDRV_SECTOR_SIZE); - - if (log_sector_size > (1ull << 23) || !is_power_of_2(log_sector_size))= { - ret =3D -EINVAL; - error_setg(errp, "Invalid log sector size %"PRIu64, log_sector_siz= e); - goto fail; - } - - s->sectorsize =3D log_sector_size; - s->sectorbits =3D blk_log_writes_log2(log_sector_size); - s->cur_log_sector =3D 1; - s->nr_entries =3D 0; - /* Open the log file */ s->log_file =3D bdrv_open_child(NULL, options, "log", bs, &child_file,= false, &local_err); @@ -121,7 +164,83 @@ static int blk_log_writes_open(BlockDriverState *bs, Q= Dict *options, int flags, goto fail; } =20 + log_append =3D qemu_opt_get_bool(opts, "log-append", false); + + if (log_append) { + struct log_write_super log_sb =3D { 0, 0, 0, 0 }; + + if (qemu_opt_find(opts, "log-sector-size")) { + ret =3D -EINVAL; + error_setg(errp, "log-append and log-sector-size are mutually " + "exclusive"); + goto fail_log; + } + + /* Read log superblock or fake one for an empty log */ + if (!bdrv_getlength(s->log_file->bs)) { + log_sb.magic =3D cpu_to_le64(WRITE_LOG_MAGIC); + log_sb.version =3D cpu_to_le64(WRITE_LOG_VERSION); + log_sb.nr_entries =3D cpu_to_le64(0); + log_sb.sectorsize =3D cpu_to_le32(BDRV_SECTOR_SIZE); + } else { + ret =3D bdrv_pread(s->log_file, 0, &log_sb, sizeof(log_sb)); + if (ret < 0) { + error_setg_errno(errp, -ret, "Could not read log superbloc= k"); + goto fail_log; + } + } + + if (log_sb.magic !=3D cpu_to_le64(WRITE_LOG_MAGIC)) { + ret =3D -EINVAL; + error_setg(errp, "Invalid log superblock magic"); + goto fail_log; + } + + if (log_sb.version !=3D cpu_to_le64(WRITE_LOG_VERSION)) { + ret =3D -EINVAL; + error_setg(errp, "Unsupported log version %"PRIu64, + le64_to_cpu(log_sb.version)); + goto fail_log; + } + + log_sector_size =3D le32_to_cpu(log_sb.sectorsize); + s->cur_log_sector =3D 1; + s->nr_entries =3D 0; + + if (blk_log_writes_sector_size_valid(log_sector_size)) { + s->cur_log_sector =3D + blk_log_writes_find_cur_log_sector(s->log_file, log_sector= _size, + le64_to_cpu(log_sb.nr_entries), &local= _err); + if (local_err) { + ret =3D -EINVAL; + error_propagate(errp, local_err); + goto fail_log; + } + + s->nr_entries =3D le64_to_cpu(log_sb.nr_entries); + } + } else { + log_sector_size =3D qemu_opt_get_size(opts, "log-sector-size", + BDRV_SECTOR_SIZE); + s->cur_log_sector =3D 1; + s->nr_entries =3D 0; + } + + if (!blk_log_writes_sector_size_valid(log_sector_size)) { + ret =3D -EINVAL; + error_setg(errp, "Invalid log sector size %"PRIu64, log_sector_siz= e); + goto fail_log; + } + + s->sectorsize =3D log_sector_size; + s->sectorbits =3D blk_log_writes_log2(log_sector_size); + ret =3D 0; +fail_log: + if (ret < 0) { + bdrv_unref_child(bs, s->log_file); + s->log_file =3D NULL; + } fail: if (ret < 0) { bdrv_unref_child(bs, bs->file); diff --git a/qapi/block-core.json b/qapi/block-core.json index a9eab8c..d1753a2 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -3062,7 +3062,8 @@ { 'struct': 'BlockdevOptionsBlklogwrites', 'data': { 'file': 'BlockdevRef', 'log': 'BlockdevRef', - '*log-sector-size': 'uint32' } } + '*log-sector-size': 'uint32', + '*log-append': 'bool' } } =20 ## # @BlockdevOptionsBlkverify: --=20 2.7.4 From nobody Tue Nov 4 19:03:27 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1530715759526323.7828810427077; Wed, 4 Jul 2018 07:49:19 -0700 (PDT) Received: from localhost ([::1]:47565 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1faj5q-0002t0-PO for importer@patchew.org; Wed, 04 Jul 2018 10:49:18 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:43793) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1faj3U-0001Up-Mh for qemu-devel@nongnu.org; Wed, 04 Jul 2018 10:46:53 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1faj3T-0007k3-JI for qemu-devel@nongnu.org; Wed, 04 Jul 2018 10:46:52 -0400 Received: from mx1.mpynet.fi ([82.197.21.84]:18672) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1faj3P-0007hG-G9; Wed, 04 Jul 2018 10:46:47 -0400 From: Ari Sundholm To: Date: Wed, 4 Jul 2018 17:46:37 +0300 Message-ID: <1530715597-30975-4-git-send-email-ari@tuxera.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1530715597-30975-1-git-send-email-ari@tuxera.com> References: <1530715597-30975-1-git-send-email-ari@tuxera.com> MIME-Version: 1.0 X-ClientProxiedBy: tuxera-exch.ad.tuxera.com (10.20.48.11) To tuxera-exch.ad.tuxera.com (10.20.48.11) Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: none X-detected-operating-system: by eggs.gnu.org: FreeBSD 9.x [fuzzy] X-Received-From: 82.197.21.84 Subject: [Qemu-devel] [PATCH 3/3] block/blklogwrites: Add an option for the update interval of the log superblock X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , "open list:blklogwrites" , Markus Armbruster , Max Reitz , Ari Sundholm Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This is a way to ensure that the log superblock is periodically updated. Before, this was only done on flush requests, which may not be enough if the VM exits abnormally, omitting the final flush. The default interval is 4096 write requests. Signed-off-by: Ari Sundholm --- block/blklogwrites.c | 19 +++++++++++++++++-- qapi/block-core.json | 6 +++++- 2 files changed, 22 insertions(+), 3 deletions(-) diff --git a/block/blklogwrites.c b/block/blklogwrites.c index 56154e7..963e4dd 100644 --- a/block/blklogwrites.c +++ b/block/blklogwrites.c @@ -55,6 +55,7 @@ typedef struct { uint32_t sectorbits; uint64_t cur_log_sector; uint64_t nr_entries; + uint64_t update_interval; } BDRVBlkLogWritesState; =20 static QemuOptsList runtime_opts =3D { @@ -71,6 +72,11 @@ static QemuOptsList runtime_opts =3D { .type =3D QEMU_OPT_SIZE, .help =3D "Log sector size", }, + { + .name =3D "log-super-update-interval", + .type =3D QEMU_OPT_NUMBER, + .help =3D "Log superblock update interval (# of write requests= )", + }, { /* end of list */ } }, }; @@ -234,6 +240,13 @@ static int blk_log_writes_open(BlockDriverState *bs, Q= Dict *options, int flags, =20 s->sectorsize =3D log_sector_size; s->sectorbits =3D blk_log_writes_log2(log_sector_size); + s->update_interval =3D qemu_opt_get_number(opts, "log-super-update-int= erval", + 4096); + if (!s->update_interval) { + ret =3D -EINVAL; + error_setg(errp, "Invalid log superblock update interval %"PRIu64, + s->update_interval); + } =20 ret =3D 0; fail_log: @@ -360,8 +373,10 @@ static void coroutine_fn blk_log_writes_co_do_log(BlkL= ogWritesLogReq *lr) lr->zero_size, 0); } =20 - /* Update super block on flush */ - if (lr->log_ret =3D=3D 0 && lr->entry.flags & LOG_FLUSH_FLAG) { + /* Update super block on flush or every update interval */ + if (lr->log_ret =3D=3D 0 && ((lr->entry.flags & LOG_FLUSH_FLAG) + || (s->nr_entries % s->update_interval =3D=3D 0))) + { struct log_write_super super =3D { .magic =3D cpu_to_le64(WRITE_LOG_MAGIC), .version =3D cpu_to_le64(WRITE_LOG_VERSION), diff --git a/qapi/block-core.json b/qapi/block-core.json index d1753a2..38b3125 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -3057,13 +3057,17 @@ # @log-sector-size: sector size used in logging writes to @file, determines # granularity of offsets and sizes of writes (default: 5= 12) # +# @log-super-update-interval: interval of write requests after which the l= og +# super block is updated to disk (default: 409= 6) +# # Since: 3.0 ## { 'struct': 'BlockdevOptionsBlklogwrites', 'data': { 'file': 'BlockdevRef', 'log': 'BlockdevRef', '*log-sector-size': 'uint32', - '*log-append': 'bool' } } + '*log-append': 'bool', + '*log-super-update-interval': 'uint64' } } =20 ## # @BlockdevOptionsBlkverify: --=20 2.7.4