[PATCH v4 00/12] hw/block/nvme: metadata and end-to-end data protection support

Klaus Jensen posted 12 patches 3 years, 2 months ago
Failed in applying to current master (apply log)
There is a newer version of this series
hw/block/nvme-dif.h   |   53 ++
hw/block/nvme-ns.h    |   49 +-
hw/block/nvme.h       |   51 +-
include/block/nvme.h  |   34 +-
hw/block/nvme-dif.c   |  513 ++++++++++++++
hw/block/nvme-ns.c    |   90 ++-
hw/block/nvme.c       | 1548 ++++++++++++++++++++++++++++++++++-------
hw/block/meson.build  |    2 +-
hw/block/trace-events |   25 +-
9 files changed, 2071 insertions(+), 294 deletions(-)
create mode 100644 hw/block/nvme-dif.h
create mode 100644 hw/block/nvme-dif.c
[PATCH v4 00/12] hw/block/nvme: metadata and end-to-end data protection support
Posted by Klaus Jensen 3 years, 2 months ago
From: Klaus Jensen <k.jensen@samsung.com>

This is v4 (RFC removed) of a series that adds support for metadata and
end-to-end data protection.

First, on the subject of metadata, in v1, support was restricted to
extended logical blocks, which was pretty trivial to implement, but
required special initialization and broke DULBE. In v2, metadata is
always stored continuously at the end of the underlying block device.
This has the advantage of not breaking DULBE since the data blocks
remains aligned and allows bdrv_block_status to be used to determinate
allocation status. It comes at the expense of complicating the extended
LBA emulation, but on the other hand it also gains support for metadata
transfered as a separate buffer.

The end-to-end data protection support blew up in terms of required
changes. This is due to the fact that a bunch of new commands has been
added to the device since v1 (zone append, compare, copy), and they all
require various special handling for protection information. If
potential reviewers would like it split up into multiple patches, each
adding pi support to one command, shout out.

The core of the series (metadata and eedp) is preceeded by a set of
patches that refactors mapping (yes, again) and tries to deal with the
qsg/iov duality mess (maybe also again?).

Support fro metadata and end-to-end data protection is all joint work
with Gollu Appalanaidu.

v4:
  * promoted from RFC
  * moved most eedp additions to nvme-dif.{c,h}. (Keith)

v3:

  * added patch with Verify command
  * added patches for multiple LBA formats and Format NVM
  * changed NvmeSG to be a union (Keith)

Gollu Appalanaidu (1):
  hw/block/nvme: add verify command

Klaus Jensen (9):
  hw/block/nvme: remove redundant len member in compare context
  hw/block/nvme: remove block accounting for write zeroes
  hw/block/nvme: fix strerror printing
  hw/block/nvme: try to deal with the iov/qsg duality
  hw/block/nvme: remove the req dependency in map functions
  hw/block/nvme: refactor nvme_dma
  hw/block/nvme: add metadata support
  hw/block/nvme: end-to-end data protection
  hw/block/nvme: add non-mdts command size limit for verify

Minwoo Im (2):
  hw/block/nvme: support multiple lba formats
  hw/block/nvme: add support for the format nvm command

 hw/block/nvme-dif.h   |   53 ++
 hw/block/nvme-ns.h    |   49 +-
 hw/block/nvme.h       |   51 +-
 include/block/nvme.h  |   34 +-
 hw/block/nvme-dif.c   |  513 ++++++++++++++
 hw/block/nvme-ns.c    |   90 ++-
 hw/block/nvme.c       | 1548 ++++++++++++++++++++++++++++++++++-------
 hw/block/meson.build  |    2 +-
 hw/block/trace-events |   25 +-
 9 files changed, 2071 insertions(+), 294 deletions(-)
 create mode 100644 hw/block/nvme-dif.h
 create mode 100644 hw/block/nvme-dif.c

-- 
2.30.1


Re: [PATCH v4 00/12] hw/block/nvme: metadata and end-to-end data protection support
Posted by Keith Busch 3 years, 2 months ago
On Mon, Mar 01, 2021 at 03:00:35PM +0100, Klaus Jensen wrote:
> From: Klaus Jensen <k.jensen@samsung.com>
> 
> This is v4 (RFC removed) of a series that adds support for metadata and
> end-to-end data protection.
> 
> First, on the subject of metadata, in v1, support was restricted to
> extended logical blocks, which was pretty trivial to implement, but
> required special initialization and broke DULBE. In v2, metadata is
> always stored continuously at the end of the underlying block device.
> This has the advantage of not breaking DULBE since the data blocks
> remains aligned and allows bdrv_block_status to be used to determinate
> allocation status. It comes at the expense of complicating the extended
> LBA emulation, but on the other hand it also gains support for metadata
> transfered as a separate buffer.
> 
> The end-to-end data protection support blew up in terms of required
> changes. This is due to the fact that a bunch of new commands has been
> added to the device since v1 (zone append, compare, copy), and they all
> require various special handling for protection information. If
> potential reviewers would like it split up into multiple patches, each
> adding pi support to one command, shout out.
> 
> The core of the series (metadata and eedp) is preceeded by a set of
> patches that refactors mapping (yes, again) and tries to deal with the
> qsg/iov duality mess (maybe also again?).
> 
> Support fro metadata and end-to-end data protection is all joint work
> with Gollu Appalanaidu.

Looks fine.

Reviewed-by: Keith Busch <kbusch@kernel.org>

Re: [PATCH v4 00/12] hw/block/nvme: metadata and end-to-end data protection support
Posted by Klaus Jensen 3 years, 2 months ago
On Mar  1 15:00, Klaus Jensen wrote:
> From: Klaus Jensen <k.jensen@samsung.com>
> 
> This is v4 (RFC removed) of a series that adds support for metadata and
> end-to-end data protection.
> 
> First, on the subject of metadata, in v1, support was restricted to
> extended logical blocks, which was pretty trivial to implement, but
> required special initialization and broke DULBE. In v2, metadata is
> always stored continuously at the end of the underlying block device.
> This has the advantage of not breaking DULBE since the data blocks
> remains aligned and allows bdrv_block_status to be used to determinate
> allocation status. It comes at the expense of complicating the extended
> LBA emulation, but on the other hand it also gains support for metadata
> transfered as a separate buffer.
> 
> The end-to-end data protection support blew up in terms of required
> changes. This is due to the fact that a bunch of new commands has been
> added to the device since v1 (zone append, compare, copy), and they all
> require various special handling for protection information. If
> potential reviewers would like it split up into multiple patches, each
> adding pi support to one command, shout out.
> 
> The core of the series (metadata and eedp) is preceeded by a set of
> patches that refactors mapping (yes, again) and tries to deal with the
> qsg/iov duality mess (maybe also again?).
> 
> Support fro metadata and end-to-end data protection is all joint work
> with Gollu Appalanaidu.
> 
> v4:
>   * promoted from RFC
>   * moved most eedp additions to nvme-dif.{c,h}. (Keith)
> 
> v3:
> 
>   * added patch with Verify command
>   * added patches for multiple LBA formats and Format NVM
>   * changed NvmeSG to be a union (Keith)
> 
> Gollu Appalanaidu (1):
>   hw/block/nvme: add verify command
> 
> Klaus Jensen (9):
>   hw/block/nvme: remove redundant len member in compare context
>   hw/block/nvme: remove block accounting for write zeroes
>   hw/block/nvme: fix strerror printing
>   hw/block/nvme: try to deal with the iov/qsg duality
>   hw/block/nvme: remove the req dependency in map functions
>   hw/block/nvme: refactor nvme_dma
>   hw/block/nvme: add metadata support
>   hw/block/nvme: end-to-end data protection
>   hw/block/nvme: add non-mdts command size limit for verify
> 
> Minwoo Im (2):
>   hw/block/nvme: support multiple lba formats
>   hw/block/nvme: add support for the format nvm command
> 
>  hw/block/nvme-dif.h   |   53 ++
>  hw/block/nvme-ns.h    |   49 +-
>  hw/block/nvme.h       |   51 +-
>  include/block/nvme.h  |   34 +-
>  hw/block/nvme-dif.c   |  513 ++++++++++++++
>  hw/block/nvme-ns.c    |   90 ++-
>  hw/block/nvme.c       | 1548 ++++++++++++++++++++++++++++++++++-------
>  hw/block/meson.build  |    2 +-
>  hw/block/trace-events |   25 +-
>  9 files changed, 2071 insertions(+), 294 deletions(-)
>  create mode 100644 hw/block/nvme-dif.h
>  create mode 100644 hw/block/nvme-dif.c
> 
> -- 
> 2.30.1
> 

Applied patches 1 through 6. I'll let the metadata and eedp patches
simmer for a few days more.