From nobody Mon Feb 9 19:04:50 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1505816823484365.83992100755313; Tue, 19 Sep 2017 03:27:03 -0700 (PDT) Received: from localhost ([::1]:41293 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1duFk6-0005db-Dn for importer@patchew.org; Tue, 19 Sep 2017 06:27:02 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45335) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1duFhz-00049r-IK for qemu-devel@nongnu.org; Tue, 19 Sep 2017 06:24:53 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1duFhx-0004dV-3q for qemu-devel@nongnu.org; Tue, 19 Sep 2017 06:24:51 -0400 Received: from mx1.redhat.com ([209.132.183.28]:50680) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1duFhm-0004Vc-BN; Tue, 19 Sep 2017 06:24:38 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 758D7267CA; Tue, 19 Sep 2017 10:24:37 +0000 (UTC) Received: from donizetti.redhat.com (ovpn-117-61.ams2.redhat.com [10.36.117.61]) by smtp.corp.redhat.com (Postfix) with ESMTP id 738D55D6A4; Tue, 19 Sep 2017 10:24:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 758D7267CA Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=pbonzini@redhat.com From: Paolo Bonzini To: qemu-devel@nongnu.org Date: Tue, 19 Sep 2017 12:24:31 +0200 Message-Id: <20170919102434.21147-2-pbonzini@redhat.com> In-Reply-To: <20170919102434.21147-1-pbonzini@redhat.com> References: <20170919102434.21147-1-pbonzini@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Tue, 19 Sep 2017 10:24:37 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 1/4] scsi, file-posix: add support for persistent reservation management X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-block@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" It is a common requirement for virtual machine to send persistent reservations, but this currently requires either running QEMU with CAP_SYS_RAWIO, or using out-of-tree patches that let an unprivileged QEMU bypass Linux's filter on SG_IO commands. As an alternative mechanism, the next patches will introduce a privileged helper to run persistent reservation commands without expanding QEMU's attack surface unnecessarily. The helper is invoked through a "pr-manager" QOM object, to which file-posix.c passes SG_IO requests for PERSISTENT RESERVE OUT and PERSISTENT RESERVE IN commands. For example: $ qemu-system-x86_64 -device virtio-scsi \ -object pr-manager-helper,id=3Dhelper0,path=3D/var/run/qemu-pr-helper= .sock -drive if=3Dnone,id=3Dhd,driver=3Draw,file.filename=3D/dev/sdb,file.p= r-manager=3Dhelper0 -device scsi-block,drive=3Dhd or: $ qemu-system-x86_64 -device virtio-scsi \ -object pr-manager-helper,id=3Dhelper0,path=3D/var/run/qemu-pr-helper= .sock -blockdev node-name=3Dhd,driver=3Draw,file.driver=3Dhost_device,file.= filename=3D/dev/sdb,file.pr-manager=3Dhelper0 -device scsi-block,drive=3Dhd Multiple pr-manager implementations are conceivable and possible, though only one is implemented right now. For example, a pr-manager could: - talk directly to the multipath daemon from a privileged QEMU (i.e. QEMU links to libmpathpersist); this makes reservation work properly with multipath, but still requires CAP_SYS_RAWIO - use the Linux IOC_PR_* ioctls (they require CAP_SYS_ADMIN though) - more interestingly, implement reservations directly in QEMU through file system locks or a shared database (e.g. sqlite) Signed-off-by: Paolo Bonzini --- Makefile.objs | 1 + block/file-posix.c | 30 +++++++++++++ docs/pr-manager.rst | 51 ++++++++++++++++++++++ include/scsi/pr-manager.h | 56 ++++++++++++++++++++++++ qapi/block-core.json | 4 ++ scsi/Makefile.objs | 2 + scsi/pr-manager.c | 109 ++++++++++++++++++++++++++++++++++++++++++= ++++ vl.c | 3 +- 8 files changed, 255 insertions(+), 1 deletion(-) create mode 100644 docs/pr-manager.rst create mode 100644 include/scsi/pr-manager.h create mode 100644 scsi/pr-manager.c diff --git a/Makefile.objs b/Makefile.objs index 0caa8a5cf8..12abaa6191 100644 --- a/Makefile.objs +++ b/Makefile.objs @@ -170,6 +170,7 @@ trace-events-subdirs +=3D qapi trace-events-subdirs +=3D accel/tcg trace-events-subdirs +=3D accel/kvm trace-events-subdirs +=3D nbd +trace-events-subdirs +=3D scsi =20 trace-events-files =3D $(SRC_PATH)/trace-events $(trace-events-subdirs:%= =3D$(SRC_PATH)/%/trace-events) =20 diff --git a/block/file-posix.c b/block/file-posix.c index 6acbd56238..ab12a2b591 100644 --- a/block/file-posix.c +++ b/block/file-posix.c @@ -33,6 +33,9 @@ #include "block/raw-aio.h" #include "qapi/qmp/qstring.h" =20 +#include "scsi/pr-manager.h" +#include "scsi/constants.h" + #if defined(__APPLE__) && (__MACH__) #include #include @@ -155,6 +158,8 @@ typedef struct BDRVRawState { bool page_cache_inconsistent:1; bool has_fallocate; bool needs_alignment; + + PRManager *pr_mgr; } BDRVRawState; =20 typedef struct BDRVRawReopenState { @@ -402,6 +407,11 @@ static QemuOptsList raw_runtime_opts =3D { .type =3D QEMU_OPT_STRING, .help =3D "file locking mode (on/off/auto, default: auto)", }, + { + .name =3D "pr-manager", + .type =3D QEMU_OPT_STRING, + .help =3D "id of persistent reservation manager object (defaul= t: none)", + }, { /* end of list */ } }, }; @@ -413,6 +423,7 @@ static int raw_open_common(BlockDriverState *bs, QDict = *options, QemuOpts *opts; Error *local_err =3D NULL; const char *filename =3D NULL; + const char *str; BlockdevAioOptions aio, aio_default; int fd, ret; struct stat st; @@ -476,6 +487,16 @@ static int raw_open_common(BlockDriverState *bs, QDict= *options, abort(); } =20 + str =3D qemu_opt_get(opts, "pr-manager"); + if (str) { + s->pr_mgr =3D pr_manager_lookup(str, &local_err); + if (local_err) { + error_propagate(errp, local_err); + ret =3D -EINVAL; + goto fail; + } + } + s->open_flags =3D open_flags; raw_parse_flags(bdrv_flags, &s->open_flags); =20 @@ -2597,6 +2618,15 @@ static BlockAIOCB *hdev_aio_ioctl(BlockDriverState *= bs, if (fd_open(bs) < 0) return NULL; =20 + if (req =3D=3D SG_IO && s->pr_mgr) { + struct sg_io_hdr *io_hdr =3D buf; + if (io_hdr->cmdp[0] =3D=3D PERSISTENT_RESERVE_OUT || + io_hdr->cmdp[0] =3D=3D PERSISTENT_RESERVE_IN) { + return pr_manager_execute(s->pr_mgr, bdrv_get_aio_context(bs), + s->fd, io_hdr, cb, opaque); + } + } + acb =3D g_new(RawPosixAIOData, 1); acb->bs =3D bs; acb->aio_type =3D QEMU_AIO_IOCTL; diff --git a/docs/pr-manager.rst b/docs/pr-manager.rst new file mode 100644 index 0000000000..b6089fb57c --- /dev/null +++ b/docs/pr-manager.rst @@ -0,0 +1,51 @@ +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +Persistent reservation managers +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +SCSI persistent Reservations allow restricting access to block devices +to specific initiators in a shared storage setup. When implementing +clustering of virtual machines, it is a common requirement for virtual +machines to send persistent reservation SCSI commands. However, +the operating system restricts sending these commands to unprivileged +programs because incorrect usage can disrupt regular operation of the +storage fabric. + +For this reason, QEMU's SCSI passthrough devices, ``scsi-block`` +and ``scsi-generic`` (both are only available on Linux) can delegate +implementation of persistent reservations to a separate object, +the "persistent reservation manager". Only PERSISTENT RESERVE OUT and +PERSISTENT RESERVE IN commands are passed to the persistent reservation +manager object; other commands are processed by QEMU as usual. + +----------------------------------------- +Defining a persistent reservation manager +----------------------------------------- + +A persistent reservation manager is an instance of a subclass of the +"pr-manager" QOM class. + +Right now only one subclass is defined, ``pr-manager-helper``, which +forwards the commands to an external privileged helper program +over Unix sockets. The helper program only allows sending persistent +reservation commands to devices for which QEMU has a file descriptor, +so that QEMU will not be able to effect persistent reservations +unless it has access to both the socket and the device. + +``pr-manager-helper`` has a single string property, ``path``, which +accepts the path to the helper program's Unix socket. For example, +the following command line defines a ``pr-manager-helper`` object and +attaches it to a SCSI passthrough device:: + + $ qemu-system-x86_64 + -device virtio-scsi \ + -object pr-manager-helper,id=3Dhelper0,path=3D/var/run/qemu-pr-h= elper.sock + -drive if=3Dnone,id=3Dhd,driver=3Draw,file.filename=3D/dev/sdb,f= ile.pr-manager=3Dhelper0 + -device scsi-block,drive=3Dhd + +Alternatively, using ``-blockdev``:: + + $ qemu-system-x86_64 + -device virtio-scsi \ + -object pr-manager-helper,id=3Dhelper0,path=3D/var/run/qemu-pr-h= elper.sock + -blockdev node-name=3Dhd,driver=3Draw,file.driver=3Dhost_device,= file.filename=3D/dev/sdb,file.pr-manager=3Dhelper0 + -device scsi-block,drive=3Dhd diff --git a/include/scsi/pr-manager.h b/include/scsi/pr-manager.h new file mode 100644 index 0000000000..b2b37d63bc --- /dev/null +++ b/include/scsi/pr-manager.h @@ -0,0 +1,56 @@ +#ifndef PR_MANAGER_H +#define PR_MANAGER_H + +#include "qom/object.h" +#include "qapi/qmp/qdict.h" +#include "qapi/visitor.h" +#include "qom/object_interfaces.h" +#include "block/aio.h" + +#define TYPE_PR_MANAGER "pr-manager" + +#define PR_MANAGER_CLASS(klass) \ + OBJECT_CLASS_CHECK(PRManagerClass, (klass), TYPE_PR_MANAGER) +#define PR_MANAGER_GET_CLASS(obj) \ + OBJECT_GET_CLASS(PRManagerClass, (obj), TYPE_PR_MANAGER) +#define PR_MANAGER(obj) \ + OBJECT_CHECK(PRManager, (obj), TYPE_PR_MANAGER) + +struct sg_io_hdr; + +typedef struct PRManager { + /* */ + Object parent; +} PRManager; + +/** + * PRManagerClass: + * @parent_class: the base class + * @run: callback invoked in thread pool context + */ +typedef struct PRManagerClass { + /* */ + ObjectClass parent_class; + + /* */ + int (*run)(PRManager *pr_mgr, int fd, struct sg_io_hdr *hdr); +} PRManagerClass; + +BlockAIOCB *pr_manager_execute(PRManager *pr_mgr, + AioContext *ctx, int fd, + struct sg_io_hdr *hdr, + BlockCompletionFunc *complete, + void *opaque); + +#ifdef CONFIG_LINUX +PRManager *pr_manager_lookup(const char *id, Error **errp); +#else +static inline PRManager *pr_manager_lookup(const char *id, Error **errp) +{ + /* The classes do not exist at all! */ + error_setg(errp, "No persistent reservation manager with id '%s'", id); + return NULL; +} +#endif + +#endif diff --git a/qapi/block-core.json b/qapi/block-core.json index bb11815608..c69a395804 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -2241,6 +2241,9 @@ # Driver specific block device options for the file backend. # # @filename: path to the image file +# @pr-manager: the id for the object that will handle persistent reservat= ions +# for this device (default: none, forward the commands via S= G_IO; +# since 2.11) # @aio: AIO backend (default: threads) (since: 2.8) # @locking: whether to enable file locking. If set to 'auto', only ena= ble # when Open File Descriptor (OFD) locking API is available @@ -2250,6 +2253,7 @@ ## { 'struct': 'BlockdevOptionsFile', 'data': { 'filename': 'str', + '*pr-manager': 'str', '*locking': 'OnOffAuto', '*aio': 'BlockdevAioOptions' } } =20 diff --git a/scsi/Makefile.objs b/scsi/Makefile.objs index 31b82a5a36..5496d2ae6a 100644 --- a/scsi/Makefile.objs +++ b/scsi/Makefile.objs @@ -1 +1,3 @@ block-obj-y +=3D utils.o + +block-obj-$(CONFIG_LINUX) +=3D pr-manager.o diff --git a/scsi/pr-manager.c b/scsi/pr-manager.c new file mode 100644 index 0000000000..cde19b87c3 --- /dev/null +++ b/scsi/pr-manager.c @@ -0,0 +1,109 @@ +/* + * Persistent reservation manager abstract class + * + * Copyright (c) 2017 Red Hat, Inc. + * + * Author: Paolo Bonzini + * + * This code is licensed under the LGPL. + * + */ + +#include "qemu/osdep.h" +#include + +#include "qapi/error.h" +#include "block/aio.h" +#include "block/thread-pool.h" +#include "scsi/pr-manager.h" +#include "trace.h" + +typedef struct PRManagerData { + PRManager *pr_mgr; + struct sg_io_hdr *hdr; + int fd; +} PRManagerData; + +static int pr_manager_worker(void *opaque) +{ + PRManagerData *data =3D opaque; + PRManager *pr_mgr =3D data->pr_mgr; + PRManagerClass *pr_mgr_class =3D + PR_MANAGER_GET_CLASS(pr_mgr); + struct sg_io_hdr *hdr =3D data->hdr; + int fd =3D data->fd; + int r; + + g_free(data); + trace_pr_manager_run(fd, hdr->cmdp[0], hdr->cmdp[1]); + + /* The is was taken in pr_manager_execute. */ + r =3D pr_mgr_class->run(pr_mgr, fd, hdr); + object_unref(OBJECT(pr_mgr)); + return r; +} + + +BlockAIOCB *pr_manager_execute(PRManager *pr_mgr, + AioContext *ctx, int fd, + struct sg_io_hdr *hdr, + BlockCompletionFunc *complete, + void *opaque) +{ + PRManagerData *data =3D g_new(PRManagerData, 1); + ThreadPool *pool =3D aio_get_thread_pool(ctx); + + trace_pr_manager_execute(fd, hdr->cmdp[0], hdr->cmdp[1], opaque); + data->pr_mgr =3D pr_mgr; + data->fd =3D fd; + data->hdr =3D hdr; + + /* The matching object_unref is in pr_manager_worker. */ + object_ref(OBJECT(pr_mgr)); + return thread_pool_submit_aio(pool, pr_manager_worker, + data, complete, opaque); +} + +static const TypeInfo pr_manager_info =3D { + .parent =3D TYPE_OBJECT, + .name =3D TYPE_PR_MANAGER, + .class_size =3D sizeof(PRManagerClass), + .abstract =3D true, + .interfaces =3D (InterfaceInfo[]) { + { TYPE_USER_CREATABLE }, + { } + } +}; + +PRManager *pr_manager_lookup(const char *id, Error **errp) +{ + Object *obj; + PRManager *pr_mgr; + + obj =3D object_resolve_path_component(object_get_objects_root(), id); + if (!obj) { + error_setg(errp, "No persistent reservation manager with id '%s'",= id); + return NULL; + } + + pr_mgr =3D (PRManager *) + object_dynamic_cast(obj, + TYPE_PR_MANAGER); + if (!pr_mgr) { + error_setg(errp, + "Object with id '%s' is not a persistent reservation ma= nager", + id); + return NULL; + } + + return pr_mgr; +} + +static void +pr_manager_register_types(void) +{ + type_register_static(&pr_manager_info); +} + + +type_init(pr_manager_register_types); diff --git a/vl.c b/vl.c index 9e62e92aea..bfee61053b 100644 --- a/vl.c +++ b/vl.c @@ -2893,7 +2893,8 @@ static int machine_set_property(void *opaque, */ static bool object_create_initial(const char *type) { - if (g_str_equal(type, "rng-egd")) { + if (g_str_equal(type, "rng-egd") || + g_str_has_prefix(type, "pr-manager-")) { return false; } =20 --=20 2.13.5