From nobody Mon Feb 9 09:15:58 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1649340069; cv=none; d=zohomail.com; s=zohoarc; b=g0Tp/ySnk8CrCn56IkfoFqEx+WpAZDTjc41uMp7X43FkayhhEeYii/AFNLVh7CZDSHdhWdbSh9KL5udAZRRfUlxIikWSPffQ/gIkE4vLrZ5ATSaDTyf5DPEesgp+kLNgGwbVxcX5OCLYVVRbtQwes2eKsPZ3cZIKJWbeTUdJ3W0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1649340069; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=2DQo/suXzBNebHq2VVb03TE2J9ma0DgHX8guaIDjRzM=; b=brJykrIo/GVG3nKGUQARG5n5qLaWa8NWBVqyKh9D5yB/jFeZAHON9ttovT3I3v/ONn38BX7UnHQaVgfDDtstsLIlSmrJMxVrI5SGFdha/+0whPTfnRuFvdxJ6/E5oXpup9R45DsII/KcB/7/UXwaervNg1Kpg4iIrkWLAs5WahU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1649340069791757.9990884244692; Thu, 7 Apr 2022 07:01:09 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-674-4NMS1yl7Plm4PwLoa1dI0Q-1; Thu, 07 Apr 2022 10:01:04 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 329ED899ECF; Thu, 7 Apr 2022 14:00:44 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1AD014118EEB; Thu, 7 Apr 2022 14:00:44 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id E28241949761; Thu, 7 Apr 2022 14:00:43 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 640AA1947BBE for ; Thu, 7 Apr 2022 14:00:41 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 55173403152; Thu, 7 Apr 2022 14:00:41 +0000 (UTC) Received: from speedmetal.lan (unknown [10.40.208.35]) by smtp.corp.redhat.com (Postfix) with ESMTP id E9EBD416374 for ; Thu, 7 Apr 2022 14:00:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1649340068; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=2DQo/suXzBNebHq2VVb03TE2J9ma0DgHX8guaIDjRzM=; b=amxqiI8r8a1s4IEXOuU53dRUdiKSMPXPHrlT20/SuZK9VJXdw+1qa/vsjmgjj0hIXc47x7 mhE7wcSVYGp1MBMuROzOCNT2S+CH1rUst6hNySzlse9cdWWruVYhimEO7FxKENWNUNZBfT dHNs2xhTw74+mq0FNmXHYzZHZjYOLdQ= X-MC-Unique: 4NMS1yl7Plm4PwLoa1dI0Q-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Peter Krempa To: libvir-list@redhat.com Subject: [PATCH 05/11] docs: Convert 'internals/command' to rst and move it to 'kbase/internals' Date: Thu, 7 Apr 2022 16:00:27 +0200 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.84 on 10.11.54.1 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1649340071652100005 Content-Type: text/plain; charset="utf-8"; x-default="true" Signed-off-by: Peter Krempa --- docs/docs.rst | 3 - docs/internals/command.html.in | 596 ------------------------------- docs/internals/meson.build | 1 - docs/kbase/index.rst | 3 + docs/kbase/internals/command.rst | 465 ++++++++++++++++++++++++ docs/kbase/internals/meson.build | 1 + 6 files changed, 469 insertions(+), 600 deletions(-) delete mode 100644 docs/internals/command.html.in create mode 100644 docs/kbase/internals/command.rst diff --git a/docs/docs.rst b/docs/docs.rst index 299c26d09b..a92c7c26ab 100644 --- a/docs/docs.rst +++ b/docs/docs.rst @@ -157,9 +157,6 @@ Project development `Event loop and worker pool `__ Libvirt's event loop and worker pool mode -`Spawning commands `__ - Spawning commands from libvirt driver code - `RPC protocol & APIs `__ RPC protocol information and API / dispatch guide diff --git a/docs/internals/command.html.in b/docs/internals/command.html.in deleted file mode 100644 index d9f53933c6..0000000000 --- a/docs/internals/command.html.in +++ /dev/null @@ -1,596 +0,0 @@ - - - - -

Spawning processes / commands from libvirt drivers

- -
    - -

    - This page describes the usage of libvirt APIs for - spawning processes / commands from libvirt drivers. - All code is required to use these APIs -

    - -

    Problems with standard POSIX APIs

    - -

    - The POSIX specification includes a number of APIs for - spawning processes / commands, but they suffer from - a number of flaws -

    - -
      -
    • fork+exec: The lowest & most flexible - level, but very hard to use correctly / safely. It - is easy to leak file descriptors, have unexpected - signal handler behaviour and not handle edge cases. - Furthermore, it is not portable to mingw. -
      • -
    • system: Convenient if you don't care - about capturing command output, but has the serious - downside that the command string is interpreted by - the shell. This makes it very dangerous to use, because - improperly validated user input can lead to exploits - via shell meta characters. -
      • -
    • popen: Inherits the flaws of - system, and has no option for bi-directional - communication. -
      • -
    • posix_spawn: A half-way house between - simplicity of system() and the flexibility of fork+exec. - It does not allow for a couple of important features - though, such as running a hook between the fork+exec - stage, or closing all open file descriptors.
      • -
    - -

    - Due to the problems mentioned with each of these, - libvirt driver code must not use any - of the above APIs. Historically libvirt provided a - higher level API known as virExec. This was wrapper - around fork+exec, in a similar style to posix_spawn, - but with a few more features. -

    - -

    - This wrapper still suffered from a number of problems. - Handling command cleanup via waitpid() is overly - complex & error prone for most usage. Building up the - argv[] + env[] string arrays is quite cumbersome and - error prone, particularly wrt memory leak / OOM handling. -

    - -

    The libvirt command execution API

    - -

    - There is now a high level API that provides a safe and - flexible way to spawn commands, which prevents the most - common errors & is easy to code against. This - code is provided in the src/util/vircommand.h - header which can be imported using #include "vircommand.h" -

    - -

    Defining commands in libvirt

    - -

    - The first step is to declare what command is to be - executed. The command name can be either a fully - qualified path, or a bare command name. In the latter - case it will be resolved wrt the $PATH - environment variable. -

    - -
    -virCommand *cmd =3D virCommandNew("/usr/bin/dnsmasq");
-
    - -

    - There is no need to check for allocation failure after - virCommandNew. This will be detected and - reported at a later time. -

    - -

    Adding arguments to the command

    - -

    - There are a number of APIs for adding arguments to a - command. To add a direct string arg -

    - -
    -virCommandAddArg(cmd, "-strict-order");
-
    - -

    - If an argument takes an attached value of the form - -arg=3Dval, then this can be done using -

    - -
    -virCommandAddArgPair(cmd, "--conf-file", "/etc/dnsmasq.conf");
-
    - -

    - If an argument needs to be formatted as if by - printf: -

    - -
    -virCommandAddArgFormat(cmd, "%d", count);
-
    - -

    - To add an entire NULL terminated array of arguments in one go, - there are two options. -

    - -
    -const char *const args[] =3D {
-    "--strict-order", "--except-interface", "lo", NULL
-};
-virCommandAddArgSet(cmd, args);
-virCommandAddArgList(cmd, "--domain", "localdomain", NULL);
-
    - -

    - This can also be done at the time of initial construction of - the virCommand * object: -

    - -
    -const char *const args[] =3D {
-    "/usr/bin/dnsmasq",
-    "--strict-order", "--except-interface",
-    "lo", "--domain", "localdomain", NULL
-};
-virCommand *cmd1 =3D virCommandNewArgs(cmd, args);
-virCommand *cmd2 =3D virCommandNewArgList("/usr/bin/dnsmasq",
-                                          "--domain", "localdomain", NULL);
-
    - -

    Setting up the environment

    - -

    - By default a command will inherit all environment variables - from the current process. Generally this is not desirable - and a customized environment will be more suitable. Any - customization done via the following APIs will prevent - inheritance of any existing environment variables unless - explicitly allowed. The first step is usually to pass through - a small number of variables from the current process. -

    - -
    -virCommandAddEnvPassCommon(cmd);
-
    - -

    - This has now set up a clean environment for the child, passing - through PATH, LD_PRELOAD, - LD_LIBRARY_PATH, HOME, - USER, LOGNAME and TMPDIR. - Furthermore it will explicitly set LC_ALL=3DC to - avoid unexpected localization of command output. Further - variables can be passed through from parent explicitly: -

    - -
    -virCommandAddEnvPass(cmd, "DISPLAY");
-virCommandAddEnvPass(cmd, "XAUTHORITY");
-
    - -

    - To define an environment variable in the child with an - separate key / value: -

    - -
    -virCommandAddEnvPair(cmd, "TERM", "xterm");
-
    - -

    - If the key/value pair is pre-formatted in the right - format, it can be set directly -

    - -
    -virCommandAddEnvString(cmd, "TERM=3Dxterm");
-
    - -

    Miscellaneous other options

    - -

    - Normally the spawned command will retain the current - process and process group as its parent. If the current - process dies, the child will then (usually) be terminated - too. If this cleanup is not desired, then the command - should be marked as daemonized: -

    - -
    -virCommandDaemonize(cmd);
-
    - -

    - When daemonizing a command, the PID visible from the - caller will be that of the intermediate process, not - the actual damonized command. If the PID of the real - command is required then a pidfile can be requested -

    - -
    -virCommandSetPidFile(cmd, "/var/run/dnsmasq.pid");
-
    - -

    - This PID file is guaranteed to be written before - the intermediate process exits. Moreover, the daemonized - process will inherit the FD of the opened and locked PID - file. -

    - -

    Reducing command privileges

    - -

    - Normally a command will inherit all privileges of - the current process. To restrict what a command can - do, it is possible to request that all its capabilities - are cleared. With this done it will only be able to - access resources for which it has explicit DAC permissions -

    - -
    -virCommandClearCaps(cmd);
-
    - -

    Managing file handles

    - -

    - To prevent unintended resource leaks to child processes, the - child defaults to closing all open file handles, and setting - stdin/out/err to /dev/null. It is possible to - allow an open file handle to be passed into the child, while - controlling whether that handle remains open in the parent or - guaranteeing that the handle will be closed in the parent after - virCommandRun, virCommandRunAsync, or virCommandFree. -

    - -
    -int sharedfd =3D open("cmd.log", "w+");
-int childfd =3D open("conf.txt", "r");
-virCommandPassFD(cmd, sharedfd, 0);
-virCommandPassFD(cmd, childfd,
-                 VIR_COMMAND_PASS_FD_CLOSE_PARENT);
-if (VIR_CLOSE(sharedfd) < 0)
-    goto cleanup;
-
    - -

    - With this, both file descriptors sharedfd and childfd in the - current process remain open as the same file descriptors in the - child. Meanwhile, after the child is spawned, sharedfd remains - open in the parent, while childfd is closed. -

    - -

    - For stdin/out/err it is sometimes necessary to map a file - handle. If a mapped file handle is a pipe fed or consumed by - the caller, then the caller should use virCommandDaemonize or - virCommandRunAsync rather than virCommandRun to avoid deadlock - (mapping a regular file is okay with virCommandRun). To attach - file descriptor 7 in the current process to stdin in the child: -

    - -
    -virCommandSetInputFD(cmd, 7);
-
    - -

    - Equivalently to redirect stdout or stderr in the child, - pass in a pointer to the desired handle -

    - -
    -int outfd =3D open("out.log", "w+");
-int errfd =3D open("err.log", "w+");
-virCommandSetOutputFD(cmd, &outfd);
-virCommandSetErrorFD(cmd, &errfd);
-
    - -

    - Alternatively it is possible to request that a pipe be - created to fetch stdout/err in the parent, by initializing - the FD to -1. -

    - -
    -int outfd =3D -1;
-int errfd =3D -1
-virCommandSetOutputFD(cmd, &outfd);
-virCommandSetErrorFD(cmd, &errfd);
-
    - -

    - Once the command is running, outfd - and errfd will be initialized with - valid file handles that can be read from. It is - permissible to pass the same pointer for both outfd - and errfd, in which case both standard streams in - the child will share the same fd in the parent. -

    - -

    - Normally, file descriptors opened to collect output from a child - process perform blocking I/O, but the parent process can request - non-blocking mode: -

    - -
    -virCommandNonblockingFDs(cmd);
-
    - -

    Feeding & capturing strings to/from the chil= d

    - -

    - Often dealing with file handles for stdin/out/err is - unnecessarily complex; an alternative is to let virCommandRun - perform the I/O and interact via string buffers. Use of a buffer - only works with virCommandRun, and cannot be mixed with pipe - file descriptors. That is, the choice is generally between - managing all I/O in the caller (any fds not specified are tied - to /dev/null), or letting virCommandRun manage all I/O via - strings (unspecified stdin is tied to /dev/null, and unspecified - output streams get logged but are otherwise discarded). -

    - -

    - It is possible to specify a string buffer to act as the data - source for the child's stdin, if there are no embedded NUL - bytes, and if the command will be run with virCommandRun: -

    - -
    -const char *input =3D "Hello World\n";
-virCommandSetInputBuffer(cmd, input);
-
    - -

    - Similarly it is possible to request that the child's - stdout/err be redirected into a string buffer, if the - output is not expected to contain NUL bytes, and if - the command will be run with virCommandRun: -

    - -
    -char *output =3D NULL, *errors =3D NULL;
-virCommandSetOutputBuffer(cmd, &output);
-virCommandSetErrorBuffer(cmd, &errors);
-
    - -

    - Once the command has finished executing, these buffers will - contain the output. Allocation is guaranteed if virCommandRun - or virCommandWait succeed (if there was no output, then the - buffer will contain an allocated empty string); if the command - failed, then the buffers usually contain a best-effort - allocation of collected information (however, on an - out-of-memory condition, the buffer may still be NULL). The - caller is responsible for freeing registered buffers, since the - buffers are designed to persist beyond virCommandFree. It - is possible to pass the same pointer to both - virCommandSetOutputBuffer and virCommandSetErrorBuffer, in which - case the child process interleaves output into a single string. -

    - -

    Setting working directory

    - -

    - Daemonized commands are always run with "/" as the current - working directory. All other commands default to running in the - same working directory as the parent process, but an alternate - directory can be specified: -

    - -
    -virCommandSetWorkingDirectory(cmd, LOCALSTATEDIR);
-
    - -

    Any additional hooks

    - -

    - If anything else is needed, it is possible to request a hook - function that is called in the child after the fork, as the - last thing before changing directories, dropping capabilities, - and executing the new process. If hook(opaque) returns - non-zero, then the child process will not be run. -

    - -
    -virCommandSetPreExecHook(cmd, hook, opaque);
-
    - -

    Logging commands

    - -

    - Sometimes, it is desirable to log what command will be run, or - even to use virCommand solely for creation of a single - consolidated string without running anything. -

    - -
    -int logfd =3D ...;
-char *timestamp =3D virTimestamp();
-char *string =3D NULL;
-
-dprintf(logfd, "%s: ", timestamp);
-VIR_FREE(timestamp);
-virCommandWriteArgLog(cmd, logfd);
-
-string =3D virCommandToString(cmd, false);
-if (string)
-    VIR_DEBUG("about to run %s", string);
-VIR_FREE(string);
-if (virCommandRun(cmd, NULL) < 0)
-    return -1;
-
    - -

    Running commands synchronously

    - -

    - For most commands, the desired behaviour is to spawn - the command, wait for it to complete & exit and then - check that its exit status is zero -

    - -
    -if (virCommandRun(cmd, NULL) < 0)
-   return -1;
-
    - -

    - Note: if the command has been daemonized - this will only block & wait for the intermediate process, - not the real command. virCommandRun will - report on any errors that have occurred upon this point - with all previous API calls. If the command fails to - run, or exits with non-zero status an error will be - reported via normal libvirt error infrastructure. If a - non-zero exit status can represent a success condition, - it is possible to request the exit status and perform - that check manually instead of letting virCommandRun - raise the error. By default, the captured status is only - for a normal exit (death from a signal is treated as an error), - but a caller can use virCommandRawStatus to get - encoded status that includes any terminating signals. -

    - -
    -int status;
-if (virCommandRun(cmd, &status) < 0)
-    return -1;
-if (status =3D=3D 1) {
-  ...do stuff...
-}
-
-virCommandRawStatus(cmd2);
-if (virCommandRun(cmd2, &status) < 0)
-    return -1;
-if (WIFEXITED(status) && WEXITSTATUS(status) =3D=3D 1) {
-  ...do stuff...
-}
-
    - -

    Running commands asynchronously

    - -

    - In certain complex scenarios, particularly special - I/O handling is required for the child's stdin/err/out - it will be necessary to run the command asynchronously - and wait for completion separately. -

    - -
    -pid_t pid;
-if (virCommandRunAsync(cmd, &pid) < 0)
-   return -1;
-
-... do something while pid is running ...
-
-int status;
-if (virCommandWait(cmd, &status) < 0)
-   return -1;
-
-if (WEXITSTATUS(status)...) {
-   ..do stuff..
-}
-
    - -

    - As with virCommandRun, the status - arg for virCommandWait can be omitted, in which - case it will validate that exit status is zero and raise an - error if not. -

    - -

    - There are two approaches to child process cleanup, determined by - how long you want to keep the virCommand object in scope. -

    - -

    1. If the virCommand object will outlast the child process, - then pass NULL for the pid argument, and the child process will - automatically be reaped at virCommandFree, unless you reap it - sooner via virCommandWait or virCommandAbort. -

    - -

    2. If the child process must exist on at least one code path - after virCommandFree, then pass a pointer for the pid argument. - Later, to clean up the child, call virPidWait or virPidAbort. - Before virCommandFree, you can still use virCommandWait or - virCommandAbort to reap the process. -

    - -

    Releasing resources

    - -

    - Once the command has been executed, or if execution - has been abandoned, it is necessary to release - resources associated with the virCommand * - object. This is done with: -

    - -
    -virCommandFree(cmd);
-
    - -

    - There is no need to check if cmd is NULL - before calling virCommandFree. This scenario - is handled automatically. If the command is still running, - it will be forcibly killed and cleaned up (via waitpid). -

    - -

    Complete examples

    - -

    - This shows a complete example usage of the APIs roughly - using the libvirt source src/util/hooks.c -

    - -
    -int runhook(const char *drvstr, const char *id,
-            const char *opstr, const char *subopstr,
-            const char *extra)
-{
-  g_autofree char *path =3D NULL;
-  g_autoptr(virCommand) cmd =3D NULL;
-
-  virBuildPath(&path, LIBVIRT_HOOK_DIR, drvstr);
-
-  cmd =3D virCommandNew(path);
-
-  virCommandAddEnvPassCommon(cmd);
-
-  virCommandAddArgList(cmd, id, opstr, subopstr, extra, NULL);
-
-  virCommandSetInputBuffer(cmd, input);
-
-  return virCommandRun(cmd, NULL);
-}
-
    - -

    - In this example, the command is being run synchronously. - A pre-formatted string is being fed to the command as - its stdin. The command takes four arguments, and has a - minimal set of environment variables passed down. In - this example, the code does not require any error checking. - All errors are reported by the virCommandRun - method, and the exit status from this is returned to - the caller to handle as desired. -

    - - - diff --git a/docs/internals/meson.build b/docs/internals/meson.build index 298a55dd88..e5f4bb0a4b 100644 --- a/docs/internals/meson.build +++ b/docs/internals/meson.build @@ -1,5 +1,4 @@ internals_in_files =3D [ - 'command', 'eventloop', 'locking', 'rpc', diff --git a/docs/kbase/index.rst b/docs/kbase/index.rst index c6748e8883..01ec5a070d 100644 --- a/docs/kbase/index.rst +++ b/docs/kbase/index.rst @@ -85,3 +85,6 @@ Internals `VM migration internals `__ VM migration implementation details, complementing the info in `migration <../migration.html>`__ + +`Spawning commands `__ + Spawning commands from libvirt driver code diff --git a/docs/kbase/internals/command.rst b/docs/kbase/internals/comman= d.rst new file mode 100644 index 0000000000..738fb5930a --- /dev/null +++ b/docs/kbase/internals/command.rst @@ -0,0 +1,465 @@ +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D +Spawning processes / commands from libvirt drivers +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D + +.. contents:: + +This page describes the usage of libvirt APIs for spawning processes / com= mands +from libvirt drivers. All code is required to use these APIs + +Problems with standard POSIX APIs +--------------------------------- + +The POSIX specification includes a number of APIs for spawning processes / +commands, but they suffer from a number of flaws + +- ``fork+exec``: The lowest & most flexible level, but very hard to use + correctly / safely. It is easy to leak file descriptors, have unexpected + signal handler behaviour and not handle edge cases. Furthermore, it is = not + portable to mingw. +- ``system``: Convenient if you don't care about capturing command output= , but + has the serious downside that the command string is interpreted by the = shell. + This makes it very dangerous to use, because improperly validated user = input + can lead to exploits via shell meta characters. +- ``popen``: Inherits the flaws of ``system``, and has no option for + bi-directional communication. +- ``posix_spawn``: A half-way house between simplicity of system() and the + flexibility of fork+exec. It does not allow for a couple of important + features though, such as running a hook between the fork+exec stage, or + closing all open file descriptors. + +Due to the problems mentioned with each of these, libvirt driver code **mu= st not +use** any of the above APIs. Historically libvirt provided a higher level = API +known as virExec. This was wrapper around fork+exec, in a similar style to +posix_spawn, but with a few more features. + +This wrapper still suffered from a number of problems. Handling command cl= eanup +via waitpid() is overly complex & error prone for most usage. Building up = the +argv[] + env[] string arrays is quite cumbersome and error prone, particul= arly +wrt memory leak / OOM handling. + +The libvirt command execution API +--------------------------------- + +There is now a high level API that provides a safe and flexible way to spa= wn +commands, which prevents the most common errors & is easy to code against.= This +code is provided in the ``src/util/vircommand.h`` header which can be impo= rted +using ``#include "vircommand.h"`` + +Defining commands in libvirt +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The first step is to declare what command is to be executed. The command n= ame +can be either a fully qualified path, or a bare command name. In the latte= r case +it will be resolved wrt the ``$PATH`` environment variable. + +:: + + virCommand *cmd =3D virCommandNew("/usr/bin/dnsmasq"); + +There is no need to check for allocation failure after ``virCommandNew``. = This +will be detected and reported at a later time. + +Adding arguments to the command +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +There are a number of APIs for adding arguments to a command. To add a dir= ect +string arg + +:: + + virCommandAddArg(cmd, "-strict-order"); + +If an argument takes an attached value of the form ``-arg=3Dval``, then th= is can +be done using + +:: + + virCommandAddArgPair(cmd, "--conf-file", "/etc/dnsmasq.conf"); + +If an argument needs to be formatted as if by ``printf``: + +:: + + virCommandAddArgFormat(cmd, "%d", count); + +To add an entire NULL terminated array of arguments in one go, there are t= wo +options. + +:: + + const char *const args[] =3D { + "--strict-order", "--except-interface", "lo", NULL + }; + virCommandAddArgSet(cmd, args); + virCommandAddArgList(cmd, "--domain", "localdomain", NULL); + +This can also be done at the time of initial construction of the +``virCommand *`` object: + +:: + + const char *const args[] =3D { + "/usr/bin/dnsmasq", + "--strict-order", "--except-interface", + "lo", "--domain", "localdomain", NULL + }; + virCommand *cmd1 =3D virCommandNewArgs(cmd, args); + virCommand *cmd2 =3D virCommandNewArgList("/usr/bin/dnsmasq", + "--domain", "localdomain", NU= LL); + +Setting up the environment +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +By default a command will inherit all environment variables from the curre= nt +process. Generally this is not desirable and a customized environment will= be +more suitable. Any customization done via the following APIs will prevent +inheritance of any existing environment variables unless explicitly allowe= d. The +first step is usually to pass through a small number of variables from the +current process. + +:: + + virCommandAddEnvPassCommon(cmd); + +This has now set up a clean environment for the child, passing through ``P= ATH``, +``LD_PRELOAD``, ``LD_LIBRARY_PATH``, ``HOME``, ``USER``, ``LOGNAME`` and +``TMPDIR``. Furthermore it will explicitly set ``LC_ALL=3DC`` to avoid une= xpected +localization of command output. Further variables can be passed through fr= om +parent explicitly: + +:: + + virCommandAddEnvPass(cmd, "DISPLAY"); + virCommandAddEnvPass(cmd, "XAUTHORITY"); + +To define an environment variable in the child with an separate key / valu= e: + +:: + + virCommandAddEnvPair(cmd, "TERM", "xterm"); + +If the key/value pair is pre-formatted in the right format, it can be set +directly + +:: + + virCommandAddEnvString(cmd, "TERM=3Dxterm"); + +Miscellaneous other options +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Normally the spawned command will retain the current process and process g= roup +as its parent. If the current process dies, the child will then (usually) = be +terminated too. If this cleanup is not desired, then the command should be +marked as daemonized: + +:: + + virCommandDaemonize(cmd); + +When daemonizing a command, the PID visible from the caller will be that o= f the +intermediate process, not the actual damonized command. If the PID of the = real +command is required then a pidfile can be requested + +:: + + virCommandSetPidFile(cmd, "/var/run/dnsmasq.pid"); + +This PID file is guaranteed to be written before the intermediate process = exits. +Moreover, the daemonized process will inherit the FD of the opened and loc= ked +PID file. + +Reducing command privileges +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Normally a command will inherit all privileges of the current process. To +restrict what a command can do, it is possible to request that all its +capabilities are cleared. With this done it will only be able to access +resources for which it has explicit DAC permissions + +:: + + virCommandClearCaps(cmd); + +Managing file handles +~~~~~~~~~~~~~~~~~~~~~ + +To prevent unintended resource leaks to child processes, the child default= s to +closing all open file handles, and setting stdin/out/err to ``/dev/null``.= It is +possible to allow an open file handle to be passed into the child, while +controlling whether that handle remains open in the parent or guaranteeing= that +the handle will be closed in the parent after virCommandRun, virCommandRun= Async, +or virCommandFree. + +:: + + int sharedfd =3D open("cmd.log", "w+"); + int childfd =3D open("conf.txt", "r"); + virCommandPassFD(cmd, sharedfd, 0); + virCommandPassFD(cmd, childfd, + VIR_COMMAND_PASS_FD_CLOSE_PARENT); + if (VIR_CLOSE(sharedfd) < 0) + goto cleanup; + +With this, both file descriptors sharedfd and childfd in the current proce= ss +remain open as the same file descriptors in the child. Meanwhile, after the +child is spawned, sharedfd remains open in the parent, while childfd is cl= osed. + +For stdin/out/err it is sometimes necessary to map a file handle. If a map= ped +file handle is a pipe fed or consumed by the caller, then the caller shoul= d use +virCommandDaemonize or virCommandRunAsync rather than virCommandRun to avo= id +deadlock (mapping a regular file is okay with virCommandRun). To attach fi= le +descriptor 7 in the current process to stdin in the child: + +:: + + virCommandSetInputFD(cmd, 7); + +Equivalently to redirect stdout or stderr in the child, pass in a pointer = to the +desired handle + +:: + + int outfd =3D open("out.log", "w+"); + int errfd =3D open("err.log", "w+"); + virCommandSetOutputFD(cmd, &outfd); + virCommandSetErrorFD(cmd, &errfd); + +Alternatively it is possible to request that a pipe be created to fetch +stdout/err in the parent, by initializing the FD to -1. + +:: + + int outfd =3D -1; + int errfd =3D -1 + virCommandSetOutputFD(cmd, &outfd); + virCommandSetErrorFD(cmd, &errfd); + +Once the command is running, ``outfd`` and ``errfd`` will be initialized w= ith +valid file handles that can be read from. It is permissible to pass the sa= me +pointer for both outfd and errfd, in which case both standard streams in t= he +child will share the same fd in the parent. + +Normally, file descriptors opened to collect output from a child process p= erform +blocking I/O, but the parent process can request non-blocking mode: + +:: + + virCommandNonblockingFDs(cmd); + +Feeding & capturing strings to/from the child +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Often dealing with file handles for stdin/out/err is unnecessarily complex= ; an +alternative is to let virCommandRun perform the I/O and interact via string +buffers. Use of a buffer only works with virCommandRun, and cannot be mixe= d with +pipe file descriptors. That is, the choice is generally between managing a= ll I/O +in the caller (any fds not specified are tied to /dev/null), or letting +virCommandRun manage all I/O via strings (unspecified stdin is tied to +/dev/null, and unspecified output streams get logged but are otherwise +discarded). + +It is possible to specify a string buffer to act as the data source for the +child's stdin, if there are no embedded NUL bytes, and if the command will= be +run with virCommandRun: + +:: + + const char *input =3D "Hello World\n"; + virCommandSetInputBuffer(cmd, input); + +Similarly it is possible to request that the child's stdout/err be redirec= ted +into a string buffer, if the output is not expected to contain NUL bytes, = and if +the command will be run with virCommandRun: + +:: + + char *output =3D NULL, *errors =3D NULL; + virCommandSetOutputBuffer(cmd, &output); + virCommandSetErrorBuffer(cmd, &errors); + +Once the command has finished executing, these buffers will contain the ou= tput. +Allocation is guaranteed if virCommandRun or virCommandWait succeed (if th= ere +was no output, then the buffer will contain an allocated empty string); if= the +command failed, then the buffers usually contain a best-effort allocation = of +collected information (however, on an out-of-memory condition, the buffer = may +still be NULL). The caller is responsible for freeing registered buffers, = since +the buffers are designed to persist beyond virCommandFree. It is possible = to +pass the same pointer to both virCommandSetOutputBuffer and +virCommandSetErrorBuffer, in which case the child process interleaves outp= ut +into a single string. + +Setting working directory +~~~~~~~~~~~~~~~~~~~~~~~~~ + +Daemonized commands are always run with "/" as the current working directo= ry. +All other commands default to running in the same working directory as the +parent process, but an alternate directory can be specified: + +:: + + virCommandSetWorkingDirectory(cmd, LOCALSTATEDIR); + +Any additional hooks +~~~~~~~~~~~~~~~~~~~~ + +If anything else is needed, it is possible to request a hook function that= is +called in the child after the fork, as the last thing before changing +directories, dropping capabilities, and executing the new process. If +hook(opaque) returns non-zero, then the child process will not be run. + +:: + + virCommandSetPreExecHook(cmd, hook, opaque); + +Logging commands +~~~~~~~~~~~~~~~~ + +Sometimes, it is desirable to log what command will be run, or even to use +virCommand solely for creation of a single consolidated string without run= ning +anything. + +:: + + int logfd =3D ...; + char *timestamp =3D virTimestamp(); + char *string =3D NULL; + + dprintf(logfd, "%s: ", timestamp); + VIR_FREE(timestamp); + virCommandWriteArgLog(cmd, logfd); + + string =3D virCommandToString(cmd, false); + if (string) + VIR_DEBUG("about to run %s", string); + VIR_FREE(string); + if (virCommandRun(cmd, NULL) < 0) + return -1; + +Running commands synchronously +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +For most commands, the desired behaviour is to spawn the command, wait for= it to +complete & exit and then check that its exit status is zero + +:: + + if (virCommandRun(cmd, NULL) < 0) + return -1; + +**Note:** if the command has been daemonized this will only block & wait f= or the +intermediate process, not the real command. ``virCommandRun`` will report = on any +errors that have occurred upon this point with all previous API calls. If = the +command fails to run, or exits with non-zero status an error will be repor= ted +via normal libvirt error infrastructure. If a non-zero exit status can rep= resent +a success condition, it is possible to request the exit status and perform= that +check manually instead of letting ``virCommandRun`` raise the error. By de= fault, +the captured status is only for a normal exit (death from a signal is trea= ted as +an error), but a caller can use ``virCommandRawStatus`` to get encoded sta= tus +that includes any terminating signals. + +:: + + int status; + if (virCommandRun(cmd, &status) < 0) + return -1; + if (status =3D=3D 1) { + ...do stuff... + } + + virCommandRawStatus(cmd2); + if (virCommandRun(cmd2, &status) < 0) + return -1; + if (WIFEXITED(status) && WEXITSTATUS(status) =3D=3D 1) { + ...do stuff... + } + +Running commands asynchronously +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In certain complex scenarios, particularly special I/O handling is require= d for +the child's stdin/err/out it will be necessary to run the command asynchro= nously +and wait for completion separately. + +:: + + pid_t pid; + if (virCommandRunAsync(cmd, &pid) < 0) + return -1; + + ... do something while pid is running ... + + int status; + if (virCommandWait(cmd, &status) < 0) + return -1; + + if (WEXITSTATUS(status)...) { + ..do stuff.. + } + +As with ``virCommandRun``, the ``status`` arg for ``virCommandWait`` can be +omitted, in which case it will validate that exit status is zero and raise= an +error if not. + +There are two approaches to child process cleanup, determined by how long = you +want to keep the virCommand object in scope. + +1. If the virCommand object will outlast the child process, then pass NULL= for +the pid argument, and the child process will automatically be reaped at +virCommandFree, unless you reap it sooner via virCommandWait or virCommand= Abort. + +2. If the child process must exist on at least one code path after +virCommandFree, then pass a pointer for the pid argument. Later, to clean = up the +child, call virPidWait or virPidAbort. Before virCommandFree, you can stil= l use +virCommandWait or virCommandAbort to reap the process. + +Releasing resources +~~~~~~~~~~~~~~~~~~~ + +Once the command has been executed, or if execution has been abandoned, it= is +necessary to release resources associated with the ``virCommand *`` object= . This +is done with: + +:: + + virCommandFree(cmd); + +There is no need to check if ``cmd`` is NULL before calling ``virCommandFr= ee``. +This scenario is handled automatically. If the command is still running, i= t will +be forcibly killed and cleaned up (via waitpid). + +Complete examples +----------------- + +This shows a complete example usage of the APIs roughly using the libvirt = source +src/util/hooks.c + +:: + + int runhook(const char *drvstr, const char *id, + const char *opstr, const char *subopstr, + const char *extra) + { + g_autofree char *path =3D NULL; + g_autoptr(virCommand) cmd =3D NULL; + + virBuildPath(&path, LIBVIRT_HOOK_DIR, drvstr); + + cmd =3D virCommandNew(path); + + virCommandAddEnvPassCommon(cmd); + + virCommandAddArgList(cmd, id, opstr, subopstr, extra, NULL); + + virCommandSetInputBuffer(cmd, input); + + return virCommandRun(cmd, NULL); + } + +In this example, the command is being run synchronously. A pre-formatted s= tring +is being fed to the command as its stdin. The command takes four arguments= , and +has a minimal set of environment variables passed down. In this example, t= he +code does not require any error checking. All errors are reported by the +``virCommandRun`` method, and the exit status from this is returned to the +caller to handle as desired. diff --git a/docs/kbase/internals/meson.build b/docs/kbase/internals/meson.= build index 923e262706..3486b21852 100644 --- a/docs/kbase/internals/meson.build +++ b/docs/kbase/internals/meson.build @@ -1,4 +1,5 @@ docs_kbase_internals_files =3D [ + 'command', 'incremental-backup', 'migration', ] --=20 2.35.1