From nobody Mon Feb 9 19:30:41 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1649340124; cv=none; d=zohomail.com; s=zohoarc; b=PjTiC9se8N8RLafeu5j6JHZpJMj2aKAGI4IhBL3G6izas9o+MRD06vUovwBcXphdGQrP4o89NsbyUtTVfgnNs0cLg50OZhS3gE1b+LYUBJ4x+qQpqpjNp1G9oRha+gCcMdLH/MwTSWakKi2eQ1YF9HEn3kgsBHBvvo/67tc5UTk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1649340124; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=3EmKA7r5NclIm+g3HZ84NJ1yUCRISt3VlHRzWAthZl4=; b=DZv+tBKwFUztJdpn8wORYugOb7IgygnAmBg4k3wgLCGxqP3eug3ZD6MY42jseRal0stM2VXb7jGMi5j0RGcy0hVMjmNv9f28GAdbA2C8ZIMEs0Yr/X+594Ll02Rdaa/Oz7BrWWlF/DwIRfAqMXAnBfVzZCDZkZvOPqactH14Ldc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1649340124403771.077280754112; Thu, 7 Apr 2022 07:02:04 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-52-KohhqUzZPIqtMjHj0dWFsA-1; Thu, 07 Apr 2022 10:01:09 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D228418F0241; Thu, 7 Apr 2022 14:00:49 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id BC60440D282F; Thu, 7 Apr 2022 14:00:49 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 6CCA01949761; Thu, 7 Apr 2022 14:00:49 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id BD09F1940347 for ; Thu, 7 Apr 2022 14:00:47 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 9FB2F40314D; Thu, 7 Apr 2022 14:00:47 +0000 (UTC) Received: from speedmetal.lan (unknown [10.40.208.35]) by smtp.corp.redhat.com (Postfix) with ESMTP id F2C88401DB1 for ; Thu, 7 Apr 2022 14:00:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1649340123; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=3EmKA7r5NclIm+g3HZ84NJ1yUCRISt3VlHRzWAthZl4=; b=UJ1bukH5p1HggR6WllxkbsNL5/f7Sb7dq26DLz7FuWp2StBMYVQBzNzieWyRM1B4JX0X55 Uu6n/RtcPDpKabkIBViKh0QnsFUDrXTJVGvGEwUwH6EfPWDb5HmcTDSiiqh2LI0XZZiOYM AHO2CRzKyhCvyYhRFSbaccqzMlnXxmU= X-MC-Unique: KohhqUzZPIqtMjHj0dWFsA-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Peter Krempa To: libvir-list@redhat.com Subject: [PATCH 10/11] docs: Convert 'internals' to RST and move it to 'kbase/internal/overview.rst' Date: Thu, 7 Apr 2022 16:00:32 +0200 Message-Id: <4c87df2b986c254f69f3b9f57165a52ff8d57aab.1649339910.git.pkrempa@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: 8bit Content-Type: application/octet-stream; x-default=true X-Zoho-Virus-Status: 1 X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1649340126551100003 Note that this document was not referenced from any top level page. This patch does a straight conversion and leaves it unreferenced. Next patch will then modify it to serve as an overview (hence the new name) of how an API call happens. Signed-off-by: Peter Krempa --- docs/internals.html.in | 119 ------------------------------ docs/kbase/internals/meson.build | 1 + docs/kbase/internals/overview.rst | 117 +++++++++++++++++++++++++++++ docs/meson.build | 1 - 4 files changed, 118 insertions(+), 120 deletions(-) delete mode 100644 docs/internals.html.in create mode 100644 docs/kbase/internals/overview.rst diff --git a/docs/internals.html.in b/docs/internals.html.in deleted file mode 100644 index e474f7ddd7..0000000000 --- a/docs/internals.html.in +++ /dev/null @@ -1,119 +0,0 @@ - - - - -

libvirt internals

- -

- This section provides documents useful to those working on the libvirt - internals, adding new public APIs, new hypervisor drivers or extending - the libvirtd daemon code. -

- - - -

Before adding new code it will be important to get a basic understanding - of the many elements involved with making any call or change to the libvirt - code. The architecture goals must be adhered to - when submitting new code. Understanding the many places that need to be - touched and the interactions between various subsystems within libvirt - will directly correlate to the ability to be successful in getting new - code accepted.

-

The following diagram depicts code flow from a client application, in - this case the libvirt provided virsh command through the - various layers to elicit a response from some chosen hypervisor. -

- -

- virConnectOpen calling sequence -

-
    -
  • "virsh -c qemu:///system list --all" -

    After the virsh code processes the input arguments, it eventually - will make a call to open the connection using a default set of - authentication credentials (virConnectAuthDefault).

    • -
  • virConnectOpenAuth() -

    Each of the virConnectOpen APIs will first call virInitialize() and - then revector through the local "do_open():" call.

    -
      -
    • virInitialize() -

      Calls the registration API for each of the drivers with - client-side only capabilities and then call the remoteRegister() - API last. This ensures the virDriverTab[] tries local drivers - first before using the remote driver.

      • -
    • Loop through virDriverTab[] entries trying to call their - respective "open" entry point (in our case remoteOpen())
      • -
    • After successful return from the virDriverTab[] open() - API, attempt to find and open other drivers (network, interface, - storage, etc.)
      • -
    -
    • -
  • remoteOpen() -

    After a couple of URI checks, a call to doRemoteOpen() is made

    -
      -
    • Determine network transport and host/port to use from URI -

      The transport will be either tls, unix, ssh, libssh2, ext, - or tcp with the default of tls. Decode the host/port if provided - or default to "localhost".

      • -
    • virNetClientRegisterAsyncIO() -

      Register an I/O callback mechanism to get returned data via - virNetClientIncomingEvent()

      • -
    • "call(...REMOTE_PROC_OPEN...)" -

      Eventually routes into virNetClientProgramCall() which will - call virNetClientSendWithReply() and eventually uses - virNetClientIO()to send the message to libvirtd and - then waits for a response using virNetClientIOEventLoop()

      • -
    • virNetClientIncomingEvent() -

      Receives the returned packet and processes through - virNetClientIOUpdateCallback()

      • -
    -
    • -
  • libvirtd Daemon -

    -
      -
    • Daemon Startup -

      The daemon initialization processing will declare itself - as a daemon via a virNetDaemonNew() call, then creates new server - using virNetServerNew() and adds that server to the main daemon - struct with virNetDaemonAddServer() call. It will then use - virDriverLoadModule() to find/load all known drivers, - set up an RPC server program using the remoteProcs[] - table via a virNetServerProgramNew() call. The table is the - corollary to the remote_procedure enum list in - the client. It lists all the functions to be called in - the same order. Once RPC is set up, networking server sockets - are opened, the various driver state initialization routines - are run from the virStateDriverTab[], the network - links are enabled, and the daemon waits for work.

      • -
    • RPC -

      When a message is received, the remoteProcs[] - table is referenced for the 'REMOTE_PROC_OPEN' call entry. - This results in remoteDispatchOpen() being called via the - virNetServerProgramDispatchCall().

      • -
    • remoteDispatchOpen() -

      The API will read the argument passed picking out the - name of the driver to be opened. The code - will then call virConnectOpen() or virConnectOpenReadOnly() - depending on the argument flags.

      • -
    • virConnectOpen() or virConnectOpenReadOnly() -

      Just like the client except that upon entry the URI - is what was passed from the client and will be found - and opened to process the data.

      -

      The returned structure data is returned via the - virNetServer interfaces to the remote driver which then - returns it to the client application.

      • -
    -
    • -
- - diff --git a/docs/kbase/internals/meson.build b/docs/kbase/internals/meson.build index 879c4b2de8..8b5daad1f9 100644 --- a/docs/kbase/internals/meson.build +++ b/docs/kbase/internals/meson.build @@ -4,6 +4,7 @@ docs_kbase_internals_files = [ 'incremental-backup', 'locking', 'migration', + 'overview', 'rpc', ] diff --git a/docs/kbase/internals/overview.rst b/docs/kbase/internals/overview.rst new file mode 100644 index 0000000000..84ea7858db --- /dev/null +++ b/docs/kbase/internals/overview.rst @@ -0,0 +1,117 @@ +========================== +libvirt internals overview +========================== + +This section provides documents useful to those working on the libvirt +internals, adding new public APIs, new hypervisor drivers or extending the +libvirtd daemon code. + +- Introduction to basic rules and guidelines for `hacking <../../hacking.html>`__ on + libvirt code +- Guide to adding `public APIs <../../api_extension.html>`__ +- Insight into libvirt `event loop and worker + pool `__ +- Approach for `spawning commands `__ from libvirt + driver code +- The libvirt `RPC infrastructure `__ +- The `Resource Lock Manager `__ + +Before adding new code it will be important to get a basic understanding of the +many elements involved with making any call or change to the libvirt code. The +architecture `goals <../../goals.html>`__ must be adhered to when submitting new code. +Understanding the many places that need to be touched and the interactions +between various subsystems within libvirt will directly correlate to the ability +to be successful in getting new code accepted. + +The following diagram depicts code flow from a client application, in this case +the libvirt provided ``virsh`` command through the various layers to elicit a +response from some chosen hypervisor. + +.. image:: ../../images/libvirt-virConnect-example.png + :alt: virConnectOpen calling sequence + +- "virsh -c qemu:///system list --all" + + After the virsh code processes the input arguments, it eventually will make a + call to open the connection using a default set of authentication credentials + (virConnectAuthDefault). + +- virConnectOpenAuth() + + Each of the virConnectOpen APIs will first call virInitialize() and then + revector through the local "do_open():" call. + + - virInitialize() + + Calls the registration API for each of the drivers with client-side only + capabilities and then call the remoteRegister() API last. This ensures the + virDriverTab[] tries local drivers first before using the remote driver. + + - Loop through virDriverTab[] entries trying to call their respective "open" + entry point (in our case remoteOpen()) + + - After successful return from the virDriverTab[] open() API, attempt to + find and open other drivers (network, interface, storage, etc.) + +- remoteOpen() + + After a couple of URI checks, a call to doRemoteOpen() is made + + - Determine network transport and host/port to use from URI + + The transport will be either tls, unix, ssh, libssh2, ext, or tcp with the + default of tls. Decode the host/port if provided or default to + "localhost". + + - virNetClientRegisterAsyncIO() + + Register an I/O callback mechanism to get returned data via + virNetClientIncomingEvent() + + - "call(...REMOTE_PROC_OPEN...)" + + Eventually routes into virNetClientProgramCall() which will call + virNetClientSendWithReply() and eventually uses virNetClientIO()to send + the message to libvirtd and then waits for a response using + virNetClientIOEventLoop() + + - virNetClientIncomingEvent() + + Receives the returned packet and processes through + virNetClientIOUpdateCallback() + +- libvirtd Daemon + + - Daemon Startup + + The daemon initialization processing will declare itself as a daemon via a + virNetDaemonNew() call, then creates new server using virNetServerNew() + and adds that server to the main daemon struct with + virNetDaemonAddServer() call. It will then use virDriverLoadModule() to + find/load all known drivers, set up an RPC server program using the + ``remoteProcs[]`` table via a virNetServerProgramNew() call. The table is + the corollary to the ``remote_procedure`` enum list in the client. It + lists all the functions to be called in the same order. Once RPC is set + up, networking server sockets are opened, the various driver state + initialization routines are run from the ``virStateDriverTab[]``, the + network links are enabled, and the daemon waits for work. + + - RPC + + When a message is received, the ``remoteProcs[]`` table is referenced for + the 'REMOTE_PROC_OPEN' call entry. This results in remoteDispatchOpen() + being called via the virNetServerProgramDispatchCall(). + + - remoteDispatchOpen() + + The API will read the argument passed picking out the ``name`` of the + driver to be opened. The code will then call virConnectOpen() or + virConnectOpenReadOnly() depending on the argument ``flags``. + + - virConnectOpen() or virConnectOpenReadOnly() + + Just like the client except that upon entry the URI is what was passed + from the client and will be found and opened to process the data. + + The returned structure data is returned via the virNetServer interfaces to + the remote driver which then returns it to the client application. diff --git a/docs/meson.build b/docs/meson.build index 4291dea6f5..9f1f1ed80d 100644 --- a/docs/meson.build +++ b/docs/meson.build @@ -25,7 +25,6 @@ docs_html_in_files = [ 'formatnwfilter', 'formatstoragecaps', 'index', - 'internals', 'remote', 'storage', 'uri', -- 2.35.1