We are generating a fresh UUID and storing it in the XML for the
default network, but this is unnecessary because the network
driver will automatically generate one if it's missing from the
XML; the fact that we only do this if the uuidgen command happens
to be available on the build machine is further proof that we can
safely skip this step.

This patch is best viewed with 'git show -w'.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
---
 src/network/meson.build | 32 +++++++-------------------------
 1 file changed, 7 insertions(+), 25 deletions(-)

diff --git a/src/network/meson.build b/src/network/meson.build
index 13dd2c26b2..3ec598c3f9 100644
--- a/src/network/meson.build
+++ b/src/network/meson.build
@@ -84,31 +84,13 @@ if conf.has('WITH_NETWORK')
     runstatedir / 'libvirt' / 'network',
   ]
 
-  uuidgen_prog = find_program('uuidgen', required: false)
-
-  if uuidgen_prog.found()
-    uuid = run_command(uuidgen_prog).stdout().strip()
-
-    configure_file(
-      input: 'default.xml.in',
-      output: '@BASENAME@',
-      command: [
-        'sed', '-e', 's|</name>|</name>\\n  <uuid>@0@</uuid>|'.format(uuid),
-        '@INPUT@',
-      ],
-      capture: true,
-      install: true,
-      install_dir: confdir / 'qemu' / 'networks',
-    )
-  else
-    configure_file(
-      input: 'default.xml.in',
-      output: '@BASENAME@',
-      copy: true,
-      install: true,
-      install_dir: confdir / 'qemu' / 'networks',
-    )
-  endif
+  configure_file(
+    input: 'default.xml.in',
+    output: '@BASENAME@',
+    copy: true,
+    install: true,
+    install_dir: confdir / 'qemu' / 'networks',
+  )
 
   meson.add_install_script(
     meson_python_prog.path(), python3_prog.path(), meson_install_symlink_prog.path(),
-- 
2.26.2

On 11/15/20 3:43 PM, Andrea Bolognani wrote:
> We are generating a fresh UUID and storing it in the XML for the
> default network, but this is unnecessary because the network
> driver will automatically generate one if it's missing from the
> XML;


But that automatically generated uuid will not be stored in the original 
xml in /etc/libvirt, so a new and different uuid will be generated every 
time libvirt is restarted.


I don't know if the solution to this is to modify libvirt so that it 
rewrites the XML for a network if it has to auto-generate any 
attributes[*], or to put a canned/static uuid value in the installed 
xml, or just to declare that we don't care if the uuid changes from one 
run of libvirtd to the next. But definitely these patches change 
behavior, so we probably need to point that out and maybe discuss it.


BTW, the whole thing of generating a different default.xml for each 
install of libvirt is a thorn in the side anyway - I looked for the BZ 
describing the problem and came up with this:


https://bugzilla.redhat.com/show_bug.cgi?id=1657041


which isn't the entire discussion, but is part of it. Basically while 
talking about this in *some venue* (maybe it was IRC, maybe a different 
BZ?) we decided that we shouldn't be (can't be?) generating files during 
the package %post_install. Our main topic of discussion was the subnet 
of the network, but adding/modifying the uuid is just another example of 
the same undesired behavior.


So *that* would be an argument in favor of at least the "don't generate 
a different uuid at install time" part of your patches. But it doesn't 
(afaics) lead to a definite resolution of "what should be done instead?".

(btw, I'm working / supposed to be working on eliminating the need to 
modify the subnet address in this same file with a new "autoaddress" 
function that will replace a fixed IP in a network definition with an 
indicator that means "find an unused subnet and use that". Doesn't solve 
the uuid bit though)


[*] I have a vague memory that there is (or was?) at least one case 
where we would automatically update the persistent definition of a 
domain or network on disk when libvirt started (it was needed due to a 
change in some attribute or something), but I don't recall the details, 
don't feel like looking for it right now, and it anyway is not the norm.

> the fact that we only do this if the uuidgen command happens
> to be available on the build machine is further proof that we can
> safely skip this step.
>
> This patch is best viewed with 'git show -w'.
>
> Signed-off-by: Andrea Bolognani <abologna@redhat.com>
> ---
>   src/network/meson.build | 32 +++++++-------------------------
>   1 file changed, 7 insertions(+), 25 deletions(-)
>
> diff --git a/src/network/meson.build b/src/network/meson.build
> index 13dd2c26b2..3ec598c3f9 100644
> --- a/src/network/meson.build
> +++ b/src/network/meson.build
> @@ -84,31 +84,13 @@ if conf.has('WITH_NETWORK')
>       runstatedir / 'libvirt' / 'network',
>     ]
>   
> -  uuidgen_prog = find_program('uuidgen', required: false)
> -
> -  if uuidgen_prog.found()
> -    uuid = run_command(uuidgen_prog).stdout().strip()
> -
> -    configure_file(
> -      input: 'default.xml.in',
> -      output: '@BASENAME@',
> -      command: [
> -        'sed', '-e', 's|</name>|</name>\\n  <uuid>@0@</uuid>|'.format(uuid),
> -        '@INPUT@',
> -      ],
> -      capture: true,
> -      install: true,
> -      install_dir: confdir / 'qemu' / 'networks',
> -    )
> -  else
> -    configure_file(
> -      input: 'default.xml.in',
> -      output: '@BASENAME@',
> -      copy: true,
> -      install: true,
> -      install_dir: confdir / 'qemu' / 'networks',
> -    )
> -  endif
> +  configure_file(
> +    input: 'default.xml.in',
> +    output: '@BASENAME@',
> +    copy: true,
> +    install: true,
> +    install_dir: confdir / 'qemu' / 'networks',
> +  )
>   
>     meson.add_install_script(
>       meson_python_prog.path(), python3_prog.path(), meson_install_symlink_prog.path(),

On Sun, Nov 15, 2020 at 07:19:24PM -0500, Laine Stump wrote:
> On 11/15/20 3:43 PM, Andrea Bolognani wrote:
> > We are generating a fresh UUID and storing it in the XML for the
> > default network, but this is unnecessary because the network
> > driver will automatically generate one if it's missing from the
> > XML;
> 
> 
> But that automatically generated uuid will not be stored in the original xml
> in /etc/libvirt, so a new and different uuid will be generated every time
> libvirt is restarted.
> 
> 
> I don't know if the solution to this is to modify libvirt so that it
> rewrites the XML for a network if it has to auto-generate any attributes[*],
> or to put a canned/static uuid value in the installed xml, or just to
> declare that we don't care if the uuid changes from one run of libvirtd to
> the next. But definitely these patches change behavior, so we probably need
> to point that out and maybe discuss it.

We definitely should not be changing UUID on every startup, so IMHO this
patch should not be applied.

I'm not seeing much point in creating some new solution in libvirtd to
replace something which already works, unless there's actually some
real problem reported which isn't mentioned in this commit message.


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

On Sun, 2020-11-15 at 19:19 -0500, Laine Stump wrote:
> On 11/15/20 3:43 PM, Andrea Bolognani wrote:
> > We are generating a fresh UUID and storing it in the XML for the
> > default network, but this is unnecessary because the network
> > driver will automatically generate one if it's missing from the
> > XML;
> 
> But that automatically generated uuid will not be stored in the original 
> xml in /etc/libvirt, so a new and different uuid will be generated every 
> time libvirt is restarted.

That doesn't seem to be the case:

  $ sudo cat /etc/libvirt/qemu/networks/default.xml
  <network>
    <name>default</name>
    <forward mode='nat'/>
    <bridge name='virbr0'/>
    <ip address='192.168.122.1' netmask='255.255.255.0'>
      <dhcp>
        <range start='192.168.122.2' end='192.168.122.254'/>
      </dhcp>
    </ip>
  </network>
  $ sudo systemctl start libvirtd
  $ sudo systemctl stop libvirtd
  $ sudo cat /etc/libvirt/qemu/networks/default.xml
  <!--
  WARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE
  OVERWRITTEN AND LOST. Changes to this xml configuration should be made using:
    virsh net-edit default
  or other application using the libvirt API.
  -->
  
  <network>
    <name>default</name>
    <uuid>212a684a-4ca4-4c42-b389-aecd676868f8</uuid>
    <forward mode='nat'/>
    <bridge name='virbr0' stp='on' delay='0'/>
    <mac address='52:54:00:ff:27:85'/>
    <ip address='192.168.122.1' netmask='255.255.255.0'>
      <dhcp>
        <range start='192.168.122.2' end='192.168.122.254'/>
      </dhcp>
    </ip>
  </network>
  $ sudo systemctl start libvirtd
  $ sudo systemctl stop libvirtd
  $ sudo cat /etc/libvirt/qemu/networks/default.xml
  <!--
  WARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE
  OVERWRITTEN AND LOST. Changes to this xml configuration should be made using:
    virsh net-edit default
  or other application using the libvirt API.
  -->
  
  <network>
    <name>default</name>
    <uuid>212a684a-4ca4-4c42-b389-aecd676868f8</uuid>
    <forward mode='nat'/>
    <bridge name='virbr0' stp='on' delay='0'/>
    <mac address='52:54:00:ff:27:85'/>
    <ip address='192.168.122.1' netmask='255.255.255.0'>
      <dhcp>
        <range start='192.168.122.2' end='192.168.122.254'/>
      </dhcp>
    </ip>
  </network>

> I don't know if the solution to this is to modify libvirt so that it 
> rewrites the XML for a network if it has to auto-generate any 
> attributes[*], or to put a canned/static uuid value in the installed 
> xml, or just to declare that we don't care if the uuid changes from one 
> run of libvirtd to the next. But definitely these patches change 
> behavior, so we probably need to point that out and maybe discuss it.
> 
> [*] I have a vague memory that there is (or was?) at least one case 
> where we would automatically update the persistent definition of a 
> domain or network on disk when libvirt started (it was needed due to a 
> change in some attribute or something), but I don't recall the details, 
> don't feel like looking for it right now, and it anyway is not the norm.

AFAICT the behavior you're talking about, and which can be seen in
action in the transcript above, is caused by the following snippet in
virNetworkLoadConfig():

  switch ((virNetworkForwardType) def->forward.type) {
  case VIR_NETWORK_FORWARD_NONE:
  case VIR_NETWORK_FORWARD_NAT:
  case VIR_NETWORK_FORWARD_ROUTE:
  case VIR_NETWORK_FORWARD_OPEN:
      if (!def->mac_specified) {
          virNetworkSetBridgeMacAddr(def);
          virNetworkSaveConfig(configDir, def, xmlopt);
      }
      break;

which can be tracked down to

  https://gitlab.com/libvirt/libvirt/-/commit/a47ae7c004e92f959b45808ca82326e8559c2f61

So, it's far from a new behavior.

Note that we only persist the new configuration if the bridge's MAC
address has been generated, not if the network UUID has, which is
something that we might want to change. But, for the use case at
hand, we know that the XML template for the default network will have
neither of those, so in practice they will both be generated during
the first startup and the result will be persisted to disk.

-- 
Andrea Bolognani / Red Hat / Virtualization

On 11/16/20 5:33 AM, Andrea Bolognani wrote:
> On Sun, 2020-11-15 at 19:19 -0500, Laine Stump wrote:
>> On 11/15/20 3:43 PM, Andrea Bolognani wrote:
>>> We are generating a fresh UUID and storing it in the XML for the
>>> default network, but this is unnecessary because the network
>>> driver will automatically generate one if it's missing from the
>>> XML;
>> But that automatically generated uuid will not be stored in the original
>> xml in /etc/libvirt, so a new and different uuid will be generated every
>> time libvirt is restarted.
> That doesn't seem to be the case:
>
>    $ sudo cat /etc/libvirt/qemu/networks/default.xml
>    <network>
>      <name>default</name>
>      <forward mode='nat'/>
>      <bridge name='virbr0'/>
>      <ip address='192.168.122.1' netmask='255.255.255.0'>
>        <dhcp>
>          <range start='192.168.122.2' end='192.168.122.254'/>
>        </dhcp>
>      </ip>
>    </network>
>    $ sudo systemctl start libvirtd
>    $ sudo systemctl stop libvirtd
>    $ sudo cat /etc/libvirt/qemu/networks/default.xml
>    <!--
>    WARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE
>    OVERWRITTEN AND LOST. Changes to this xml configuration should be made using:
>      virsh net-edit default
>    or other application using the libvirt API.
>    -->
>    
>    <network>
>      <name>default</name>
>      <uuid>212a684a-4ca4-4c42-b389-aecd676868f8</uuid>
>      <forward mode='nat'/>
>      <bridge name='virbr0' stp='on' delay='0'/>
>      <mac address='52:54:00:ff:27:85'/>
>      <ip address='192.168.122.1' netmask='255.255.255.0'>
>        <dhcp>
>          <range start='192.168.122.2' end='192.168.122.254'/>
>        </dhcp>
>      </ip>
>    </network>
>    $ sudo systemctl start libvirtd
>    $ sudo systemctl stop libvirtd
>    $ sudo cat /etc/libvirt/qemu/networks/default.xml
>    <!--
>    WARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE
>    OVERWRITTEN AND LOST. Changes to this xml configuration should be made using:
>      virsh net-edit default
>    or other application using the libvirt API.
>    -->
>    
>    <network>
>      <name>default</name>
>      <uuid>212a684a-4ca4-4c42-b389-aecd676868f8</uuid>
>      <forward mode='nat'/>
>      <bridge name='virbr0' stp='on' delay='0'/>
>      <mac address='52:54:00:ff:27:85'/>
>      <ip address='192.168.122.1' netmask='255.255.255.0'>
>        <dhcp>
>          <range start='192.168.122.2' end='192.168.122.254'/>
>        </dhcp>
>      </ip>
>    </network>


But... I did a nearly identical experiment last night before posting my 
reply, and came up with different results, which is why I posted...


Ah! So the difference is that in your example, you have no MAC address 
either, and the code you point out down below is noticing that change, 
and causing the entire network config to be re-written to the disk. I 
hadn't removed the MAC address, so in my case it didn't re-write the config.


(...and now I've read to the bottom of your reply and see that you've 
already figured that out :-P)


I guess if we're okay with re-writing the config at daemon start to add 
a MAC address, then we should be okay with doing that to add a uuid 
(they're really just differently-used examples of the same thing). But 
if we're going to do that, the check to see if the config should be 
modified to check both uuid and mac address (even if in current practice 
both are absent from the proto-default.xml, that may not be the case in 
the future).


Alternately, if we're *not* okay with rewriting the config at daemon 
start time (even though, as you've pointed out, we've been doing that 
for 6 years already), then we would need to *stop* doing that for MAC 
address. (Which would mean we would need to figure out a different way 
to get a fixed MAC address into the config file, and seeing that the 
commit you reference actually *removes* my original code that added it 
in at install time, we can't really go back there (also because that's 
moving the code in the wrong direction - Fedora SilverBlue people said 
in the BZ I referenced that they don't want %post_install scripts 
modifying the files that are installed).


I think I'm okay with the former (adding a check for changed uuid onto 
your patches); it gives us a stable uuid (as long as the modified disk 
contents are saved for the next reboot) while eliminating another bit of 
the undesirable code that runs during %post_install.

Is there anything else wrt. pre-built images that we're not taking into 
consideration though?


>> I don't know if the solution to this is to modify libvirt so that it
>> rewrites the XML for a network if it has to auto-generate any
>> attributes[*], or to put a canned/static uuid value in the installed
>> xml, or just to declare that we don't care if the uuid changes from one
>> run of libvirtd to the next. But definitely these patches change
>> behavior, so we probably need to point that out and maybe discuss it.
>>
>> [*] I have a vague memory that there is (or was?) at least one case
>> where we would automatically update the persistent definition of a
>> domain or network on disk when libvirt started (it was needed due to a
>> change in some attribute or something), but I don't recall the details,
>> don't feel like looking for it right now, and it anyway is not the norm.
> AFAICT the behavior you're talking about, and which can be seen in
> action in the transcript above, is caused by the following snippet in
> virNetworkLoadConfig():
>
>    switch ((virNetworkForwardType) def->forward.type) {
>    case VIR_NETWORK_FORWARD_NONE:
>    case VIR_NETWORK_FORWARD_NAT:
>    case VIR_NETWORK_FORWARD_ROUTE:
>    case VIR_NETWORK_FORWARD_OPEN:
>        if (!def->mac_specified) {
>            virNetworkSetBridgeMacAddr(def);
>            virNetworkSaveConfig(configDir, def, xmlopt);
>        }
>        break;
>
> which can be tracked down to
>
>    https://gitlab.com/libvirt/libvirt/-/commit/a47ae7c004e92f959b45808ca82326e8559c2f61
>
> So, it's far from a new behavior.
>
> Note that we only persist the new configuration if the bridge's MAC
> address has been generated, not if the network UUID has, which is
> something that we might want to change. But, for the use case at
> hand, we know that the XML template for the default network will have
> neither of those, so in practice they will both be generated during
> the first startup and the result will be persisted to disk.
>

On Mon, 2020-11-16 at 10:00 -0500, Laine Stump wrote:
> On 11/16/20 5:33 AM, Andrea Bolognani wrote:
> > On Sun, 2020-11-15 at 19:19 -0500, Laine Stump wrote:
> > > On 11/15/20 3:43 PM, Andrea Bolognani wrote:
> > > > We are generating a fresh UUID and storing it in the XML for the
> > > > default network, but this is unnecessary because the network
> > > > driver will automatically generate one if it's missing from the
> > > > XML;
> > >
> > > But that automatically generated uuid will not be stored in the original
> > > xml in /etc/libvirt, so a new and different uuid will be generated every
> > > time libvirt is restarted.
> > 
> > That doesn't seem to be the case:
> > 
> > [... a bunch of command line nonsense ...]
> 
> But... I did a nearly identical experiment last night before posting my 
> reply, and came up with different results, which is why I posted...
> 
> 
> Ah! So the difference is that in your example, you have no MAC address 
> either, and the code you point out down below is noticing that change, 
> and causing the entire network config to be re-written to the disk. I 
> hadn't removed the MAC address, so in my case it didn't re-write the config.
> 
> 
> (...and now I've read to the bottom of your reply and see that you've 
> already figured that out :-P)

  "And that's why you always read the entire message before
   you start replying to it."
                                              -- J. Walter Weatherman

> I guess if we're okay with re-writing the config at daemon start to add 
> a MAC address, then we should be okay with doing that to add a uuid 
> (they're really just differently-used examples of the same thing). But 
> if we're going to do that, the check to see if the config should be 
> modified to check both uuid and mac address (even if in current practice 
> both are absent from the proto-default.xml, that may not be the case in 
> the future).
> 
> 
> Alternately, if we're *not* okay with rewriting the config at daemon 
> start time (even though, as you've pointed out, we've been doing that 
> for 6 years already), then we would need to *stop* doing that for MAC 
> address. (Which would mean we would need to figure out a different way 
> to get a fixed MAC address into the config file, and seeing that the 
> commit you reference actually *removes* my original code that added it 
> in at install time, we can't really go back there (also because that's 
> moving the code in the wrong direction - Fedora SilverBlue people said 
> in the BZ I referenced that they don't want %post_install scripts 
> modifying the files that are installed).

I don't see how we could possibly not be okay with writing back the
updated configuration: filling in whatever blanks are present in the
user-provided configuration and then writing the result back to disk
is quite central to how we handle domains and other objects, and I
see no reason networks should behave any differently.

If anything, I think we should not extend the check to UUIDs, but
just drop it completely and unconditionally write the (possibly)
updated file file to disk. I'll try cooking a patch for that.

> I think I'm okay with the former (adding a check for changed uuid onto 
> your patches); it gives us a stable uuid (as long as the modified disk 
> contents are saved for the next reboot) while eliminating another bit of 
> the undesirable code that runs during %post_install.
> 
> Is there anything else wrt. pre-built images that we're not taking into 
> consideration though?

Honestly I was not considering the pre-built images angle at all, so
there's probably *loads* that I'm not taking into account O:-)

-- 
Andrea Bolognani / Red Hat / Virtualization

On 11/16/20 11:43 AM, Andrea Bolognani wrote:
> On Mon, 2020-11-16 at 10:00 -0500, Laine Stump wrote:
>> On 11/16/20 5:33 AM, Andrea Bolognani wrote:
>>> On Sun, 2020-11-15 at 19:19 -0500, Laine Stump wrote:
>>>> On 11/15/20 3:43 PM, Andrea Bolognani wrote:
>>>>> We are generating a fresh UUID and storing it in the XML for the
>>>>> default network, but this is unnecessary because the network
>>>>> driver will automatically generate one if it's missing from the
>>>>> XML;
>>>> But that automatically generated uuid will not be stored in the original
>>>> xml in /etc/libvirt, so a new and different uuid will be generated every
>>>> time libvirt is restarted.
>>> That doesn't seem to be the case:
>>>
>>> [... a bunch of command line nonsense ...]
>> But... I did a nearly identical experiment last night before posting my
>> reply, and came up with different results, which is why I posted...
>>
>>
>> Ah! So the difference is that in your example, you have no MAC address
>> either, and the code you point out down below is noticing that change,
>> and causing the entire network config to be re-written to the disk. I
>> hadn't removed the MAC address, so in my case it didn't re-write the config.
>>
>>
>> (...and now I've read to the bottom of your reply and see that you've
>> already figured that out :-P)
>    "And that's why you always read the entire message before
>     you start replying to it."
>                                                -- J. Walter Weatherman
>
>> I guess if we're okay with re-writing the config at daemon start to add
>> a MAC address, then we should be okay with doing that to add a uuid
>> (they're really just differently-used examples of the same thing). But
>> if we're going to do that, the check to see if the config should be
>> modified to check both uuid and mac address (even if in current practice
>> both are absent from the proto-default.xml, that may not be the case in
>> the future).
>>
>>
>> Alternately, if we're *not* okay with rewriting the config at daemon
>> start time (even though, as you've pointed out, we've been doing that
>> for 6 years already), then we would need to *stop* doing that for MAC
>> address. (Which would mean we would need to figure out a different way
>> to get a fixed MAC address into the config file, and seeing that the
>> commit you reference actually *removes* my original code that added it
>> in at install time, we can't really go back there (also because that's
>> moving the code in the wrong direction - Fedora SilverBlue people said
>> in the BZ I referenced that they don't want %post_install scripts
>> modifying the files that are installed).
> I don't see how we could possibly not be okay with writing back the
> updated configuration: filling in whatever blanks are present in the
> user-provided configuration and then writing the result back to disk
> is quite central to how we handle domains and other objects, and I
> see no reason networks should behave any differently.
>
> If anything, I think we should not extend the check to UUIDs, but
> just drop it completely and unconditionally write the (possibly)
> updated file file to disk. I'll try cooking a patch for that.


I understand why you would consider doing this (low/no maintenance, 
easier to verify correct results), but I don't think it's a generally 
good idea to rewrite every config file every time we restart libvirtd. 
Some media has slow write speed and a limited number of writes in its 
lifetime (sure, it's not 1982 and we're not talking about a 2716 EPROM, 
but still :-)) and some people may rely on /etc/libvirt to be in a 
read-only filesystem for some reason (I haven't tried that, but am just 
assuming that it would work). Also, the date stamp on a file can provide 
useful clues about when something got screwed up (although admittedly it 
is useless in the quest to actually undo the screwup)


Except for three cases (that I can think of; I'm sure there are others), 
what is read from libvirt's config files in /etc is exactly what would 
be written if we were to re-write it immediately after a read-parse-format:


1) the first read of an xml file that was generated outside libvirt, and 
has been manually put into /etc/libvirt, e.g. xml files that are a part 
of a package installation/upgrade. NB: all other cases of externally 
generated files being placed in /etc/libvirt are expressly forbidden 
(well, at least "discouraged and declared unsupported") by libvirt 
documentation.


2) a user manually edits a file in /etc/libvirt - see (1)


3) upgrading libvirt causes something to be parsed differently than it 
was before. This should never happen, as there should always be enough 
info in the xml we write to the disk after its initial arrival via the 
libvirt API that any future read/write cycles are completely idempotent.


>
>> I think I'm okay with the former (adding a check for changed uuid onto
>> your patches); it gives us a stable uuid (as long as the modified disk
>> contents are saved for the next reboot) while eliminating another bit of
>> the undesirable code that runs during %post_install.
>>
>> Is there anything else wrt. pre-built images that we're not taking into
>> consideration though?
> Honestly I was not considering the pre-built images angle at all, so
> there's probably *loads* that I'm not taking into account O:-)


Or maybe I'm just paranoid. That is just as likely.


(Hmmph. After typing this my brain has figured out that the 
EPROM-writing card that should be stored away in a box somewhere inside 
another box on a shelf in my garage is an ISA-bus card, and so I haven't 
owned any computer it would plug into in at least 11 years. What will I 
do if I ever need to reprogram the EPROM on my 1980's homebrew 
computer?? :-( Should my lesson be that I should throw away less old 
hardware? Or more?)

On Tue, 2020-11-17 at 10:47 -0500, Laine Stump wrote:
> On 11/16/20 11:43 AM, Andrea Bolognani wrote:
> > I don't see how we could possibly not be okay with writing back the
> > updated configuration: filling in whatever blanks are present in the
> > user-provided configuration and then writing the result back to disk
> > is quite central to how we handle domains and other objects, and I
> > see no reason networks should behave any differently.
> > 
> > If anything, I think we should not extend the check to UUIDs, but
> > just drop it completely and unconditionally write the (possibly)
> > updated file file to disk. I'll try cooking a patch for that.
> 
> I understand why you would consider doing this (low/no maintenance, 
> easier to verify correct results), but I don't think it's a generally 
> good idea to rewrite every config file every time we restart libvirtd. 
> Some media has slow write speed and a limited number of writes in its 
> lifetime (sure, it's not 1982 and we're not talking about a 2716 EPROM, 
> but still :-)) and some people may rely on /etc/libvirt to be in a 
> read-only filesystem for some reason (I haven't tried that, but am just 
> assuming that it would work).

I would absolutely expect that to *not* be the case, if nothing else
because I don't think anyone is actually QA-ing that scenario.

Even assuming it does indeed work, you'd end up with a pretty limited
version of libvirt, where features like 'virsh define' don't work.

Honestly I would be very surprised if anyone actually used libvirt
this way.

> Except for three cases (that I can think of; I'm sure there are others), 
> what is read from libvirt's config files in /etc is exactly what would 
> be written if we were to re-write it immediately after a read-parse-format:
> 
> 1) the first read of an xml file that was generated outside libvirt, and 
> has been manually put into /etc/libvirt, e.g. xml files that are a part 
> of a package installation/upgrade. NB: all other cases of externally 
> generated files being placed in /etc/libvirt are expressly forbidden 
> (well, at least "discouraged and declared unsupported") by libvirt 
> documentation.
> 
> 2) a user manually edits a file in /etc/libvirt - see (1)

Okay, but from libvirt's point of view there's really no way to tell
apart a file that's been dropped into /etc/libvirt by the package
manager as opposed to the admin, is there? So they will ultimately
have to be treated the same way.

> 3) upgrading libvirt causes something to be parsed differently than it 
> was before. This should never happen, as there should always be enough 
> info in the xml we write to the disk after its initial arrival via the 
> libvirt API that any future read/write cycles are completely idempotent.

Except for randomly-generated data such as, I don't know, MAC
addresses or UUIDs :)

So, I played with this a bit to understand the state of the art.

  $ for f in storage/default qemu/networks/default \
             qemu/cirros nwfilter/allow-arp; do
      f="/etc/libvirt/$f.xml"
      echo "### $f"
      sudo cat "$f"
    done
  ### /etc/libvirt/storage/default.xml
  <pool type='dir'>
    <name>default</name>
    <target>
      <path>/var/lib/libvirt/images</path>
    </target>
  </pool>
  ### /etc/libvirt/qemu/networks/default.xml
  <network>
    <name>default</name>
    <forward mode='nat'/>
    <bridge name='virbr0'/>
    <mac address='52:54:00:42:5a:e7'/>
    <ip address='192.168.122.1' netmask='255.255.255.0'>
      <dhcp>
        <range start='192.168.122.2' end='192.168.122.254'/>
      </dhcp>
    </ip>
  </network>
  ### /etc/libvirt/qemu/cirros.xml
  <domain type='kvm'>
    <name>cirros</name>
    <memory unit='KiB'>131072</memory>
    <vcpu placement='static'>1</vcpu>
    <os>
      <type arch='x86_64' machine='q35'>hvm</type>
    </os>
    <features>
      <acpi/>
      <apic/>
    </features>
    <cpu mode='host-passthrough'/>
    <devices>
      <emulator>/usr/bin/qemu-system-x86_64</emulator>
      <disk type='file' device='disk'>
        <driver name='qemu' type='qcow2'/>
        <source file='/var/lib/libvirt/images/cirros.qcow2'/>
        <target dev='vda' bus='virtio'/>
      </disk>
      <controller type='usb' model='none'/>
      <controller type='pci' model='pcie-root'/>
      <serial type='pty'>
        <target type='isa-serial'/>
      </serial>
      <console type='pty'>
        <target type='serial'/>
      </console>
      <memballoon model='none'/>
      <rng model='virtio'>
        <backend model='random'>/dev/urandom</backend>
      </rng>
    </devices>
  </domain>
  ### /etc/libvirt/nwfilter/allow-arp.xml
  <filter name='allow-arp' chain='arp'>
    <rule action='accept' direction='inout'/>
  </filter>

  $ for f in storage/default qemu/networks/default \
             qemu/cirros nwfilter/allow-arp; do
      f="/etc/libvirt/$f.xml"
      sudo md5sum "$f"
    done
  be781432b877fa98092bc45610060fff  /etc/libvirt/storage/default.xml
  1233c7b5bd043a8d209e18abdccbb251  /etc/libvirt/qemu/networks/default.xml
  2976a665897bf8ce64a7dbf36af3adbf  /etc/libvirt/qemu/cirros.xml
  5fdb011f22de738feae3189f5f6b2e91  /etc/libvirt/nwfilter/allow-arp.xml

  $ sudo systemctl start libvirtd
  $ sudo systemctl stop libvirtd 2>/dev/null

  $ for f in storage/default qemu/networks/default \
             qemu/cirros nwfilter/allow-arp; do
      f="/etc/libvirt/$f.xml"
      sudo md5sum "$f"
    done
  be781432b877fa98092bc45610060fff  /etc/libvirt/storage/default.xml
  1233c7b5bd043a8d209e18abdccbb251  /etc/libvirt/qemu/networks/default.xml
  2976a665897bf8ce64a7dbf36af3adbf  /etc/libvirt/qemu/cirros.xml
  7aff93debdd3e646492d8763d03d6b62  /etc/libvirt/nwfilter/allow-arp.xml

Okay, so everything is the same except for the nwfilter. Let's see
what changed there:

  $ sudo cat /etc/libvirt/nwfilter/allow-arp.xml
  <!--
  WARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE
  OVERWRITTEN AND LOST. Changes to this xml configuration should be made using:
    virsh nwfilter-edit allow-arp
  or other application using the libvirt API.
  -->

  <filter name='allow-arp' chain='arp' priority='-500'>
    <uuid>e4487541-6b88-415c-9aaf-a7f1d3d4263a</uuid>
    <rule action='accept' direction='inout' priority='500'/>
  </filter>

This behavior is implemented in virNWFilterObjListLoadConfig():

  /* We generated a UUID, make it permanent by saving the config to disk */  
  if (!def->uuid_specified &&                                                
      virNWFilterSaveConfig(configDir, def) < 0)                             
      goto error;                                                            

Looks reasonable enough to me.

Of course, since other objects don't have similar snippets in the
relevant places, they will get a different UUID every single time the
daemon is restarted:

  $ sudo systemctl stop libvirtd 2>/dev/null
  $ for i in $(seq 1 3); do
      sleep 1; sudo systemctl start libvirtd
      sleep 1; sudo virsh pool-dumpxml --inactive default | grep uuid
      sudo systemctl stop libvirtd 2>/dev/null
    done
    <uuid>2bfe8afb-0366-443a-96cd-b706df4e5221</uuid>
    <uuid>6804e364-e3a9-494d-8054-93ebe85970a6</uuid>
    <uuid>6be460c9-8cfb-403c-a791-fdc1d505a0ee</uuid>

  $ sudo systemctl stop libvirtd 2>/dev/null
  $ for i in $(seq 1 3); do
      sleep 1; sudo systemctl start libvirtd
      sleep 1; sudo virsh net-dumpxml --inactive default | grep uuid
      sudo systemctl stop libvirtd 2>/dev/null
    done
    <uuid>22c61987-43a0-4499-a54d-d1247d36c881</uuid>
    <uuid>8360afda-4b5c-445d-97f8-abe81d877e7d</uuid>
    <uuid>821ddd59-c2b0-47c6-9d2f-0708e4e2a9bc</uuid>

  $ sudo systemctl stop libvirtd 2>/dev/null
  $ for i in $(seq 1 3); do
      sleep 1; sudo systemctl start libvirtd
      sleep 1; sudo virsh dumpxml --inactive cirros | grep uuid
      sudo systemctl stop libvirtd 2>/dev/null
    done
    <uuid>b74581ea-a215-4fe9-8365-c6cfceb2b18e</uuid>
    <uuid>d00d902b-ebca-4213-9411-d27f14363fdb</uuid>
    <uuid>74d50bb0-b54a-4a7e-82eb-410ccc12fd9b</uuid>

That's not cool, right? The configuration files might not have
reached /etc/libvirt the Proper Way™, but we stop so very short
of dealing with them reasonably...

If writing the file back to disk every single time is a concern due
to performance reasons, which I can actually see being the case for
large enough hosts, can we at least agree that when UUIDs and MAC
addresses have been generated by libvirt then we need to store them
persistently?

If nothing else, I would argue we definitely need this to be the case
for at least nwfilters, networks and storage pools, which are all
things for which some configuration can be reasonably provided
through the package manager - before the corresponding APIs are even
available.

-- 
Andrea Bolognani / Red Hat / Virtualization

On 11/17/20 12:45 PM, Andrea Bolognani wrote:
> On Tue, 2020-11-17 at 10:47 -0500, Laine Stump wrote:
>> On 11/16/20 11:43 AM, Andrea Bolognani wrote:
>>> I don't see how we could possibly not be okay with writing back the
>>> updated configuration: filling in whatever blanks are present in the
>>> user-provided configuration and then writing the result back to disk
>>> is quite central to how we handle domains and other objects, and I
>>> see no reason networks should behave any differently.
>>>
>>> If anything, I think we should not extend the check to UUIDs, but
>>> just drop it completely and unconditionally write the (possibly)
>>> updated file file to disk. I'll try cooking a patch for that.
>>
>> I understand why you would consider doing this (low/no maintenance,
>> easier to verify correct results), but I don't think it's a generally
>> good idea to rewrite every config file every time we restart libvirtd.
>> Some media has slow write speed and a limited number of writes in its
>> lifetime (sure, it's not 1982 and we're not talking about a 2716 EPROM,
>> but still :-)) and some people may rely on /etc/libvirt to be in a
>> read-only filesystem for some reason (I haven't tried that, but am just
>> assuming that it would work).
> 
> I would absolutely expect that to *not* be the case, if nothing else
> because I don't think anyone is actually QA-ing that scenario.
> 
> Even assuming it does indeed work, you'd end up with a pretty limited
> version of libvirt, where features like 'virsh define' don't work.
> 
> Honestly I would be very surprised if anyone actually used libvirt
> this way.

I never presume to know the depths of... er... "uniqueness" that users 
will plumb. (half /s) But yeah, sure - *I* would never do that, and 
would never recommend anyone to do that.

Hmm, although - maybe a copy-on-write overlay filesystem would be 
useful, and I recall (very possibly *incorrectly*) about some sort of 
copy-on-write overlay being used for a "live CD" (maybe even some 
incarnation of the Fedora Live CD?) that would keep track of all updates 
to the filesystem in a separate file/partition/whatever that would 
eventually fill up. (I may be mixing that up with some even-more-ancient 
memory of a filesystem for a write-once CD-R though - with my brain you 
just never know what you're gonna get!)


> 
>> Except for three cases (that I can think of; I'm sure there are others),
>> what is read from libvirt's config files in /etc is exactly what would
>> be written if we were to re-write it immediately after a read-parse-format:
>>
>> 1) the first read of an xml file that was generated outside libvirt, and
>> has been manually put into /etc/libvirt, e.g. xml files that are a part
>> of a package installation/upgrade. NB: all other cases of externally
>> generated files being placed in /etc/libvirt are expressly forbidden
>> (well, at least "discouraged and declared unsupported") by libvirt
>> documentation.
>>
>> 2) a user manually edits a file in /etc/libvirt - see (1)
> 
> Okay, but from libvirt's point of view there's really no way to tell
> apart a file that's been dropped into /etc/libvirt by the package
> manager as opposed to the admin, is there? So they will ultimately
> have to be treated the same way.

Yep. That's why the "see (1)". I'd already said "3", and couldn't go 
back on that (just pretend my up-arrow doesn't work, okay?), but 
realized while typing that (1) and (2) were essentially the same thing.

> 
>> 3) upgrading libvirt causes something to be parsed differently than it
>> was before. This should never happen, as there should always be enough
>> info in the xml we write to the disk after its initial arrival via the
>> libvirt API that any future read/write cycles are completely idempotent.
> 
> Except for randomly-generated data such as, I don't know, MAC
> addresses or UUIDs :)

Note that I said *should*.


> 
> So, I played with this a bit to understand the state of the art.
 >
> 
>    $ for f in storage/default qemu/networks/default \
>               qemu/cirros nwfilter/allow-arp; do
>        f="/etc/libvirt/$f.xml"
>        echo "### $f"
>        sudo cat "$f"
>      done
>    ### /etc/libvirt/storage/default.xml
>    <pool type='dir'>
>      <name>default</name>
>      <target>
>        <path>/var/lib/libvirt/images</path>
>      </target>
>    </pool>
>    ### /etc/libvirt/qemu/networks/default.xml
>    <network>
>      <name>default</name>
>      <forward mode='nat'/>
>      <bridge name='virbr0'/>
>      <mac address='52:54:00:42:5a:e7'/>
>      <ip address='192.168.122.1' netmask='255.255.255.0'>
>        <dhcp>
>          <range start='192.168.122.2' end='192.168.122.254'/>
>        </dhcp>
>      </ip>
>    </network>
>    ### /etc/libvirt/qemu/cirros.xml
>    <domain type='kvm'>
>      <name>cirros</name>
>      <memory unit='KiB'>131072</memory>
>      <vcpu placement='static'>1</vcpu>
>      <os>
>        <type arch='x86_64' machine='q35'>hvm</type>
>      </os>
>      <features>
>        <acpi/>
>        <apic/>
>      </features>
>      <cpu mode='host-passthrough'/>
>      <devices>
>        <emulator>/usr/bin/qemu-system-x86_64</emulator>
>        <disk type='file' device='disk'>
>          <driver name='qemu' type='qcow2'/>
>          <source file='/var/lib/libvirt/images/cirros.qcow2'/>
>          <target dev='vda' bus='virtio'/>
>        </disk>
>        <controller type='usb' model='none'/>
>        <controller type='pci' model='pcie-root'/>
>        <serial type='pty'>
>          <target type='isa-serial'/>
>        </serial>
>        <console type='pty'>
>          <target type='serial'/>
>        </console>
>        <memballoon model='none'/>
>        <rng model='virtio'>
>          <backend model='random'>/dev/urandom</backend>
>        </rng>
>      </devices>
>    </domain>
>    ### /etc/libvirt/nwfilter/allow-arp.xml
>    <filter name='allow-arp' chain='arp'>
>      <rule action='accept' direction='inout'/>
>    </filter>
> 
>    $ for f in storage/default qemu/networks/default \
>               qemu/cirros nwfilter/allow-arp; do
>        f="/etc/libvirt/$f.xml"
>        sudo md5sum "$f"
>      done
>    be781432b877fa98092bc45610060fff  /etc/libvirt/storage/default.xml
>    1233c7b5bd043a8d209e18abdccbb251  /etc/libvirt/qemu/networks/default.xml
>    2976a665897bf8ce64a7dbf36af3adbf  /etc/libvirt/qemu/cirros.xml
>    5fdb011f22de738feae3189f5f6b2e91  /etc/libvirt/nwfilter/allow-arp.xml
> 
>    $ sudo systemctl start libvirtd
>    $ sudo systemctl stop libvirtd 2>/dev/null
> 
>    $ for f in storage/default qemu/networks/default \
>               qemu/cirros nwfilter/allow-arp; do
>        f="/etc/libvirt/$f.xml"
>        sudo md5sum "$f"
>      done
>    be781432b877fa98092bc45610060fff  /etc/libvirt/storage/default.xml
>    1233c7b5bd043a8d209e18abdccbb251  /etc/libvirt/qemu/networks/default.xml
>    2976a665897bf8ce64a7dbf36af3adbf  /etc/libvirt/qemu/cirros.xml
>    7aff93debdd3e646492d8763d03d6b62  /etc/libvirt/nwfilter/allow-arp.xml
> 
> Okay, so everything is the same except for the nwfilter. Let's see
> what changed there:
> 
>    $ sudo cat /etc/libvirt/nwfilter/allow-arp.xml
>    <!--
>    WARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE
>    OVERWRITTEN AND LOST. Changes to this xml configuration should be made using:
>      virsh nwfilter-edit allow-arp
>    or other application using the libvirt API.
>    -->
> 
>    <filter name='allow-arp' chain='arp' priority='-500'>
>      <uuid>e4487541-6b88-415c-9aaf-a7f1d3d4263a</uuid>
>      <rule action='accept' direction='inout' priority='500'/>
>    </filter>
> 
> This behavior is implemented in virNWFilterObjListLoadConfig():
> 
>    /* We generated a UUID, make it permanent by saving the config to disk */
>    if (!def->uuid_specified &&
>        virNWFilterSaveConfig(configDir, def) < 0)
>        goto error;
> 
> Looks reasonable enough to me.
> 
> Of course, since other objects don't have similar snippets in the
> relevant places, they will get a different UUID every single time the
> daemon is restarted:
> 
>    $ sudo systemctl stop libvirtd 2>/dev/null
>    $ for i in $(seq 1 3); do
>        sleep 1; sudo systemctl start libvirtd
>        sleep 1; sudo virsh pool-dumpxml --inactive default | grep uuid
>        sudo systemctl stop libvirtd 2>/dev/null
>      done
>      <uuid>2bfe8afb-0366-443a-96cd-b706df4e5221</uuid>
>      <uuid>6804e364-e3a9-494d-8054-93ebe85970a6</uuid>
>      <uuid>6be460c9-8cfb-403c-a791-fdc1d505a0ee</uuid>
> 
>    $ sudo systemctl stop libvirtd 2>/dev/null
>    $ for i in $(seq 1 3); do
>        sleep 1; sudo systemctl start libvirtd
>        sleep 1; sudo virsh net-dumpxml --inactive default | grep uuid
>        sudo systemctl stop libvirtd 2>/dev/null
>      done
>      <uuid>22c61987-43a0-4499-a54d-d1247d36c881</uuid>
>      <uuid>8360afda-4b5c-445d-97f8-abe81d877e7d</uuid>
>      <uuid>821ddd59-c2b0-47c6-9d2f-0708e4e2a9bc</uuid>
> 
>    $ sudo systemctl stop libvirtd 2>/dev/null
>    $ for i in $(seq 1 3); do
>        sleep 1; sudo systemctl start libvirtd
>        sleep 1; sudo virsh dumpxml --inactive cirros | grep uuid
>        sudo systemctl stop libvirtd 2>/dev/null
>      done
>      <uuid>b74581ea-a215-4fe9-8365-c6cfceb2b18e</uuid>
>      <uuid>d00d902b-ebca-4213-9411-d27f14363fdb</uuid>
>      <uuid>74d50bb0-b54a-4a7e-82eb-410ccc12fd9b</uuid>
> 
> That's not cool, right? 

The fact that you took the time to do such a nicely documented 
experiment *is* cool. The results are not, though.

> The configuration files might not have
> reached /etc/libvirt the Proper Way™, but we stop so very short
> of dealing with them reasonably...

The most bothersome part of this to me is that the treatment is 
inconsistent. If we're going to re-write the config when parsing causes 
a change, then we should do that for all types of config, and all 
changes. And that goes double for changes to things that are supposed to 
remain stable (which, I guess, is "everything").

> If writing the file back to disk every single time is a concern due
> to performance reasons, which I can actually see being the case for
> large enough hosts, can we at least agree that when UUIDs and MAC
> addresses have been generated by libvirt then we need to store them
> persistently?

Yep. I'm right there with you on this - I was only objecting to the idea 
of always re-writing no matter what.

> If nothing else, I would argue we definitely need this to be the case
> for at least nwfilters, networks and storage pools, which are all
> things for which some configuration can be reasonably provided
> through the package manager - before the corresponding APIs are even
> available.

I think it should be done consistently across all config objects. I 
would suggest making some sort of pseudo-standardized function/table for 
every config object type that would maintain a list of those items that 
need to be checked, but that just seems like over-engineering that would 
lead to lots of code that not everybody would know about and/or 
remember, and so it wouldn't be maintained anyway...