numa.c | 5 +++++ qemu-options.hx | 12 ++++++++---- 2 files changed, 13 insertions(+), 4 deletions(-)
Amend -numa option docs and print warnings if 'mem' option or default RAM
splitting between nodes is used. It's intended to discourage users from using
configuration that allows only to fake NUMA on guest side while leading
to reduced performance of the guest due to inability to properly configure
VM's RAM on the host.
In NUMA case, it's recommended to always explicitly configure guest RAM
using -numa node,memdev={backend-id} option.
Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
numa.c | 5 +++++
qemu-options.hx | 12 ++++++++----
2 files changed, 13 insertions(+), 4 deletions(-)
diff --git a/numa.c b/numa.c
index 3875e1e..42838f9 100644
--- a/numa.c
+++ b/numa.c
@@ -121,6 +121,8 @@ static void parse_numa_node(MachineState *ms, NumaNodeOptions *node,
if (node->has_mem) {
numa_info[nodenr].node_mem = node->mem;
+ warn_report("Parameter -numa node,mem is obsolete,"
+ " use -numa node,memdev instead");
}
if (node->has_memdev) {
Object *o;
@@ -407,6 +409,9 @@ void numa_complete_configuration(MachineState *ms)
if (i == nb_numa_nodes) {
assert(mc->numa_auto_assign_ram);
mc->numa_auto_assign_ram(mc, numa_info, nb_numa_nodes, ram_size);
+ warn_report("Default splitting of RAM between nodes is obsolete,"
+ " Use '-numa node,memdev' to explicitly define RAM"
+ " allocation per node");
}
numa_total = 0;
diff --git a/qemu-options.hx b/qemu-options.hx
index 1cf9aac..61035cb 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -206,10 +206,14 @@ For example:
-numa cpu,node-id=0,socket-id=0 -numa cpu,node-id=1,socket-id=1
@end example
-@samp{mem} assigns a given RAM amount to a node. @samp{memdev}
-assigns RAM from a given memory backend device to a node. If
-@samp{mem} and @samp{memdev} are omitted in all nodes, RAM is
-split equally between them.
+@samp{memdev} assigns RAM from a given memory backend device to a node.
+
+Legacy options/behaviour: @samp{mem} assigns a given RAM amount to a node.
+If @samp{mem} and @samp{memdev} are omitted in all nodes, RAM is split equally
+between them. Option @samp{mem} and default RAM splitting are obsolete as they
+do not provide means to manage RAM on the host side and only allow QEMU to fake
+NUMA support which in practice could degrade VM performance.
+It's advised to always explicitly configure NUMA RAM by using the @samp{memdev} option.
@samp{mem} and @samp{memdev} are mutually exclusive. Furthermore,
if one node uses @samp{memdev}, all of them have to use it.
--
2.7.4
On Wed, Mar 06, 2019 at 06:33:25PM +0100, Igor Mammedov wrote:
> Amend -numa option docs and print warnings if 'mem' option or default RAM
> splitting between nodes is used. It's intended to discourage users from using
> configuration that allows only to fake NUMA on guest side while leading
> to reduced performance of the guest due to inability to properly configure
> VM's RAM on the host.
>
> In NUMA case, it's recommended to always explicitly configure guest RAM
> using -numa node,memdev={backend-id} option.
>
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> ---
> numa.c | 5 +++++
> qemu-options.hx | 12 ++++++++----
> 2 files changed, 13 insertions(+), 4 deletions(-)
>
> diff --git a/numa.c b/numa.c
> index 3875e1e..42838f9 100644
> --- a/numa.c
> +++ b/numa.c
> @@ -121,6 +121,8 @@ static void parse_numa_node(MachineState *ms, NumaNodeOptions *node,
>
> if (node->has_mem) {
> numa_info[nodenr].node_mem = node->mem;
> + warn_report("Parameter -numa node,mem is obsolete,"
> + " use -numa node,memdev instead");
My comments from v1 still apply. We must not do this as long as
libvirt has no choice but to continue using this feature.
> }
> if (node->has_memdev) {
> Object *o;
> @@ -407,6 +409,9 @@ void numa_complete_configuration(MachineState *ms)
> if (i == nb_numa_nodes) {
> assert(mc->numa_auto_assign_ram);
> mc->numa_auto_assign_ram(mc, numa_info, nb_numa_nodes, ram_size);
> + warn_report("Default splitting of RAM between nodes is obsolete,"
> + " Use '-numa node,memdev' to explicitly define RAM"
> + " allocation per node");
> }
>
> numa_total = 0;
> diff --git a/qemu-options.hx b/qemu-options.hx
> index 1cf9aac..61035cb 100644
> --- a/qemu-options.hx
> +++ b/qemu-options.hx
> @@ -206,10 +206,14 @@ For example:
> -numa cpu,node-id=0,socket-id=0 -numa cpu,node-id=1,socket-id=1
> @end example
>
> -@samp{mem} assigns a given RAM amount to a node. @samp{memdev}
> -assigns RAM from a given memory backend device to a node. If
> -@samp{mem} and @samp{memdev} are omitted in all nodes, RAM is
> -split equally between them.
> +@samp{memdev} assigns RAM from a given memory backend device to a node.
> +
> +Legacy options/behaviour: @samp{mem} assigns a given RAM amount to a node.
> +If @samp{mem} and @samp{memdev} are omitted in all nodes, RAM is split equally
> +between them. Option @samp{mem} and default RAM splitting are obsolete as they
> +do not provide means to manage RAM on the host side and only allow QEMU to fake
> +NUMA support which in practice could degrade VM performance.
> +It's advised to always explicitly configure NUMA RAM by using the @samp{memdev} option.
>
> @samp{mem} and @samp{memdev} are mutually exclusive. Furthermore,
> if one node uses @samp{memdev}, all of them have to use it.
> --
> 2.7.4
>
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
On Wed, 6 Mar 2019 18:16:08 +0000
Daniel P. Berrangé <berrange@redhat.com> wrote:
> On Wed, Mar 06, 2019 at 06:33:25PM +0100, Igor Mammedov wrote:
> > Amend -numa option docs and print warnings if 'mem' option or default RAM
> > splitting between nodes is used. It's intended to discourage users from using
> > configuration that allows only to fake NUMA on guest side while leading
> > to reduced performance of the guest due to inability to properly configure
> > VM's RAM on the host.
> >
> > In NUMA case, it's recommended to always explicitly configure guest RAM
> > using -numa node,memdev={backend-id} option.
> >
> > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > ---
> > numa.c | 5 +++++
> > qemu-options.hx | 12 ++++++++----
> > 2 files changed, 13 insertions(+), 4 deletions(-)
> >
> > diff --git a/numa.c b/numa.c
> > index 3875e1e..42838f9 100644
> > --- a/numa.c
> > +++ b/numa.c
> > @@ -121,6 +121,8 @@ static void parse_numa_node(MachineState *ms, NumaNodeOptions *node,
> >
> > if (node->has_mem) {
> > numa_info[nodenr].node_mem = node->mem;
> > + warn_report("Parameter -numa node,mem is obsolete,"
> > + " use -numa node,memdev instead");
>
> My comments from v1 still apply. We must not do this as long as
> libvirt has no choice but to continue using this feature.
It has a choice to use 'memdev' whenever creating a new VM and continue using 'mem' with exiting VMs.
>
> > }
> > if (node->has_memdev) {
> > Object *o;
> > @@ -407,6 +409,9 @@ void numa_complete_configuration(MachineState *ms)
> > if (i == nb_numa_nodes) {
> > assert(mc->numa_auto_assign_ram);
> > mc->numa_auto_assign_ram(mc, numa_info, nb_numa_nodes, ram_size);
> > + warn_report("Default splitting of RAM between nodes is obsolete,"
> > + " Use '-numa node,memdev' to explicitly define RAM"
> > + " allocation per node");
> > }
> >
> > numa_total = 0;
> > diff --git a/qemu-options.hx b/qemu-options.hx
> > index 1cf9aac..61035cb 100644
> > --- a/qemu-options.hx
> > +++ b/qemu-options.hx
> > @@ -206,10 +206,14 @@ For example:
> > -numa cpu,node-id=0,socket-id=0 -numa cpu,node-id=1,socket-id=1
> > @end example
> >
> > -@samp{mem} assigns a given RAM amount to a node. @samp{memdev}
> > -assigns RAM from a given memory backend device to a node. If
> > -@samp{mem} and @samp{memdev} are omitted in all nodes, RAM is
> > -split equally between them.
> > +@samp{memdev} assigns RAM from a given memory backend device to a node.
> > +
> > +Legacy options/behaviour: @samp{mem} assigns a given RAM amount to a node.
> > +If @samp{mem} and @samp{memdev} are omitted in all nodes, RAM is split equally
> > +between them. Option @samp{mem} and default RAM splitting are obsolete as they
> > +do not provide means to manage RAM on the host side and only allow QEMU to fake
> > +NUMA support which in practice could degrade VM performance.
> > +It's advised to always explicitly configure NUMA RAM by using the @samp{memdev} option.
> >
> > @samp{mem} and @samp{memdev} are mutually exclusive. Furthermore,
> > if one node uses @samp{memdev}, all of them have to use it.
> > --
> > 2.7.4
> >
>
> Regards,
> Daniel
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
On Wed, Mar 06, 2019 at 07:54:17PM +0100, Igor Mammedov wrote:
> On Wed, 6 Mar 2019 18:16:08 +0000
> Daniel P. Berrangé <berrange@redhat.com> wrote:
>
> > On Wed, Mar 06, 2019 at 06:33:25PM +0100, Igor Mammedov wrote:
> > > Amend -numa option docs and print warnings if 'mem' option or default RAM
> > > splitting between nodes is used. It's intended to discourage users from using
> > > configuration that allows only to fake NUMA on guest side while leading
> > > to reduced performance of the guest due to inability to properly configure
> > > VM's RAM on the host.
> > >
> > > In NUMA case, it's recommended to always explicitly configure guest RAM
> > > using -numa node,memdev={backend-id} option.
> > >
> > > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > > ---
> > > numa.c | 5 +++++
> > > qemu-options.hx | 12 ++++++++----
> > > 2 files changed, 13 insertions(+), 4 deletions(-)
> > >
> > > diff --git a/numa.c b/numa.c
> > > index 3875e1e..42838f9 100644
> > > --- a/numa.c
> > > +++ b/numa.c
> > > @@ -121,6 +121,8 @@ static void parse_numa_node(MachineState *ms, NumaNodeOptions *node,
> > >
> > > if (node->has_mem) {
> > > numa_info[nodenr].node_mem = node->mem;
> > > + warn_report("Parameter -numa node,mem is obsolete,"
> > > + " use -numa node,memdev instead");
> >
> > My comments from v1 still apply. We must not do this as long as
> > libvirt has no choice but to continue using this feature.
> It has a choice to use 'memdev' whenever creating a new VM and continue
> using 'mem' with exiting VMs.
Unfortunately we don't have such a choice. Libvirt has no concept of the
distinction between an 'existing' and 'new' VM. It just receives an XML
file from the mgmt application and with transient guests, we have no
persistent configuration record of the VM. So we've no way of knowing
whether this VM was previously running on this same host, or another
host, or is completely new.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
On Thu, 7 Mar 2019 10:07:05 +0000
Daniel P. Berrangé <berrange@redhat.com> wrote:
> On Wed, Mar 06, 2019 at 07:54:17PM +0100, Igor Mammedov wrote:
> > On Wed, 6 Mar 2019 18:16:08 +0000
> > Daniel P. Berrangé <berrange@redhat.com> wrote:
> >
> > > On Wed, Mar 06, 2019 at 06:33:25PM +0100, Igor Mammedov wrote:
> > > > Amend -numa option docs and print warnings if 'mem' option or default RAM
> > > > splitting between nodes is used. It's intended to discourage users from using
> > > > configuration that allows only to fake NUMA on guest side while leading
> > > > to reduced performance of the guest due to inability to properly configure
> > > > VM's RAM on the host.
> > > >
> > > > In NUMA case, it's recommended to always explicitly configure guest RAM
> > > > using -numa node,memdev={backend-id} option.
> > > >
> > > > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > > > ---
> > > > numa.c | 5 +++++
> > > > qemu-options.hx | 12 ++++++++----
> > > > 2 files changed, 13 insertions(+), 4 deletions(-)
> > > >
> > > > diff --git a/numa.c b/numa.c
> > > > index 3875e1e..42838f9 100644
> > > > --- a/numa.c
> > > > +++ b/numa.c
> > > > @@ -121,6 +121,8 @@ static void parse_numa_node(MachineState *ms, NumaNodeOptions *node,
> > > >
> > > > if (node->has_mem) {
> > > > numa_info[nodenr].node_mem = node->mem;
> > > > + warn_report("Parameter -numa node,mem is obsolete,"
> > > > + " use -numa node,memdev instead");
> > >
> > > My comments from v1 still apply. We must not do this as long as
> > > libvirt has no choice but to continue using this feature.
> > It has a choice to use 'memdev' whenever creating a new VM and continue
> > using 'mem' with exiting VMs.
>
> Unfortunately we don't have such a choice. Libvirt has no concept of the
> distinction between an 'existing' and 'new' VM. It just receives an XML
> file from the mgmt application and with transient guests, we have no
> persistent configuration record of the VM. So we've no way of knowing
> whether this VM was previously running on this same host, or another
> host, or is completely new.
In case of transient VM, libvirt might be able to use machine version
as deciding which option to use (memdev is around more than 4 years since 2.1)
(or QEMU could provide introspection into what machine version (not)supports,
like it was discussed before)
As discussed elsewhere (v1 tread|IRC), there are users (mainly CI) for which
fake NUMA is sufficient and they do not ask for explicit pinning, so libvirt
defaults to legacy -numa node,mem option.
Those users do not care no aware that they should use memdev instead
(I'm not sure if they are able to ask libvirt for non pinned numa memory
which results in memdev being used).
This patch doesn't obsolete anything yet, it serves purpose to inform users
that they are using legacy option and advises replacement option
so that users would know to what they should adapt to.
Once we deprecate and then remove 'mem' for new machines only (while keeping
'mem' working on old machine versions). The new nor old libvirt won't be able
to start new machine type with 'mem' option and have to use memdev variant,
so we don't have migration issues with new machines and old ones continue
working with 'mem'.
That keeps QEMU's promise not to break existing configurations while let us
move forward with new machines.
> Regards,
> Daniel
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
On Tue, Mar 19, 2019 at 02:08:01PM +0100, Igor Mammedov wrote:
> On Thu, 7 Mar 2019 10:07:05 +0000
> Daniel P. Berrangé <berrange@redhat.com> wrote:
>
> > On Wed, Mar 06, 2019 at 07:54:17PM +0100, Igor Mammedov wrote:
> > > On Wed, 6 Mar 2019 18:16:08 +0000
> > > Daniel P. Berrangé <berrange@redhat.com> wrote:
> > >
> > > > On Wed, Mar 06, 2019 at 06:33:25PM +0100, Igor Mammedov wrote:
> > > > > Amend -numa option docs and print warnings if 'mem' option or default RAM
> > > > > splitting between nodes is used. It's intended to discourage users from using
> > > > > configuration that allows only to fake NUMA on guest side while leading
> > > > > to reduced performance of the guest due to inability to properly configure
> > > > > VM's RAM on the host.
> > > > >
> > > > > In NUMA case, it's recommended to always explicitly configure guest RAM
> > > > > using -numa node,memdev={backend-id} option.
> > > > >
> > > > > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > > > > ---
> > > > > numa.c | 5 +++++
> > > > > qemu-options.hx | 12 ++++++++----
> > > > > 2 files changed, 13 insertions(+), 4 deletions(-)
> > > > >
> > > > > diff --git a/numa.c b/numa.c
> > > > > index 3875e1e..42838f9 100644
> > > > > --- a/numa.c
> > > > > +++ b/numa.c
> > > > > @@ -121,6 +121,8 @@ static void parse_numa_node(MachineState *ms, NumaNodeOptions *node,
> > > > >
> > > > > if (node->has_mem) {
> > > > > numa_info[nodenr].node_mem = node->mem;
> > > > > + warn_report("Parameter -numa node,mem is obsolete,"
> > > > > + " use -numa node,memdev instead");
> > > >
> > > > My comments from v1 still apply. We must not do this as long as
> > > > libvirt has no choice but to continue using this feature.
> > > It has a choice to use 'memdev' whenever creating a new VM and continue
> > > using 'mem' with exiting VMs.
> >
> > Unfortunately we don't have such a choice. Libvirt has no concept of the
> > distinction between an 'existing' and 'new' VM. It just receives an XML
> > file from the mgmt application and with transient guests, we have no
> > persistent configuration record of the VM. So we've no way of knowing
> > whether this VM was previously running on this same host, or another
> > host, or is completely new.
> In case of transient VM, libvirt might be able to use machine version
> as deciding which option to use (memdev is around more than 4 years since 2.1)
> (or QEMU could provide introspection into what machine version (not)supports,
> like it was discussed before)
>
> As discussed elsewhere (v1 tread|IRC), there are users (mainly CI) for which
> fake NUMA is sufficient and they do not ask for explicit pinning, so libvirt
> defaults to legacy -numa node,mem option.
> Those users do not care no aware that they should use memdev instead
> (I'm not sure if they are able to ask libvirt for non pinned numa memory
> which results in memdev being used).
> This patch doesn't obsolete anything yet, it serves purpose to inform users
> that they are using legacy option and advises replacement option
> so that users would know to what they should adapt to.
>
> Once we deprecate and then remove 'mem' for new machines only (while keeping
> 'mem' working on old machine versions). The new nor old libvirt won't be able
> to start new machine type with 'mem' option and have to use memdev variant,
> so we don't have migration issues with new machines and old ones continue
> working with 'mem'.
I'm not seeing what has changed which would enable us to deprecate
something only for new machines. That's not possible from libvirt's
POV as old libvirt will support new machines & thus we have to
continue using "mem" for all machines in the scenarios where we
currently use it.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Daniel P. Berrangé <berrange@redhat.com> writes:
> On Tue, Mar 19, 2019 at 02:08:01PM +0100, Igor Mammedov wrote:
>> On Thu, 7 Mar 2019 10:07:05 +0000
>> Daniel P. Berrangé <berrange@redhat.com> wrote:
>>
>> > On Wed, Mar 06, 2019 at 07:54:17PM +0100, Igor Mammedov wrote:
>> > > On Wed, 6 Mar 2019 18:16:08 +0000
>> > > Daniel P. Berrangé <berrange@redhat.com> wrote:
[...]
>> > > > My comments from v1 still apply. We must not do this as long as
>> > > > libvirt has no choice but to continue using this feature.
>> > > It has a choice to use 'memdev' whenever creating a new VM and continue
>> > > using 'mem' with exiting VMs.
>> >
>> > Unfortunately we don't have such a choice. Libvirt has no concept of the
>> > distinction between an 'existing' and 'new' VM. It just receives an XML
>> > file from the mgmt application and with transient guests, we have no
>> > persistent configuration record of the VM. So we've no way of knowing
>> > whether this VM was previously running on this same host, or another
>> > host, or is completely new.
>> In case of transient VM, libvirt might be able to use machine version
>> as deciding which option to use (memdev is around more than 4 years since 2.1)
>> (or QEMU could provide introspection into what machine version (not)supports,
>> like it was discussed before)
>>
>> As discussed elsewhere (v1 tread|IRC), there are users (mainly CI) for which
>> fake NUMA is sufficient and they do not ask for explicit pinning, so libvirt
>> defaults to legacy -numa node,mem option.
>> Those users do not care no aware that they should use memdev instead
>> (I'm not sure if they are able to ask libvirt for non pinned numa memory
>> which results in memdev being used).
>> This patch doesn't obsolete anything yet, it serves purpose to inform users
>> that they are using legacy option and advises replacement option
>> so that users would know to what they should adapt to.
>>
>> Once we deprecate and then remove 'mem' for new machines only (while keeping
>> 'mem' working on old machine versions). The new nor old libvirt won't be able
>> to start new machine type with 'mem' option and have to use memdev variant,
>> so we don't have migration issues with new machines and old ones continue
>> working with 'mem'.
>
> I'm not seeing what has changed which would enable us to deprecate
> something only for new machines. That's not possible from libvirt's
> POV as old libvirt will support new machines & thus we have to
> continue using "mem" for all machines in the scenarios where we
> currently use it.
We're going in circles. Igor keeps telling you QEMU needs to shed dead
weight, badly. In Igor's words:
We really need to figure out how to introduce breaking change on
management (CLI) side* in QEMU and make it digestible for libvirt
and others.
(* at least for new machine types).
You keep telling us QEMU can't ever deprecate stuff libvirt uses,
because libvirt promised forward and backward compatibility forever.
Okay, my turn to speak in absolutes.
QEMU needs to evolve, or it'll go extinct.
If we remain complacent about the rate QEMU currently evolves (and the
herculean effort evolving it often takes), QEMU will go extinct.
If we permit overambitious promises of backward compatibility to hold us
back, QEMU will go extinct.
If we permit overambitious promises *some other project made* to hold us
back, QEMU will go extinct.
End of absolutes.
I'm with Igor on this one. I'm all for QEMU going the extra mile to
help libvirt, simply because that helps a very large fraction of our
users. I'm now asking libvirt to extend the courtesy back to QEMU.
Please sit down and think earnestly about how to best soften the
compatibility promise you made so you can cope with changes we feel QEMU
has to make.
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
On Wed, Mar 20, 2019 at 07:46:33 +0100, Markus Armbruster wrote: [...] > I'm with Igor on this one. I'm all for QEMU going the extra mile to > help libvirt, simply because that helps a very large fraction of our > users. I'm now asking libvirt to extend the courtesy back to QEMU. > > Please sit down and think earnestly about how to best soften the > compatibility promise you made so you can cope with changes we feel QEMU > has to make. The compatibility promise of libvirt can in this case go only as far as qemu is going to allow us. Obviously qemu dropping that option would not be nice unless libvirt has a contingency plan, but once it's gone we can't do much more. Libvirt needs to adapt as well so that we do our job of shielding users from inconsistencies of configuration. Peter Disclaimer: I did not think how to approach this in libvirt at this point. -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
On Wed, Mar 20, 2019 at 07:46:33AM +0100, Markus Armbruster wrote: > We're going in circles. Igor keeps telling you QEMU needs to shed dead > weight, badly. In Igor's words: > > We really need to figure out how to introduce breaking change on > management (CLI) side* in QEMU and make it digestible for libvirt > and others. > (* at least for new machine types). > > You keep telling us QEMU can't ever deprecate stuff libvirt uses, > because libvirt promised forward and backward compatibility forever. Note that libvirt didn't want to promise compatibility with live migration from new -> old libvirt. We did break this a few times in the past, and we received very clear feedback that users/mgmt apps don't want their live migration to be broken in this way. > I'm with Igor on this one. I'm all for QEMU going the extra mile to > help libvirt, simply because that helps a very large fraction of our > users. I'm now asking libvirt to extend the courtesy back to QEMU. This isn't about helping libvirt - this is about helping the users of libvirt & QEMU, who *want* this back compatibility to be able to live migrate their VMs in both directions. Any time libvirt has had problems in this area we get bug reports requiring us to fix it. This is why we don't want to do a change which would knowingly create a problem which will result in more bugs being reported against libvirt/QEMU > Please sit down and think earnestly about how to best soften the > compatibility promise you made so you can cope with changes we feel QEMU > has to make. Please don't blame libvirt for giving users the live migration compatibility we have been asked to provide to them. QEMU can change its impl, but users none the less expect live migration to remain compatible for their VMs. I did think initially we could do this by assocating the changed syntax with the machine type, until I was reminded that this does not work for the backwards compatibility direction, which users and mgmt apps have required libvirt to support. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Daniel P. Berrangé <berrange@redhat.com> writes: > On Wed, Mar 20, 2019 at 07:46:33AM +0100, Markus Armbruster wrote: >> We're going in circles. Igor keeps telling you QEMU needs to shed dead >> weight, badly. In Igor's words: >> >> We really need to figure out how to introduce breaking change on >> management (CLI) side* in QEMU and make it digestible for libvirt >> and others. >> (* at least for new machine types). >> >> You keep telling us QEMU can't ever deprecate stuff libvirt uses, >> because libvirt promised forward and backward compatibility forever. > > Note that libvirt didn't want to promise compatibility with live > migration from new -> old libvirt. We did break this a few times > in the past, and we received very clear feedback that users/mgmt > apps don't want their live migration to be broken in this way. > >> I'm with Igor on this one. I'm all for QEMU going the extra mile to >> help libvirt, simply because that helps a very large fraction of our >> users. I'm now asking libvirt to extend the courtesy back to QEMU. > > This isn't about helping libvirt - this is about helping the users of > libvirt & QEMU, We're in violent agreement on this point. However, ... > who *want* this back compatibility to be able to live > migrate their VMs in both directions. ... the users who want this are not the only users who matter, and I doubt even they want it at any cost. When dragging around a bad idea forever is expensive, we need a way to phase it out in an orderly manner. Similarly, when defaults go bad, we need a way to transition to better ones. QEMU has a way: versioned machine types. You've told us not to use them for that, because (if I understood you correctly) it won't work when a live migration's destination runs a sufficiently outdated libvirt. So, it's not just "we must maintain live migration compatibility backwards and forwards, forever, and at any cost", it's "the same, regardless of libvirt version". Sorry, that feels like at least one bridge too far. I've said that before[*], but got no reply. Perhaps the proposal I made there is unworkable. I've been wrong before. All I want is ... > Any time libvirt has had problems > in this area we get bug reports requiring us to fix it. This is why we > don't want to do a change which would knowingly create a problem which > will result in more bugs being reported against libvirt/QEMU ... this: >> Please sit down and think earnestly about how to best soften the >> compatibility promise you made so you can cope with changes we feel QEMU >> has to make. > > Please don't blame libvirt for giving users the live migration > compatibility we have been asked to provide to them. Our users' wish for maximal migration compatibility is understandable. However, "maximal" comes at a price. Buy paying it uncritically, we rob other users (or perhaps even the same ones) of improvments we could've bought instead. Please sit down and think earnestly what could be done in libvirt to let QEMU use versioned machine types for deprecating features. Don't tell us "nothing can be done". That feels as if you were trying to take us hostage. It's not meant that way, but it feels that way. Tell us what could be done. Describe the drawbacks. Feel free to tell us why you'd rather not do it. We can take it from there. > QEMU can change its impl, but users none the less expect live > migration to remain compatible for their VMs. > > I did think initially we could do this by assocating the changed > syntax with the machine type, until I was reminded that this does > not work for the backwards compatibility direction, which users > and mgmt apps have required libvirt to support. [*] Message-ID: <871s3fxce6.fsf@dusky.pond.sub.org> https://lists.nongnu.org/archive/html/qemu-devel/2019-03/msg03052.html -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
On Tue, 19 Mar 2019 14:51:07 +0000
Daniel P. Berrangé <berrange@redhat.com> wrote:
> On Tue, Mar 19, 2019 at 02:08:01PM +0100, Igor Mammedov wrote:
> > On Thu, 7 Mar 2019 10:07:05 +0000
> > Daniel P. Berrangé <berrange@redhat.com> wrote:
> >
> > > On Wed, Mar 06, 2019 at 07:54:17PM +0100, Igor Mammedov wrote:
> > > > On Wed, 6 Mar 2019 18:16:08 +0000
> > > > Daniel P. Berrangé <berrange@redhat.com> wrote:
> > > >
> > > > > On Wed, Mar 06, 2019 at 06:33:25PM +0100, Igor Mammedov wrote:
> > > > > > Amend -numa option docs and print warnings if 'mem' option or default RAM
> > > > > > splitting between nodes is used. It's intended to discourage users from using
> > > > > > configuration that allows only to fake NUMA on guest side while leading
> > > > > > to reduced performance of the guest due to inability to properly configure
> > > > > > VM's RAM on the host.
> > > > > >
> > > > > > In NUMA case, it's recommended to always explicitly configure guest RAM
> > > > > > using -numa node,memdev={backend-id} option.
> > > > > >
> > > > > > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > > > > > ---
> > > > > > numa.c | 5 +++++
> > > > > > qemu-options.hx | 12 ++++++++----
> > > > > > 2 files changed, 13 insertions(+), 4 deletions(-)
> > > > > >
> > > > > > diff --git a/numa.c b/numa.c
> > > > > > index 3875e1e..42838f9 100644
> > > > > > --- a/numa.c
> > > > > > +++ b/numa.c
> > > > > > @@ -121,6 +121,8 @@ static void parse_numa_node(MachineState *ms, NumaNodeOptions *node,
> > > > > >
> > > > > > if (node->has_mem) {
> > > > > > numa_info[nodenr].node_mem = node->mem;
> > > > > > + warn_report("Parameter -numa node,mem is obsolete,"
> > > > > > + " use -numa node,memdev instead");
> > > > >
> > > > > My comments from v1 still apply. We must not do this as long as
> > > > > libvirt has no choice but to continue using this feature.
> > > > It has a choice to use 'memdev' whenever creating a new VM and continue
> > > > using 'mem' with exiting VMs.
> > >
> > > Unfortunately we don't have such a choice. Libvirt has no concept of the
> > > distinction between an 'existing' and 'new' VM. It just receives an XML
> > > file from the mgmt application and with transient guests, we have no
> > > persistent configuration record of the VM. So we've no way of knowing
> > > whether this VM was previously running on this same host, or another
> > > host, or is completely new.
> > In case of transient VM, libvirt might be able to use machine version
> > as deciding which option to use (memdev is around more than 4 years since 2.1)
> > (or QEMU could provide introspection into what machine version (not)supports,
> > like it was discussed before)
> >
> > As discussed elsewhere (v1 tread|IRC), there are users (mainly CI) for which
> > fake NUMA is sufficient and they do not ask for explicit pinning, so libvirt
> > defaults to legacy -numa node,mem option.
> > Those users do not care no aware that they should use memdev instead
> > (I'm not sure if they are able to ask libvirt for non pinned numa memory
> > which results in memdev being used).
> > This patch doesn't obsolete anything yet, it serves purpose to inform users
> > that they are using legacy option and advises replacement option
> > so that users would know to what they should adapt to.
> >
> > Once we deprecate and then remove 'mem' for new machines only (while keeping
> > 'mem' working on old machine versions). The new nor old libvirt won't be able
> > to start new machine type with 'mem' option and have to use memdev variant,
> > so we don't have migration issues with new machines and old ones continue
> > working with 'mem'.
>
> I'm not seeing what has changed which would enable us to deprecate
> something only for new machines. That's not possible from libvirt's
> POV as old libvirt will support new machines & thus we have to
> continue using "mem" for all machines in the scenarios where we
> currently use it.
There are several issues here:
1. how old libvirt you are talking about?
2. old libvirt + new QEMU won't be able to start QEMU with
new machine with 'mem' option so we don't have live migration,
it's rather management issue where mgmt should not try to migrate
to such host (if it manged to end up with not compatible package
bundle it is not QEMU nor libvirt problem per se).
3. in generic dropping features per machine or for all machines at once
is the same, since there would be old libvirt that uses removed
CLI option and it won't be able to start new QEMU with that option,
even worse it would affect all machines. So we should agree on new
reasonable deprecation period (if current one isn't sufficient)
that would allow users to adapt to a breaking change.
3. in case of downstream, it ships a compatible bundle and if user installs
a QEMU from newer release without other new bits it would fall under
unsupported category and the first thing support would tell to update
other part along with QEMU. What I'm saying it's downstream distro job
to organize upgrade path/track dependencies and backport/invent compat
layer to earlier releases if necessary.
So it's rather questionable if we should care about arbitrarily old
libvirt with new QEMU in case of new machines (especially upstream).
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
On Wed, Mar 20, 2019 at 11:26:34AM +0100, Igor Mammedov wrote:
> On Tue, 19 Mar 2019 14:51:07 +0000
> Daniel P. Berrangé <berrange@redhat.com> wrote:
>
> > On Tue, Mar 19, 2019 at 02:08:01PM +0100, Igor Mammedov wrote:
> > > On Thu, 7 Mar 2019 10:07:05 +0000
> > > Daniel P. Berrangé <berrange@redhat.com> wrote:
> > >
> > > > On Wed, Mar 06, 2019 at 07:54:17PM +0100, Igor Mammedov wrote:
> > > > > On Wed, 6 Mar 2019 18:16:08 +0000
> > > > > Daniel P. Berrangé <berrange@redhat.com> wrote:
> > > > >
> > > > > > On Wed, Mar 06, 2019 at 06:33:25PM +0100, Igor Mammedov wrote:
> > > > > > > Amend -numa option docs and print warnings if 'mem' option or default RAM
> > > > > > > splitting between nodes is used. It's intended to discourage users from using
> > > > > > > configuration that allows only to fake NUMA on guest side while leading
> > > > > > > to reduced performance of the guest due to inability to properly configure
> > > > > > > VM's RAM on the host.
> > > > > > >
> > > > > > > In NUMA case, it's recommended to always explicitly configure guest RAM
> > > > > > > using -numa node,memdev={backend-id} option.
> > > > > > >
> > > > > > > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > > > > > > ---
> > > > > > > numa.c | 5 +++++
> > > > > > > qemu-options.hx | 12 ++++++++----
> > > > > > > 2 files changed, 13 insertions(+), 4 deletions(-)
> > > > > > >
> > > > > > > diff --git a/numa.c b/numa.c
> > > > > > > index 3875e1e..42838f9 100644
> > > > > > > --- a/numa.c
> > > > > > > +++ b/numa.c
> > > > > > > @@ -121,6 +121,8 @@ static void parse_numa_node(MachineState *ms, NumaNodeOptions *node,
> > > > > > >
> > > > > > > if (node->has_mem) {
> > > > > > > numa_info[nodenr].node_mem = node->mem;
> > > > > > > + warn_report("Parameter -numa node,mem is obsolete,"
> > > > > > > + " use -numa node,memdev instead");
> > > > > >
> > > > > > My comments from v1 still apply. We must not do this as long as
> > > > > > libvirt has no choice but to continue using this feature.
> > > > > It has a choice to use 'memdev' whenever creating a new VM and continue
> > > > > using 'mem' with exiting VMs.
> > > >
> > > > Unfortunately we don't have such a choice. Libvirt has no concept of the
> > > > distinction between an 'existing' and 'new' VM. It just receives an XML
> > > > file from the mgmt application and with transient guests, we have no
> > > > persistent configuration record of the VM. So we've no way of knowing
> > > > whether this VM was previously running on this same host, or another
> > > > host, or is completely new.
> > > In case of transient VM, libvirt might be able to use machine version
> > > as deciding which option to use (memdev is around more than 4 years since 2.1)
> > > (or QEMU could provide introspection into what machine version (not)supports,
> > > like it was discussed before)
> > >
> > > As discussed elsewhere (v1 tread|IRC), there are users (mainly CI) for which
> > > fake NUMA is sufficient and they do not ask for explicit pinning, so libvirt
> > > defaults to legacy -numa node,mem option.
> > > Those users do not care no aware that they should use memdev instead
> > > (I'm not sure if they are able to ask libvirt for non pinned numa memory
> > > which results in memdev being used).
> > > This patch doesn't obsolete anything yet, it serves purpose to inform users
> > > that they are using legacy option and advises replacement option
> > > so that users would know to what they should adapt to.
> > >
> > > Once we deprecate and then remove 'mem' for new machines only (while keeping
> > > 'mem' working on old machine versions). The new nor old libvirt won't be able
> > > to start new machine type with 'mem' option and have to use memdev variant,
> > > so we don't have migration issues with new machines and old ones continue
> > > working with 'mem'.
> >
> > I'm not seeing what has changed which would enable us to deprecate
> > something only for new machines. That's not possible from libvirt's
> > POV as old libvirt will support new machines & thus we have to
> > continue using "mem" for all machines in the scenarios where we
> > currently use it.
> There are several issues here:
> 1. how old libvirt you are talking about?
Any release prior to the one that changes the use of "mem".
IOW, if we changed "mem" in libvirt 5.2.0, then it would break compat
with libvirt 5.1.0 from the previous month's release (and of course
all versions before 5.1.0 by implication).
> 2. old libvirt + new QEMU won't be able to start QEMU with
> new machine with 'mem' option so we don't have live migration,
> it's rather management issue where mgmt should not try to migrate
> to such host (if it manged to end up with not compatible package
> bundle it is not QEMU nor libvirt problem per se).
I don't think this is a mgmt issue. When a new QEMU release comes out
it is valid to use it with an existing release of libvirt. You might
need new libvirt if you want to use new features from QEMU that didn't
exist previously, but existing QEMU features should generally work.
With QEMU's deprecation policy, you're not going to be able to use
arbitrarily old libvirt as at some point you will hit a version of
libvirt that uses the old deprecated approach, instead of the new
preferred approach. Whether this is a problem or not depends on
the features you are using too. eg if we a CLI arg with a new
preferred replacement, & you were never using that CLI arg in the
first place, the incompatibility doesn't affect you.
QEMU deprecation period is two releases, plus however long in the
dev cycle it was deprecated before release. In the best case, libvirt
from 12 months ago will have stopped using the deprecated feature.
In the worst case, where it is very hard to change libvirt, we might
still be using the deprecated feature right up until the end of the
deprecation period. That should be the exception & we try not to get
into that case as it is painful for users to deploy a new QEMU and
find it breaks with their intsalled libvirt.
> 3. in generic dropping features per machine or for all machines at once
> is the same, since there would be old libvirt that uses removed
> CLI option and it won't be able to start new QEMU with that option,
> even worse it would affect all machines. So we should agree on new
> reasonable deprecation period (if current one isn't sufficient)
> that would allow users to adapt to a breaking change.
If a feature is completely dropped by QEMU with no replacement, there's
nothing libvirt can do to preserve existing VMs that use that feature.
Obviously this is painful for users, so QEMU doesn't do that without
compelling reason, such as the feature being unfixably broken.
This isn't the case with "mem" though - it is an existing feature
whose impl is being changed for a different impl. We're just telling
apps to change the way they imple the feature from "mem" to "memdev",
which breaks live migration compat across whichever version of the
app makes the change.
> 3. in case of downstream, it ships a compatible bundle and if user installs
> a QEMU from newer release without other new bits it would fall under
> unsupported category and the first thing support would tell to update
> other part along with QEMU. What I'm saying it's downstream distro job
> to organize upgrade path/track dependencies and backport/invent compat
> layer to earlier releases if necessary.
In terms of preserving back compat, the distro's hands are tied by what
the upstream QEMU does to some extent. If upstream rips out the infra
needed to provide the back compat in the distro, they'll have to revert
all those upstream changes which can be non-trivial. Considering the
distro maintainers are often upstream maintainers too, that's not a net
win. The maintainer has saved themselves some work upstream, but created
themselves a bigger amount of work downstream.
> So it's rather questionable if we should care about arbitrarily old
> libvirt with new QEMU in case of new machines (especially upstream).
As noted above, with the deprecation feature policy new QEMU is not
likely to be compatible with arbitrarily old libvirt, but can usually
be expected to be compatible with upto 12 month old libvirt in the
best case, unless libvirt is really slow at adapting to deprecation
warnings.
So the challenge with tieing it to the new QEMU machine type is that
machine type is potentially used by a libvirt upto perhaps 12 months
old.
Somehow the older libvirt has to know to use the new QEMU feature
"memdev" that wasn't present required for any of the machine types
it knew about when it was first released.
This could be solved if QEMU has some machine type based property
that indicates whether "memdev" is required for a given machine,
but crucially *does not* actually activate that property until
several releases later.
We're too late for 4.0, so lets consider QEMU 4.1 as the
next release of QEMU, which opens for dev in April 2019.
QEMU 4.1 could introduce a machine type property "requires-memdev"
which defaults to "false" for all existing machine types. It
could add a deprecation that says a *future* machine type will
report "requires-memdev=true". IOW, "pc-i440fx-4.1" and
"pc-i440fx-4.2 must still report "requires-memdev=false",
Libvirt 5.4.0 (May 2019) can now add support for "requires-memdev"
property. This would be effectively a no-op at time of this libvirt
release, since no QEMU would be reporting "requires-memdev=true"
for many months to come yet.
Now, after 2 QEMU releases with the deprecation wawrning, when
the QEMU 5.0.0 dev cycle opens in Jan 2020, the new "pc-i440fx-5.0"
machine type can be made to report "requires-memdev=true".
IOW, in April 2020 when QEMU 5.0.0 comes out, "mem" would no
longer be supported for new machine types. Libvirt at this
time would be upto 6.4.0 but that's co-incidental since it
would already be doing the right thing since 5.4.0.
IOW, this QEMU 5.0.0 would work correctly with libvirt versions
in the range 5.4.0 to 6.4.0 (and future).
If a user had libvirt < 5.4.0 (ie older than May 2019) nothing
would stop them using the "pc-i440fx-5.0" machine type, but
libvirt would be liable to use "mem" instead of "memdev" and
if that happened they would be unable to live migrate to a
host newer libvirt which honours "requires-memdev=true"
So in summary the key to being able to tie deprecations to machine
type versions, is for QEMU to add a mechanism to report the desired
new feature usage approach against the machine type, but then ensure
the mechanism continues to report the old approach for 2 more releases.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
On Wed, Mar 20, 2019 at 11:51:51AM +0000, Daniel P. Berrangé wrote: > On Wed, Mar 20, 2019 at 11:26:34AM +0100, Igor Mammedov wrote: [...] > > So it's rather questionable if we should care about arbitrarily old > > libvirt with new QEMU in case of new machines (especially upstream). > > As noted above, with the deprecation feature policy new QEMU is not > likely to be compatible with arbitrarily old libvirt, but can usually > be expected to be compatible with upto 12 month old libvirt in the > best case, unless libvirt is really slow at adapting to deprecation > warnings. > > So the challenge with tieing it to the new QEMU machine type is that > machine type is potentially used by a libvirt upto perhaps 12 months > old. > I'd like to understand one assumption here: why exactly do we need to make (e.g.) libvirt 4.8.0 (Oct 2018) compatible with _all_ machine-types in QEMU 4.1 (~Aug 2019), including pc-4.1.0? People who need backwards compatibility already have a huge list of old machine-types to choose from. After all, pc-4.1.0 is surely a new feature from QEMU that didn't exist previously. > Somehow the older libvirt has to know to use the new QEMU feature > "memdev" that wasn't present required for any of the machine types > it knew about when it was first released. > > > This could be solved if QEMU has some machine type based property > that indicates whether "memdev" is required for a given machine, > but crucially *does not* actually activate that property until > several releases later. > > We're too late for 4.0, so lets consider QEMU 4.1 as the > next release of QEMU, which opens for dev in April 2019. > > QEMU 4.1 could introduce a machine type property "requires-memdev" > which defaults to "false" for all existing machine types. It > could add a deprecation that says a *future* machine type will > report "requires-memdev=true". IOW, "pc-i440fx-4.1" and > "pc-i440fx-4.2 must still report "requires-memdev=false", > > Libvirt 5.4.0 (May 2019) can now add support for "requires-memdev" > property. This would be effectively a no-op at time of this libvirt > release, since no QEMU would be reporting "requires-memdev=true" > for many months to come yet. > > Now, after 2 QEMU releases with the deprecation wawrning, when > the QEMU 5.0.0 dev cycle opens in Jan 2020, the new "pc-i440fx-5.0" > machine type can be made to report "requires-memdev=true". > > IOW, in April 2020 when QEMU 5.0.0 comes out, "mem" would no > longer be supported for new machine types. Libvirt at this > time would be upto 6.4.0 but that's co-incidental since it > would already be doing the right thing since 5.4.0. > > IOW, this QEMU 5.0.0 would work correctly with libvirt versions > in the range 5.4.0 to 6.4.0 (and future). > > If a user had libvirt < 5.4.0 (ie older than May 2019) nothing > would stop them using the "pc-i440fx-5.0" machine type, but > libvirt would be liable to use "mem" instead of "memdev" and > if that happened they would be unable to live migrate to a > host newer libvirt which honours "requires-memdev=true" > > > So in summary the key to being able to tie deprecations to machine > type versions, is for QEMU to add a mechanism to report the desired > new feature usage approach against the machine type, but then ensure > the mechanism continues to report the old approach for 2 more releases. This proposal seems to work, but I'm worried that the code in libvirt for using the new mechanism will be left completely unused and untested by our users for a whole year (until we release a QEMU version that sets requires-memdev=true). If a feature is deprecated, I would expect the management stack to stop using the deprecated feature by default as soon as possible, not 1 year after it was deprecated. -- Eduardo -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
On Wed, Mar 20, 2019 at 10:32:53AM -0300, Eduardo Habkost wrote: > On Wed, Mar 20, 2019 at 11:51:51AM +0000, Daniel P. Berrangé wrote: > > On Wed, Mar 20, 2019 at 11:26:34AM +0100, Igor Mammedov wrote: > [...] > > > So it's rather questionable if we should care about arbitrarily old > > > libvirt with new QEMU in case of new machines (especially upstream). > > > > As noted above, with the deprecation feature policy new QEMU is not > > likely to be compatible with arbitrarily old libvirt, but can usually > > be expected to be compatible with upto 12 month old libvirt in the > > best case, unless libvirt is really slow at adapting to deprecation > > warnings. > > > > So the challenge with tieing it to the new QEMU machine type is that > > machine type is potentially used by a libvirt upto perhaps 12 months > > old. > > > > I'd like to understand one assumption here: why exactly do we > need to make (e.g.) libvirt 4.8.0 (Oct 2018) compatible with > _all_ machine-types in QEMU 4.1 (~Aug 2019), including pc-4.1.0? > People who need backwards compatibility already have a huge list > of old machine-types to choose from. > > After all, pc-4.1.0 is surely a new feature from QEMU that > didn't exist previously. The new machine type is reported as the default expansion of the "pc" alias (or equivalently "q35"), so it will get used automatically for any case where the mgmt app / admin has not explicitly requested a versioned type. Libvirt queries this info from QEMU so that it will always use the latested versioned machine type unless overriden. This was generally seen as a good thing, because new machine types enable new desired tweaks which can improve performance or fix bugs, and so on. > > Somehow the older libvirt has to know to use the new QEMU feature > > "memdev" that wasn't present required for any of the machine types > > it knew about when it was first released. > > > > > > This could be solved if QEMU has some machine type based property > > that indicates whether "memdev" is required for a given machine, > > but crucially *does not* actually activate that property until > > several releases later. > > > > We're too late for 4.0, so lets consider QEMU 4.1 as the > > next release of QEMU, which opens for dev in April 2019. > > > > QEMU 4.1 could introduce a machine type property "requires-memdev" > > which defaults to "false" for all existing machine types. It > > could add a deprecation that says a *future* machine type will > > report "requires-memdev=true". IOW, "pc-i440fx-4.1" and > > "pc-i440fx-4.2 must still report "requires-memdev=false", > > > > Libvirt 5.4.0 (May 2019) can now add support for "requires-memdev" > > property. This would be effectively a no-op at time of this libvirt > > release, since no QEMU would be reporting "requires-memdev=true" > > for many months to come yet. > > > > Now, after 2 QEMU releases with the deprecation wawrning, when > > the QEMU 5.0.0 dev cycle opens in Jan 2020, the new "pc-i440fx-5.0" > > machine type can be made to report "requires-memdev=true". > > > > IOW, in April 2020 when QEMU 5.0.0 comes out, "mem" would no > > longer be supported for new machine types. Libvirt at this > > time would be upto 6.4.0 but that's co-incidental since it > > would already be doing the right thing since 5.4.0. > > > > IOW, this QEMU 5.0.0 would work correctly with libvirt versions > > in the range 5.4.0 to 6.4.0 (and future). > > > > If a user had libvirt < 5.4.0 (ie older than May 2019) nothing > > would stop them using the "pc-i440fx-5.0" machine type, but > > libvirt would be liable to use "mem" instead of "memdev" and > > if that happened they would be unable to live migrate to a > > host newer libvirt which honours "requires-memdev=true" > > > > > > So in summary the key to being able to tie deprecations to machine > > type versions, is for QEMU to add a mechanism to report the desired > > new feature usage approach against the machine type, but then ensure > > the mechanism continues to report the old approach for 2 more releases. > > This proposal seems to work, but I'm worried that the code in > libvirt for using the new mechanism will be left completely > unused and untested by our users for a whole year (until we > release a QEMU version that sets requires-memdev=true). Yes, that is a challenge. The risk is that libvirt thinks it has adapted its code to only use memdev, but might have missed a codepath & this won't be noticed immediately. Maybe there is a way to get this exercised by automated CI where migration compat is a non-issue. > If a feature is deprecated, I would expect the management stack > to stop using the deprecated feature by default as soon as > possible, not 1 year after it was deprecated. True, but the challenge here is that we need to stop using the feature in a way that isn't going to break ability to live migrate VMs spawned by previous versions of libvirt. Most of the time when we stop using a deprecated feature there isn't this complication, so there is zero downside to using the new feature immediately. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
On Wed, 20 Mar 2019 13:46:59 +0000 Daniel P. Berrangé <berrange@redhat.com> wrote: > On Wed, Mar 20, 2019 at 10:32:53AM -0300, Eduardo Habkost wrote: > > On Wed, Mar 20, 2019 at 11:51:51AM +0000, Daniel P. Berrangé wrote: > > > On Wed, Mar 20, 2019 at 11:26:34AM +0100, Igor Mammedov wrote: > > [...] [...] > > If a feature is deprecated, I would expect the management stack > > to stop using the deprecated feature by default as soon as > > possible, not 1 year after it was deprecated. > > True, but the challenge here is that we need to stop using the > feature in a way that isn't going to break ability to live migrate > VMs spawned by previous versions of libvirt. VM should be able to start in the first place, if we disable 'mem' on new machine, old libvirt using 'mem' won't be able to start VM with it, it won't ever come to migration point. (it's a clear signal to user about mis-configured host, at least this old/new issue shouldn't happen in downstream as it ships compatible set of packages). [...] > > Regards, > Daniel -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
On Wed, Mar 20, 2019 at 01:46:59PM +0000, Daniel P. Berrangé wrote: > On Wed, Mar 20, 2019 at 10:32:53AM -0300, Eduardo Habkost wrote: > > On Wed, Mar 20, 2019 at 11:51:51AM +0000, Daniel P. Berrangé wrote: > > > On Wed, Mar 20, 2019 at 11:26:34AM +0100, Igor Mammedov wrote: > > [...] > > > > So it's rather questionable if we should care about arbitrarily old > > > > libvirt with new QEMU in case of new machines (especially upstream). > > > > > > As noted above, with the deprecation feature policy new QEMU is not > > > likely to be compatible with arbitrarily old libvirt, but can usually > > > be expected to be compatible with upto 12 month old libvirt in the > > > best case, unless libvirt is really slow at adapting to deprecation > > > warnings. > > > > > > So the challenge with tieing it to the new QEMU machine type is that > > > machine type is potentially used by a libvirt upto perhaps 12 months > > > old. > > > > > > > I'd like to understand one assumption here: why exactly do we > > need to make (e.g.) libvirt 4.8.0 (Oct 2018) compatible with > > _all_ machine-types in QEMU 4.1 (~Aug 2019), including pc-4.1.0? > > People who need backwards compatibility already have a huge list > > of old machine-types to choose from. > > > > After all, pc-4.1.0 is surely a new feature from QEMU that > > didn't exist previously. > > The new machine type is reported as the default expansion of the > "pc" alias (or equivalently "q35"), so it will get used automatically > for any case where the mgmt app / admin has not explicitly requested > a versioned type. Libvirt queries this info from QEMU so that it > will always use the latested versioned machine type unless overriden. > This was generally seen as a good thing, because new machine types > enable new desired tweaks which can improve performance or fix > bugs, and so on. Understood. Maybe we should rethink the current approach. The mechanism for choosing the default machine-type should leave room for cases where the newest machine-type isn't compatible with current libvirt. This way we won't need to wait for one year before using a new feature by default. Let me understand the requirements & expectations here: do we really need to make libvirt 5.4.0 (May 2019) to automatically translate "pc" to pc-4.1.0 if running QEMU 4.1.0 (Aug 2019)? What would happen if libvirt 5.4.0 translated pc to pc-4.0.0 (even if running QEMU 4.1.0), and libvirt 5.8.0 (Sep 2019) translated pc to pc-4.1.0? > > > Somehow the older libvirt has to know to use the new QEMU feature > > > "memdev" that wasn't present required for any of the machine types > > > it knew about when it was first released. > > > > > > > > > This could be solved if QEMU has some machine type based property > > > that indicates whether "memdev" is required for a given machine, > > > but crucially *does not* actually activate that property until > > > several releases later. > > > > > > We're too late for 4.0, so lets consider QEMU 4.1 as the > > > next release of QEMU, which opens for dev in April 2019. > > > > > > QEMU 4.1 could introduce a machine type property "requires-memdev" > > > which defaults to "false" for all existing machine types. It > > > could add a deprecation that says a *future* machine type will > > > report "requires-memdev=true". IOW, "pc-i440fx-4.1" and > > > "pc-i440fx-4.2 must still report "requires-memdev=false", > > > > > > Libvirt 5.4.0 (May 2019) can now add support for "requires-memdev" > > > property. This would be effectively a no-op at time of this libvirt > > > release, since no QEMU would be reporting "requires-memdev=true" > > > for many months to come yet. > > > > > > Now, after 2 QEMU releases with the deprecation wawrning, when > > > the QEMU 5.0.0 dev cycle opens in Jan 2020, the new "pc-i440fx-5.0" > > > machine type can be made to report "requires-memdev=true". > > > > > > IOW, in April 2020 when QEMU 5.0.0 comes out, "mem" would no > > > longer be supported for new machine types. Libvirt at this > > > time would be upto 6.4.0 but that's co-incidental since it > > > would already be doing the right thing since 5.4.0. > > > > > > IOW, this QEMU 5.0.0 would work correctly with libvirt versions > > > in the range 5.4.0 to 6.4.0 (and future). > > > > > > If a user had libvirt < 5.4.0 (ie older than May 2019) nothing > > > would stop them using the "pc-i440fx-5.0" machine type, but > > > libvirt would be liable to use "mem" instead of "memdev" and > > > if that happened they would be unable to live migrate to a > > > host newer libvirt which honours "requires-memdev=true" > > > > > > > > > So in summary the key to being able to tie deprecations to machine > > > type versions, is for QEMU to add a mechanism to report the desired > > > new feature usage approach against the machine type, but then ensure > > > the mechanism continues to report the old approach for 2 more releases. > > > > This proposal seems to work, but I'm worried that the code in > > libvirt for using the new mechanism will be left completely > > unused and untested by our users for a whole year (until we > > release a QEMU version that sets requires-memdev=true). > > Yes, that is a challenge. The risk is that libvirt thinks it > has adapted its code to only use memdev, but might have missed > a codepath & this won't be noticed immediately. Maybe there is > a way to get this exercised by automated CI where migration > compat is a non-issue. > > > If a feature is deprecated, I would expect the management stack > > to stop using the deprecated feature by default as soon as > > possible, not 1 year after it was deprecated. > > True, but the challenge here is that we need to stop using the > feature in a way that isn't going to break ability to live migrate > VMs spawned by previous versions of libvirt. Most of the time when > we stop using a deprecated feature there isn't this complication, > so there is zero downside to using the new feature immediately. Keeping the ability to live migrate to older libvirt is easy to implement on older machine-types. The problem is more complex only because we want libvirt to use machine-types that didn't exist yet when libvirt was released. I still don't know if this is a requirement we really want to keep. -- Eduardo -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
On Wed, Mar 20, 2019 at 11:46:20AM -0300, Eduardo Habkost wrote: > On Wed, Mar 20, 2019 at 01:46:59PM +0000, Daniel P. Berrangé wrote: > > On Wed, Mar 20, 2019 at 10:32:53AM -0300, Eduardo Habkost wrote: > > > On Wed, Mar 20, 2019 at 11:51:51AM +0000, Daniel P. Berrangé wrote: > > > > On Wed, Mar 20, 2019 at 11:26:34AM +0100, Igor Mammedov wrote: > > > [...] > > > > > So it's rather questionable if we should care about arbitrarily old > > > > > libvirt with new QEMU in case of new machines (especially upstream). > > > > > > > > As noted above, with the deprecation feature policy new QEMU is not > > > > likely to be compatible with arbitrarily old libvirt, but can usually > > > > be expected to be compatible with upto 12 month old libvirt in the > > > > best case, unless libvirt is really slow at adapting to deprecation > > > > warnings. > > > > > > > > So the challenge with tieing it to the new QEMU machine type is that > > > > machine type is potentially used by a libvirt upto perhaps 12 months > > > > old. > > > > > > > > > > I'd like to understand one assumption here: why exactly do we > > > need to make (e.g.) libvirt 4.8.0 (Oct 2018) compatible with > > > _all_ machine-types in QEMU 4.1 (~Aug 2019), including pc-4.1.0? > > > People who need backwards compatibility already have a huge list > > > of old machine-types to choose from. > > > > > > After all, pc-4.1.0 is surely a new feature from QEMU that > > > didn't exist previously. > > > > The new machine type is reported as the default expansion of the > > "pc" alias (or equivalently "q35"), so it will get used automatically > > for any case where the mgmt app / admin has not explicitly requested > > a versioned type. Libvirt queries this info from QEMU so that it > > will always use the latested versioned machine type unless overriden. > > This was generally seen as a good thing, because new machine types > > enable new desired tweaks which can improve performance or fix > > bugs, and so on. > > Understood. Maybe we should rethink the current approach. The > mechanism for choosing the default machine-type should leave room > for cases where the newest machine-type isn't compatible with > current libvirt. This way we won't need to wait for one year > before using a new feature by default. > > Let me understand the requirements & expectations here: do we > really need to make libvirt 5.4.0 (May 2019) to automatically > translate "pc" to pc-4.1.0 if running QEMU 4.1.0 (Aug 2019)? > What would happen if libvirt 5.4.0 translated pc to pc-4.0.0 > (even if running QEMU 4.1.0), and libvirt 5.8.0 (Sep 2019) > translated pc to pc-4.1.0? Nothing would break if you did that. Libvirt explicitly always expands "pc" to a versioned machine precisely so that it is not vulnerable to changes in the expansion in future. I don't know how we would actually achieve this in practice though. The naive approach would be for libvirt to simply not honour the QEMU default machine type at all. Libvirt could simply hardcode the expansion of "pc" to "pc-i440fx-X.Y" based on the newest QEMU that exists at time of libvirt release. This would work except for the fact that the long long distros all fork QEMU and throwaway the standard versioned machine types and create their own with different names. So if libvirt harcoded a desired machine type itself, upstream libvirt would cease to work on those distros which is something we really want to avoid. It created significant pain for us when libvirt upstream was unable to work with the QEMU in RHEL-6 because it needed various non-upstream hacks in order to deal with the partiallly implemented QMP monitor. I really wish distros would not fork the machine types in QEMU but that boat has sailed long ago, and it would require much better upstream testing to avoid it anyway :-( > > > > Somehow the older libvirt has to know to use the new QEMU feature > > > > "memdev" that wasn't present required for any of the machine types > > > > it knew about when it was first released. > > > > > > > > > > > > This could be solved if QEMU has some machine type based property > > > > that indicates whether "memdev" is required for a given machine, > > > > but crucially *does not* actually activate that property until > > > > several releases later. > > > > > > > > We're too late for 4.0, so lets consider QEMU 4.1 as the > > > > next release of QEMU, which opens for dev in April 2019. > > > > > > > > QEMU 4.1 could introduce a machine type property "requires-memdev" > > > > which defaults to "false" for all existing machine types. It > > > > could add a deprecation that says a *future* machine type will > > > > report "requires-memdev=true". IOW, "pc-i440fx-4.1" and > > > > "pc-i440fx-4.2 must still report "requires-memdev=false", > > > > > > > > Libvirt 5.4.0 (May 2019) can now add support for "requires-memdev" > > > > property. This would be effectively a no-op at time of this libvirt > > > > release, since no QEMU would be reporting "requires-memdev=true" > > > > for many months to come yet. > > > > > > > > Now, after 2 QEMU releases with the deprecation wawrning, when > > > > the QEMU 5.0.0 dev cycle opens in Jan 2020, the new "pc-i440fx-5.0" > > > > machine type can be made to report "requires-memdev=true". > > > > > > > > IOW, in April 2020 when QEMU 5.0.0 comes out, "mem" would no > > > > longer be supported for new machine types. Libvirt at this > > > > time would be upto 6.4.0 but that's co-incidental since it > > > > would already be doing the right thing since 5.4.0. > > > > > > > > IOW, this QEMU 5.0.0 would work correctly with libvirt versions > > > > in the range 5.4.0 to 6.4.0 (and future). > > > > > > > > If a user had libvirt < 5.4.0 (ie older than May 2019) nothing > > > > would stop them using the "pc-i440fx-5.0" machine type, but > > > > libvirt would be liable to use "mem" instead of "memdev" and > > > > if that happened they would be unable to live migrate to a > > > > host newer libvirt which honours "requires-memdev=true" > > > > > > > > > > > > So in summary the key to being able to tie deprecations to machine > > > > type versions, is for QEMU to add a mechanism to report the desired > > > > new feature usage approach against the machine type, but then ensure > > > > the mechanism continues to report the old approach for 2 more releases. > > > > > > This proposal seems to work, but I'm worried that the code in > > > libvirt for using the new mechanism will be left completely > > > unused and untested by our users for a whole year (until we > > > release a QEMU version that sets requires-memdev=true). > > > > Yes, that is a challenge. The risk is that libvirt thinks it > > has adapted its code to only use memdev, but might have missed > > a codepath & this won't be noticed immediately. Maybe there is > > a way to get this exercised by automated CI where migration > > compat is a non-issue. > > > > > If a feature is deprecated, I would expect the management stack > > > to stop using the deprecated feature by default as soon as > > > possible, not 1 year after it was deprecated. > > > > True, but the challenge here is that we need to stop using the > > feature in a way that isn't going to break ability to live migrate > > VMs spawned by previous versions of libvirt. Most of the time when > > we stop using a deprecated feature there isn't this complication, > > so there is zero downside to using the new feature immediately. > > Keeping the ability to live migrate to older libvirt is easy to > implement on older machine-types. The problem is more complex > only because we want libvirt to use machine-types that didn't > exist yet when libvirt was released. I still don't know if this > is a requirement we really want to keep. As above, I don't think it is important to keep it, but I don't have any good ideas about how to get different libvirt versions to use a different machine type version. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
On Wed, 20 Mar 2019 11:51:51 +0000
Daniel P. Berrangé <berrange@redhat.com> wrote:
> On Wed, Mar 20, 2019 at 11:26:34AM +0100, Igor Mammedov wrote:
> > On Tue, 19 Mar 2019 14:51:07 +0000
> > Daniel P. Berrangé <berrange@redhat.com> wrote:
> >
> > > On Tue, Mar 19, 2019 at 02:08:01PM +0100, Igor Mammedov wrote:
> > > > On Thu, 7 Mar 2019 10:07:05 +0000
> > > > Daniel P. Berrangé <berrange@redhat.com> wrote:
> > > >
> > > > > On Wed, Mar 06, 2019 at 07:54:17PM +0100, Igor Mammedov wrote:
> > > > > > On Wed, 6 Mar 2019 18:16:08 +0000
> > > > > > Daniel P. Berrangé <berrange@redhat.com> wrote:
> > > > > >
> > > > > > > On Wed, Mar 06, 2019 at 06:33:25PM +0100, Igor Mammedov wrote:
> > > > > > > > Amend -numa option docs and print warnings if 'mem' option or default RAM
> > > > > > > > splitting between nodes is used. It's intended to discourage users from using
> > > > > > > > configuration that allows only to fake NUMA on guest side while leading
> > > > > > > > to reduced performance of the guest due to inability to properly configure
> > > > > > > > VM's RAM on the host.
> > > > > > > >
> > > > > > > > In NUMA case, it's recommended to always explicitly configure guest RAM
> > > > > > > > using -numa node,memdev={backend-id} option.
> > > > > > > >
> > > > > > > > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > > > > > > > ---
> > > > > > > > numa.c | 5 +++++
> > > > > > > > qemu-options.hx | 12 ++++++++----
> > > > > > > > 2 files changed, 13 insertions(+), 4 deletions(-)
> > > > > > > >
> > > > > > > > diff --git a/numa.c b/numa.c
> > > > > > > > index 3875e1e..42838f9 100644
> > > > > > > > --- a/numa.c
> > > > > > > > +++ b/numa.c
> > > > > > > > @@ -121,6 +121,8 @@ static void parse_numa_node(MachineState *ms, NumaNodeOptions *node,
> > > > > > > >
> > > > > > > > if (node->has_mem) {
> > > > > > > > numa_info[nodenr].node_mem = node->mem;
> > > > > > > > + warn_report("Parameter -numa node,mem is obsolete,"
> > > > > > > > + " use -numa node,memdev instead");
> > > > > > >
> > > > > > > My comments from v1 still apply. We must not do this as long as
> > > > > > > libvirt has no choice but to continue using this feature.
> > > > > > It has a choice to use 'memdev' whenever creating a new VM and continue
> > > > > > using 'mem' with exiting VMs.
> > > > >
> > > > > Unfortunately we don't have such a choice. Libvirt has no concept of the
> > > > > distinction between an 'existing' and 'new' VM. It just receives an XML
> > > > > file from the mgmt application and with transient guests, we have no
> > > > > persistent configuration record of the VM. So we've no way of knowing
> > > > > whether this VM was previously running on this same host, or another
> > > > > host, or is completely new.
> > > > In case of transient VM, libvirt might be able to use machine version
> > > > as deciding which option to use (memdev is around more than 4 years since 2.1)
> > > > (or QEMU could provide introspection into what machine version (not)supports,
> > > > like it was discussed before)
> > > >
> > > > As discussed elsewhere (v1 tread|IRC), there are users (mainly CI) for which
> > > > fake NUMA is sufficient and they do not ask for explicit pinning, so libvirt
> > > > defaults to legacy -numa node,mem option.
> > > > Those users do not care no aware that they should use memdev instead
> > > > (I'm not sure if they are able to ask libvirt for non pinned numa memory
> > > > which results in memdev being used).
> > > > This patch doesn't obsolete anything yet, it serves purpose to inform users
> > > > that they are using legacy option and advises replacement option
> > > > so that users would know to what they should adapt to.
> > > >
> > > > Once we deprecate and then remove 'mem' for new machines only (while keeping
> > > > 'mem' working on old machine versions). The new nor old libvirt won't be able
> > > > to start new machine type with 'mem' option and have to use memdev variant,
> > > > so we don't have migration issues with new machines and old ones continue
> > > > working with 'mem'.
> > >
> > > I'm not seeing what has changed which would enable us to deprecate
> > > something only for new machines. That's not possible from libvirt's
> > > POV as old libvirt will support new machines & thus we have to
> > > continue using "mem" for all machines in the scenarios where we
> > > currently use it.
> > There are several issues here:
> > 1. how old libvirt you are talking about?
>
> Any release prior to the one that changes the use of "mem".
>
> IOW, if we changed "mem" in libvirt 5.2.0, then it would break compat
> with libvirt 5.1.0 from the previous month's release (and of course
> all versions before 5.1.0 by implication).
>
> > 2. old libvirt + new QEMU won't be able to start QEMU with
> > new machine with 'mem' option so we don't have live migration,
> > it's rather management issue where mgmt should not try to migrate
> > to such host (if it manged to end up with not compatible package
> > bundle it is not QEMU nor libvirt problem per se).
>
> I don't think this is a mgmt issue. When a new QEMU release comes out
> it is valid to use it with an existing release of libvirt. You might
> need new libvirt if you want to use new features from QEMU that didn't
> exist previously, but existing QEMU features should generally work.
>
> With QEMU's deprecation policy, you're not going to be able to use
> arbitrarily old libvirt as at some point you will hit a version of
> libvirt that uses the old deprecated approach, instead of the new
> preferred approach. Whether this is a problem or not depends on
> the features you are using too. eg if we a CLI arg with a new
> preferred replacement, & you were never using that CLI arg in the
> first place, the incompatibility doesn't affect you.
>
> QEMU deprecation period is two releases, plus however long in the
> dev cycle it was deprecated before release. In the best case, libvirt
> from 12 months ago will have stopped using the deprecated feature.
> In the worst case, where it is very hard to change libvirt, we might
> still be using the deprecated feature right up until the end of the
> deprecation period. That should be the exception & we try not to get
> into that case as it is painful for users to deploy a new QEMU and
> find it breaks with their intsalled libvirt.
>
> > 3. in generic dropping features per machine or for all machines at once
> > is the same, since there would be old libvirt that uses removed
> > CLI option and it won't be able to start new QEMU with that option,
> > even worse it would affect all machines. So we should agree on new
> > reasonable deprecation period (if current one isn't sufficient)
> > that would allow users to adapt to a breaking change.
>
> If a feature is completely dropped by QEMU with no replacement, there's
> nothing libvirt can do to preserve existing VMs that use that feature.
> Obviously this is painful for users, so QEMU doesn't do that without
> compelling reason, such as the feature being unfixably broken.
>
> This isn't the case with "mem" though - it is an existing feature
> whose impl is being changed for a different impl. We're just telling
> apps to change the way they imple the feature from "mem" to "memdev",
> which breaks live migration compat across whichever version of the
> app makes the change.
>
> > 3. in case of downstream, it ships a compatible bundle and if user installs
> > a QEMU from newer release without other new bits it would fall under
> > unsupported category and the first thing support would tell to update
> > other part along with QEMU. What I'm saying it's downstream distro job
> > to organize upgrade path/track dependencies and backport/invent compat
> > layer to earlier releases if necessary.
>
> In terms of preserving back compat, the distro's hands are tied by what
> the upstream QEMU does to some extent. If upstream rips out the infra
> needed to provide the back compat in the distro, they'll have to revert
> all those upstream changes which can be non-trivial. Considering the
> distro maintainers are often upstream maintainers too, that's not a net
> win. The maintainer has saved themselves some work upstream, but created
> themselves a bigger amount of work downstream.
I don't agree with some above said points but I will put this discussion off
for later and jump strait down more technical part below.
> > So it's rather questionable if we should care about arbitrarily old
> > libvirt with new QEMU in case of new machines (especially upstream).
>
> As noted above, with the deprecation feature policy new QEMU is not
> likely to be compatible with arbitrarily old libvirt, but can usually
> be expected to be compatible with upto 12 month old libvirt in the
> best case, unless libvirt is really slow at adapting to deprecation
> warnings.
>
> So the challenge with tieing it to the new QEMU machine type is that
> machine type is potentially used by a libvirt upto perhaps 12 months
> old.
Seems a bit much but if there is consensus I'd go with it,
at least it allows us to move forward in a year (when 'mem'
is banned on new machines)
> Somehow the older libvirt has to know to use the new QEMU feature
> "memdev" that wasn't present required for any of the machine types
> it knew about when it was first released.
>
>
> This could be solved if QEMU has some machine type based property
> that indicates whether "memdev" is required for a given machine,
> but crucially *does not* actually activate that property until
> several releases later.
>
> We're too late for 4.0, so lets consider QEMU 4.1 as the
> next release of QEMU, which opens for dev in April 2019.
>
> QEMU 4.1 could introduce a machine type property "requires-memdev"
> which defaults to "false" for all existing machine types. It
> could add a deprecation that says a *future* machine type will
> report "requires-memdev=true". IOW, "pc-i440fx-4.1" and
> "pc-i440fx-4.2 must still report "requires-memdev=false",
>
> Libvirt 5.4.0 (May 2019) can now add support for "requires-memdev"
> property. This would be effectively a no-op at time of this libvirt
> release, since no QEMU would be reporting "requires-memdev=true"
> for many months to come yet.
>
> Now, after 2 QEMU releases with the deprecation wawrning, when
> the QEMU 5.0.0 dev cycle opens in Jan 2020, the new "pc-i440fx-5.0"
> machine type can be made to report "requires-memdev=true".
>
> IOW, in April 2020 when QEMU 5.0.0 comes out, "mem" would
> no longer be supported for new machine types. Libvirt at this
^^^^^^^^^^^^^^^^^^^^^^^
> time would be upto 6.4.0 but that's co-incidental since it
> would already be doing the right thing since 5.4.0.
>
> IOW, this QEMU 5.0.0 would work correctly with libvirt versions
> in the range 5.4.0 to 6.4.0 (and future).
> If a user had libvirt < 5.4.0 (ie older than May 2019) nothing
> would stop them using the "pc-i440fx-5.0" machine type, but
> libvirt would be liable to use "mem" instead of "memdev" and
> if that happened they would be unable to live migrate to a
> host newer libvirt which honours "requires-memdev=true"
I failed to parse this section in connection '^' underlined part,
I'm reading 'no longer be supported' as it's not possible to start
QEMU -M machine_foo.requires-memdev=true with 'mem' option.
Is it what you've meant?
> So in summary the key to being able to tie deprecations to machine
> type versions, is for QEMU to add a mechanism to report the desired
> new feature usage approach against the machine type, but then ensure
> the mechanism continues to report the old approach for 2 more releases.
so that makes QEMU deprecation period effectively 3 releases (assuming 4 months cadence).
>
> Regards,
> Daniel
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
On Wed, Mar 20, 2019 at 04:20:19PM +0100, Igor Mammedov wrote: > > This could be solved if QEMU has some machine type based property > > that indicates whether "memdev" is required for a given machine, > > but crucially *does not* actually activate that property until > > several releases later. > > > > We're too late for 4.0, so lets consider QEMU 4.1 as the > > next release of QEMU, which opens for dev in April 2019. > > > > QEMU 4.1 could introduce a machine type property "requires-memdev" > > which defaults to "false" for all existing machine types. It > > could add a deprecation that says a *future* machine type will > > report "requires-memdev=true". IOW, "pc-i440fx-4.1" and > > "pc-i440fx-4.2 must still report "requires-memdev=false", > > > > Libvirt 5.4.0 (May 2019) can now add support for "requires-memdev" > > property. This would be effectively a no-op at time of this libvirt > > release, since no QEMU would be reporting "requires-memdev=true" > > for many months to come yet. > > > > Now, after 2 QEMU releases with the deprecation wawrning, when > > the QEMU 5.0.0 dev cycle opens in Jan 2020, the new "pc-i440fx-5.0" > > machine type can be made to report "requires-memdev=true". > > > > IOW, in April 2020 when QEMU 5.0.0 comes out, "mem" would > > no longer be supported for new machine types. Libvirt at this > ^^^^^^^^^^^^^^^^^^^^^^^ > > > time would be upto 6.4.0 but that's co-incidental since it > > would already be doing the right thing since 5.4.0. > > > > IOW, this QEMU 5.0.0 would work correctly with libvirt versions > > in the range 5.4.0 to 6.4.0 (and future). > > > If a user had libvirt < 5.4.0 (ie older than May 2019) nothing > > would stop them using the "pc-i440fx-5.0" machine type, but > > libvirt would be liable to use "mem" instead of "memdev" and > > > if that happened they would be unable to live migrate to a > > host newer libvirt which honours "requires-memdev=true" > I failed to parse this section in connection '^' underlined part, > I'm reading 'no longer be supported' as it's not possible to start > QEMU -M machine_foo.requires-memdev=true with 'mem' option. > Is it what you've meant? I wasn't actually meaning QEMU to forbid it when i wrote this, but on reflection, it would make sense to forbid it, as that would avoid the user getting into a messy situation with versions of libvirt that predate knowledge of the requires-memdev property. > > So in summary the key to being able to tie deprecations to machine > > type versions, is for QEMU to add a mechanism to report the desired > > new feature usage approach against the machine type, but then ensure > > the mechanism continues to report the old approach for 2 more releases. > > so that makes QEMU deprecation period effectively 3 releases (assuming > 4 months cadence). There's a distinction betweenm releases and development cycles here. The deprecation policy is defined as 2 releases, which means between 2 and 3 development cycles depending on when in the dev cycle the deprecation is added (start vs the end of the dev cycle) Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
On Wed, 20 Mar 2019 15:24:42 +0000
Daniel P. Berrangé <berrange@redhat.com> wrote:
> On Wed, Mar 20, 2019 at 04:20:19PM +0100, Igor Mammedov wrote:
> > > This could be solved if QEMU has some machine type based property
> > > that indicates whether "memdev" is required for a given machine,
> > > but crucially *does not* actually activate that property until
> > > several releases later.
> > >
> > > We're too late for 4.0, so lets consider QEMU 4.1 as the
> > > next release of QEMU, which opens for dev in April 2019.
> > >
> > > QEMU 4.1 could introduce a machine type property "requires-memdev"
> > > which defaults to "false" for all existing machine types. It
> > > could add a deprecation that says a *future* machine type will
> > > report "requires-memdev=true". IOW, "pc-i440fx-4.1" and
> > > "pc-i440fx-4.2 must still report "requires-memdev=false",
> > >
> > > Libvirt 5.4.0 (May 2019) can now add support for "requires-memdev"
> > > property. This would be effectively a no-op at time of this libvirt
> > > release, since no QEMU would be reporting "requires-memdev=true"
> > > for many months to come yet.
> > >
> > > Now, after 2 QEMU releases with the deprecation wawrning, when
> > > the QEMU 5.0.0 dev cycle opens in Jan 2020, the new "pc-i440fx-5.0"
> > > machine type can be made to report "requires-memdev=true".
> > >
> > > IOW, in April 2020 when QEMU 5.0.0 comes out, "mem" would
> > > no longer be supported for new machine types. Libvirt at this
> > ^^^^^^^^^^^^^^^^^^^^^^^
> >
> > > time would be upto 6.4.0 but that's co-incidental since it
> > > would already be doing the right thing since 5.4.0.
> > >
> > > IOW, this QEMU 5.0.0 would work correctly with libvirt versions
> > > in the range 5.4.0 to 6.4.0 (and future).
> >
> > > If a user had libvirt < 5.4.0 (ie older than May 2019) nothing
> > > would stop them using the "pc-i440fx-5.0" machine type, but
> > > libvirt would be liable to use "mem" instead of "memdev" and
> >
> > > if that happened they would be unable to live migrate to a
> > > host newer libvirt which honours "requires-memdev=true"
> > I failed to parse this section in connection '^' underlined part,
> > I'm reading 'no longer be supported' as it's not possible to start
> > QEMU -M machine_foo.requires-memdev=true with 'mem' option.
> > Is it what you've meant?
>
> I wasn't actually meaning QEMU to forbid it when i wrote this,
> but on reflection, it would make sense to forbid it, as that
> would avoid the user getting into a messy situation with
> versions of libvirt that predate knowledge of the requires-memdev
> property.
Forbidding is my goal as it (at least for new machine types)
- removes possibility of mis-configuration
- allows in new machine to switch to frontend-backend memory model
in clean way consolidating/unifying memory management
(i.e. not need to map 'mem' to memdev, which from recent
migration experiment appears to be impossible to do reliably)
- remove someday 'mem' and all related code from QEMU once
the last old machine where it was possible to use if deprecated
(well, it's rather far fetched goal for that we need to come up
with schedule/policyhow/when we would deprecate old machines).
> > > So in summary the key to being able to tie deprecations to machine
> > > type versions, is for QEMU to add a mechanism to report the desired
> > > new feature usage approach against the machine type, but then ensure
> > > the mechanism continues to report the old approach for 2 more releases.
> >
> > so that makes QEMU deprecation period effectively 3 releases (assuming
> > 4 months cadence).
>
> There's a distinction betweenm releases and development cycles here.
> The deprecation policy is defined as 2 releases, which means between
> 2 and 3 development cycles depending on when in the dev cycle the
> deprecation is added (start vs the end of the dev cycle)
>
> Regards,
> Daniel
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
© 2016 - 2024 Red Hat, Inc.