> -----Original Message-----
> From: Juan Quintela <quintela@redhat.com>
> Sent: Monday, October 23, 2023 6:48 PM
> To: Daniel P.Berrangé <berrange@redhat.com>
> Cc: Liu, Yuan1 <yuan1.liu@intel.com>; Peter Xu <peterx@redhat.com>;
> farosas@suse.de; leobras@redhat.com; qemu-devel@nongnu.org; Zou,
> Nanhai <nanhai.zou@intel.com>
> Subject: Re: [PATCH 0/5] Live Migration Acceleration with IAA Compression
> 
> Daniel P. Berrangé <berrange@redhat.com> wrote:
> > On Mon, Oct 23, 2023 at 08:33:44AM +0000, Liu, Yuan1 wrote:
> >> > -----Original Message-----
> >> > From: Daniel P. Berrangé <berrange@redhat.com>
> >> > Sent: Thursday, October 19, 2023 11:32 PM
> >> > To: Peter Xu <peterx@redhat.com>
> >> > Cc: Juan Quintela <quintela@redhat.com>; Liu, Yuan1
> >> > <yuan1.liu@intel.com>; farosas@suse.de; leobras@redhat.com; qemu-
> >> > devel@nongnu.org; Zou, Nanhai <nanhai.zou@intel.com>
> >> > Subject: Re: [PATCH 0/5] Live Migration Acceleration with IAA
> >> > Compression
> >> >
> >> > On Thu, Oct 19, 2023 at 11:23:31AM -0400, Peter Xu wrote:
> >> > > On Thu, Oct 19, 2023 at 03:52:14PM +0100, Daniel P. Berrangé wrote:
> >> > > > On Thu, Oct 19, 2023 at 01:40:23PM +0200, Juan Quintela wrote:
> >> > > > > Yuan Liu <yuan1.liu@intel.com> wrote:
> >> > > > > > Hi,
> >> > > > > >
> >> > > > > > I am writing to submit a code change aimed at enhancing
> >> > > > > > live migration acceleration by leveraging the compression
> >> > > > > > capability of the Intel In-Memory Analytics Accelerator (IAA).
> >> > > > > >
> >> > > > > > Enabling compression functionality during the live
> >> > > > > > migration process can enhance performance, thereby reducing
> >> > > > > > downtime and network bandwidth requirements. However, this
> >> > > > > > improvement comes at the cost of additional CPU resources,
> >> > > > > > posing a challenge for cloud service providers in terms of
> >> > > > > > resource allocation. To address this challenge, I have
> >> > > > > > focused on offloading the compression
> >> > overhead to the IAA hardware, resulting in performance gains.
> >> > > > > >
> >> > > > > > The implementation of the IAA (de)compression code is based
> >> > > > > > on Intel Query Processing Library (QPL), an open-source
> >> > > > > > software project designed for IAA high-level software
> programming.
> >> > > > >
> >> > > > > After reviewing the patches:
> >> > > > >
> >> > > > > - why are you doing this on top of old compression code, that is
> >> > > > >   obsolete, deprecated and buggy
> >> Some users have not enabled the multifd feature yet, but they will
> >> decide whether to enable the compression feature based on the load
> >> situation. So I'm wondering if, without multifd, the compression
> >> functionality will no longer be available?
> >>
> >> > > > > - why are you not doing it on top of multifd.
> >> I plan to submit the support for multifd independently because the
> >> multifd compression and legacy compression code are separate.
> >
> > So the core question her (for migration maintainers) is whether
> > contributors should be spending any time at all on non-multifd code,
> > or if new features should be exclusively for multifd ?
> 
> Only for multifd.
> 
> Comparison right now:
> - compression (can be done better in multifd)
> - plain precopy (we can satturate faster networks with multifd)
> - xbzrle: right now only non-multifd (plan to add as another multifd
>           compression method)
> - exec: This is a hard one.  Fabiano is about to submit a file based
>         multifd method.  Advantages over exec:
>           * much less space used (it writes each page at the right
>             position, no overhead and never the same page on the two
>             streams)
>           * We can give proper errors, exec is very bad when the exec'd
>             process gives an error.
>         Disadvantages:
>           * libvirt (or any management app) needs to wait for
>             compression to end, and launch the exec command by hand.
>             I wanted to discuss this with libvirt, if it would be
>             possible to remove the use of exec compression.
> - rdma: This is a hard one
>         Current implementation is a mess
>         It is almost un-maintained
>         There are two-three years old patches to move it on top of
>         multifd
> - postcopy: Not implemented.  This is the real reason that we can't
>         deprecate precopy and put multifd as default.
> - snapshots:  They are to coupled with qcow2.  It should be possible to
>         do something more sensible with multifd + file, but we need to walk that
>         path when multifd + file hit the tree.
> 
> > I doesn't make a lot of sense over the long term to have people
> > spending time implementing the same features twice. IOW, should we be
> > directly contributors explicitly towards multifd only, and even
> > consider deprecating non-multifd code at some time ?
> 
> Intel submited something similarish to this on top of QAT several months back.
> I already advised them not to use any time on top of old compression code and
> just do things on top of multifd.
> 
> Once that we are here, what are the differ]ences of QPL and QAT?
> Previous submission used qatzip-devel.
Thank you very much for the QAT suggestions. QPL is utilized for IAA, and qatzip-devel is utilized for QAT, both of them are compatible with zlib. 
Qatzip-devel exclusively supports synchronous compression and does not support batch operations. Consequently, for single-page compression, the performance improvement may not be significant. And QPL supports both synchronous and asynchronous compressions.