[PATCH v2 0/2] ui/vnc: Do not copy z_stream

Akihiko Odaki posted 2 patches 5 months, 2 weeks ago
Failed in applying to current master (apply log)
There is a newer version of this series
ui/vnc.h              |  14 ++-
ui/vnc-enc-tight.c    | 341 ++++++++++++++++++++++++++------------------------
ui/vnc-enc-zlib.c     |  34 ++---
ui/vnc-enc-zrle.c     |  69 +++++-----
ui/vnc-jobs.c         |   9 +-
ui/vnc.c              |  56 ++++-----
ui/vnc-enc-zrle.c.inc |   2 +-
7 files changed, 258 insertions(+), 267 deletions(-)
[PATCH v2 0/2] ui/vnc: Do not copy z_stream
Posted by Akihiko Odaki 5 months, 2 weeks ago
vnc_worker_thread_loop() copies z_stream stored in its local VncState to
the persistent VncState, and the copied one is freed with deflateEnd()
later. However, deflateEnd() refuses to operate with a copied z_stream
and returns Z_STREAM_ERROR, leaking the allocated memory.

Avoid copying the zlib state to fix the memory leak.

Signed-off-by: Akihiko Odaki <odaki@rsg.ci.i.u-tokyo.ac.jp>
---
Changes in v2:
- Rebased.
- Link to v1: https://lore.kernel.org/qemu-devel/20250417-zlib-v1-0-34fad73b843b@daynix.com

---
Akihiko Odaki (2):
      ui/vnc: Introduce the VncWorker type
      ui/vnc: Do not copy z_stream

 ui/vnc.h              |  14 ++-
 ui/vnc-enc-tight.c    | 341 ++++++++++++++++++++++++++------------------------
 ui/vnc-enc-zlib.c     |  34 ++---
 ui/vnc-enc-zrle.c     |  69 +++++-----
 ui/vnc-jobs.c         |   9 +-
 ui/vnc.c              |  56 ++++-----
 ui/vnc-enc-zrle.c.inc |   2 +-
 7 files changed, 258 insertions(+), 267 deletions(-)
---
base-commit: f0737158b483e7ec2b2512145aeab888b85cc1f7
change-id: 20250417-zlib-ce3034f8bc3c

Best regards,
-- 
Akihiko Odaki <odaki@rsg.ci.i.u-tokyo.ac.jp>
Re: [PATCH v2 0/2] ui/vnc: Do not copy z_stream
Posted by Marc-André Lureau 5 months, 2 weeks ago
Hi Akihiko

On Sat, May 31, 2025 at 4:55 PM Akihiko Odaki <odaki@rsg.ci.i.u-tokyo.ac.jp>
wrote:

> vnc_worker_thread_loop() copies z_stream stored in its local VncState to
> the persistent VncState, and the copied one is freed with deflateEnd()
> later. However, deflateEnd() refuses to operate with a copied z_stream
> and returns Z_STREAM_ERROR, leaking the allocated memory.
>
> Avoid copying the zlib state to fix the memory leak.
>
> Signed-off-by: Akihiko Odaki <odaki@rsg.ci.i.u-tokyo.ac.jp>
> ---
> Changes in v2:
> - Rebased.
> - Link to v1:
> https://lore.kernel.org/qemu-devel/20250417-zlib-v1-0-34fad73b843b@daynix.com
>
> ---
> Akihiko Odaki (2):
>       ui/vnc: Introduce the VncWorker type
>       ui/vnc: Do not copy z_stream
>
>  ui/vnc.h              |  14 ++-
>  ui/vnc-enc-tight.c    | 341
> ++++++++++++++++++++++++++------------------------
>  ui/vnc-enc-zlib.c     |  34 ++---
>  ui/vnc-enc-zrle.c     |  69 +++++-----
>  ui/vnc-jobs.c         |   9 +-
>  ui/vnc.c              |  56 ++++-----
>  ui/vnc-enc-zrle.c.inc |   2 +-
>  7 files changed, 258 insertions(+), 267 deletions(-)
> ---
> base-commit: f0737158b483e7ec2b2512145aeab888b85cc1f7
> change-id: 20250417-zlib-ce3034f8bc3c
>

Can you rebase again? thanks