[PATCH 1/3] ui/vnc.c: replace big endian flag with byte order value

Daniel P. Berrangé posted 3 patches 6 months ago
Maintainers: "Marc-André Lureau" <marcandre.lureau@redhat.com>
There is a newer version of this series
[PATCH 1/3] ui/vnc.c: replace big endian flag with byte order value
Posted by Daniel P. Berrangé 6 months ago
From: Daniel P. Berrangé <berrange@redhat.com>

It will make it easier to do certain comparisons in future if we
store G_BIG_ENDIAN/G_LITTLE_ENDIAN directly, instead of a boolean
flag.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 ui/vnc-enc-tight.c | 2 +-
 ui/vnc-enc-zrle.c  | 2 +-
 ui/vnc-jobs.c      | 2 +-
 ui/vnc.c           | 6 +++---
 ui/vnc.h           | 2 +-
 5 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/ui/vnc-enc-tight.c b/ui/vnc-enc-tight.c
index 41f559eb83..f51f618e46 100644
--- a/ui/vnc-enc-tight.c
+++ b/ui/vnc-enc-tight.c
@@ -150,7 +150,7 @@ tight_detect_smooth_image24(VncState *vs, int w, int h)
      * If client is big-endian, color samples begin from the second
      * byte (offset 1) of a 32-bit pixel value.
      */
-    off = vs->client_be;
+    off = vs->client_bo == G_BIG_ENDIAN ? 1 : 0;
 
     memset(stats, 0, sizeof (stats));
 
diff --git a/ui/vnc-enc-zrle.c b/ui/vnc-enc-zrle.c
index bd33b89063..9032e82ee5 100644
--- a/ui/vnc-enc-zrle.c
+++ b/ui/vnc-enc-zrle.c
@@ -255,7 +255,7 @@ static void zrle_write_u8(VncState *vs, uint8_t value)
 static int zrle_send_framebuffer_update(VncState *vs, int x, int y,
                                         int w, int h)
 {
-    bool be = vs->client_be;
+    bool be = vs->client_bo == G_BIG_ENDIAN;
     size_t bytes;
     int zywrle_level;
 
diff --git a/ui/vnc-jobs.c b/ui/vnc-jobs.c
index fcca7ec632..fd019e0a35 100644
--- a/ui/vnc-jobs.c
+++ b/ui/vnc-jobs.c
@@ -188,7 +188,7 @@ static void vnc_async_encoding_start(VncState *orig, VncState *local)
     local->lossy_rect = orig->lossy_rect;
     local->write_pixels = orig->write_pixels;
     local->client_pf = orig->client_pf;
-    local->client_be = orig->client_be;
+    local->client_bo = orig->client_bo;
     local->tight = orig->tight;
     local->zlib = orig->zlib;
     local->hextile = orig->hextile;
diff --git a/ui/vnc.c b/ui/vnc.c
index 9e097dc4b4..2c640d8be4 100644
--- a/ui/vnc.c
+++ b/ui/vnc.c
@@ -891,7 +891,7 @@ void vnc_convert_pixel(VncState *vs, uint8_t *buf, uint32_t v)
         buf[0] = v;
         break;
     case 2:
-        if (vs->client_be) {
+        if (vs->client_bo == G_BIG_ENDIAN) {
             buf[0] = v >> 8;
             buf[1] = v;
         } else {
@@ -901,7 +901,7 @@ void vnc_convert_pixel(VncState *vs, uint8_t *buf, uint32_t v)
         break;
     default:
     case 4:
-        if (vs->client_be) {
+        if (vs->client_bo == G_BIG_ENDIAN) {
             buf[0] = v >> 24;
             buf[1] = v >> 16;
             buf[2] = v >> 8;
@@ -2312,7 +2312,7 @@ static void set_pixel_format(VncState *vs, int bits_per_pixel,
     vs->client_pf.bits_per_pixel = bits_per_pixel;
     vs->client_pf.bytes_per_pixel = bits_per_pixel / 8;
     vs->client_pf.depth = bits_per_pixel == 32 ? 24 : bits_per_pixel;
-    vs->client_be = big_endian_flag;
+    vs->client_bo = big_endian_flag ? G_BIG_ENDIAN : G_LITTLE_ENDIAN;
 
     if (!true_color_flag) {
         send_color_map(vs);
diff --git a/ui/vnc.h b/ui/vnc.h
index acc53a2cc1..f2a627dcdf 100644
--- a/ui/vnc.h
+++ b/ui/vnc.h
@@ -323,7 +323,7 @@ struct VncState
     VncWritePixels *write_pixels;
     PixelFormat client_pf;
     pixman_format_code_t client_format;
-    bool client_be;
+    int client_bo; /* G_LITTLE_ENDIAN or G_BIG_ENDIAN */
 
     CaptureVoiceOut *audio_cap;
     struct audsettings as;
-- 
2.49.0


Re: [PATCH 1/3] ui/vnc.c: replace big endian flag with byte order value
Posted by Philippe Mathieu-Daudé 6 months ago
On 13/5/25 12:08, Daniel P. Berrangé wrote:
> From: Daniel P. Berrangé <berrange@redhat.com>
> 
> It will make it easier to do certain comparisons in future if we
> store G_BIG_ENDIAN/G_LITTLE_ENDIAN directly, instead of a boolean
> flag.
> 
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
>   ui/vnc-enc-tight.c | 2 +-
>   ui/vnc-enc-zrle.c  | 2 +-
>   ui/vnc-jobs.c      | 2 +-
>   ui/vnc.c           | 6 +++---
>   ui/vnc.h           | 2 +-
>   5 files changed, 7 insertions(+), 7 deletions(-)


> diff --git a/ui/vnc.h b/ui/vnc.h
> index acc53a2cc1..f2a627dcdf 100644
> --- a/ui/vnc.h
> +++ b/ui/vnc.h
> @@ -323,7 +323,7 @@ struct VncState
>       VncWritePixels *write_pixels;
>       PixelFormat client_pf;
>       pixman_format_code_t client_format;
> -    bool client_be;
> +    int client_bo; /* G_LITTLE_ENDIAN or G_BIG_ENDIAN */

At least we don't care about G_PDP_ENDIAN ;)

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>


Re: [PATCH 1/3] ui/vnc.c: replace big endian flag with byte order value
Posted by Philippe Mathieu-Daudé 6 months ago
On 13/5/25 13:08, Philippe Mathieu-Daudé wrote:
> On 13/5/25 12:08, Daniel P. Berrangé wrote:
>> From: Daniel P. Berrangé <berrange@redhat.com>
>>
>> It will make it easier to do certain comparisons in future if we
>> store G_BIG_ENDIAN/G_LITTLE_ENDIAN directly, instead of a boolean
>> flag.
>>
>> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
>> ---
>>   ui/vnc-enc-tight.c | 2 +-
>>   ui/vnc-enc-zrle.c  | 2 +-
>>   ui/vnc-jobs.c      | 2 +-
>>   ui/vnc.c           | 6 +++---
>>   ui/vnc.h           | 2 +-
>>   5 files changed, 7 insertions(+), 7 deletions(-)
> 
> 
>> diff --git a/ui/vnc.h b/ui/vnc.h
>> index acc53a2cc1..f2a627dcdf 100644
>> --- a/ui/vnc.h
>> +++ b/ui/vnc.h
>> @@ -323,7 +323,7 @@ struct VncState
>>       VncWritePixels *write_pixels;
>>       PixelFormat client_pf;
>>       pixman_format_code_t client_format;
>> -    bool client_be;
>> +    int client_bo; /* G_LITTLE_ENDIAN or G_BIG_ENDIAN */

'bo' = 'big order'?

Maybe 'client_endian' or even 'client_endianness'?

> 
> At least we don't care about G_PDP_ENDIAN ;)
> 
> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> 


Re: [PATCH 1/3] ui/vnc.c: replace big endian flag with byte order value
Posted by BALATON Zoltan 6 months ago
On Tue, 13 May 2025, Philippe Mathieu-Daudé wrote:
> On 13/5/25 13:08, Philippe Mathieu-Daudé wrote:
>> On 13/5/25 12:08, Daniel P. Berrangé wrote:
>>> From: Daniel P. Berrangé <berrange@redhat.com>
>>> 
>>> It will make it easier to do certain comparisons in future if we
>>> store G_BIG_ENDIAN/G_LITTLE_ENDIAN directly, instead of a boolean
>>> flag.
>>> 
>>> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
>>> ---
>>>   ui/vnc-enc-tight.c | 2 +-
>>>   ui/vnc-enc-zrle.c  | 2 +-
>>>   ui/vnc-jobs.c      | 2 +-
>>>   ui/vnc.c           | 6 +++---
>>>   ui/vnc.h           | 2 +-
>>>   5 files changed, 7 insertions(+), 7 deletions(-)
>> 
>> 
>>> diff --git a/ui/vnc.h b/ui/vnc.h
>>> index acc53a2cc1..f2a627dcdf 100644
>>> --- a/ui/vnc.h
>>> +++ b/ui/vnc.h
>>> @@ -323,7 +323,7 @@ struct VncState
>>>       VncWritePixels *write_pixels;
>>>       PixelFormat client_pf;
>>>       pixman_format_code_t client_format;
>>> -    bool client_be;
>>> +    int client_bo; /* G_LITTLE_ENDIAN or G_BIG_ENDIAN */
>
> 'bo' = 'big order'?
>
> Maybe 'client_endian' or even 'client_endianness'?

I also thought bo makes no sense. What's the problem with be which was 
more obvious as it is? If you need to change it client_endian would still 
be more descriptive, I can't guess what client_bo stands for without the 
comment and even with that it's not obvious.

Regards,
BALATON Zoltan
Re: [PATCH 1/3] ui/vnc.c: replace big endian flag with byte order value
Posted by Daniel P. Berrangé 6 months ago
On Tue, May 13, 2025 at 12:11:05PM +0100, Philippe Mathieu-Daudé wrote:
> On 13/5/25 13:08, Philippe Mathieu-Daudé wrote:
> > On 13/5/25 12:08, Daniel P. Berrangé wrote:
> > > From: Daniel P. Berrangé <berrange@redhat.com>
> > > 
> > > It will make it easier to do certain comparisons in future if we
> > > store G_BIG_ENDIAN/G_LITTLE_ENDIAN directly, instead of a boolean
> > > flag.
> > > 
> > > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> > > ---
> > >   ui/vnc-enc-tight.c | 2 +-
> > >   ui/vnc-enc-zrle.c  | 2 +-
> > >   ui/vnc-jobs.c      | 2 +-
> > >   ui/vnc.c           | 6 +++---
> > >   ui/vnc.h           | 2 +-
> > >   5 files changed, 7 insertions(+), 7 deletions(-)
> > 
> > 
> > > diff --git a/ui/vnc.h b/ui/vnc.h
> > > index acc53a2cc1..f2a627dcdf 100644
> > > --- a/ui/vnc.h
> > > +++ b/ui/vnc.h
> > > @@ -323,7 +323,7 @@ struct VncState
> > >       VncWritePixels *write_pixels;
> > >       PixelFormat client_pf;
> > >       pixman_format_code_t client_format;
> > > -    bool client_be;
> > > +    int client_bo; /* G_LITTLE_ENDIAN or G_BIG_ENDIAN */
> 
> 'bo' = 'big order'?

bo == byte order.


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|


Re: [PATCH 1/3] ui/vnc.c: replace big endian flag with byte order value
Posted by Philippe Mathieu-Daudé 6 months ago
On 13/5/25 13:13, Daniel P. Berrangé wrote:
> On Tue, May 13, 2025 at 12:11:05PM +0100, Philippe Mathieu-Daudé wrote:
>> On 13/5/25 13:08, Philippe Mathieu-Daudé wrote:
>>> On 13/5/25 12:08, Daniel P. Berrangé wrote:
>>>> From: Daniel P. Berrangé <berrange@redhat.com>
>>>>
>>>> It will make it easier to do certain comparisons in future if we
>>>> store G_BIG_ENDIAN/G_LITTLE_ENDIAN directly, instead of a boolean
>>>> flag.
>>>>
>>>> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
>>>> ---
>>>>    ui/vnc-enc-tight.c | 2 +-
>>>>    ui/vnc-enc-zrle.c  | 2 +-
>>>>    ui/vnc-jobs.c      | 2 +-
>>>>    ui/vnc.c           | 6 +++---
>>>>    ui/vnc.h           | 2 +-
>>>>    5 files changed, 7 insertions(+), 7 deletions(-)
>>>
>>>
>>>> diff --git a/ui/vnc.h b/ui/vnc.h
>>>> index acc53a2cc1..f2a627dcdf 100644
>>>> --- a/ui/vnc.h
>>>> +++ b/ui/vnc.h
>>>> @@ -323,7 +323,7 @@ struct VncState
>>>>        VncWritePixels *write_pixels;
>>>>        PixelFormat client_pf;
>>>>        pixman_format_code_t client_format;
>>>> -    bool client_be;
>>>> +    int client_bo; /* G_LITTLE_ENDIAN or G_BIG_ENDIAN */
>>
>> 'bo' = 'big order'?
> 
> bo == byte order.

Oh of course. Since no GLib endian enum, preferably:

   int client_bo; /* byte order: G_LITTLE_ENDIAN or G_BIG_ENDIAN */