-LONG_MIN cannot be represented in a long and hence is UB, for being one
larger than LONG_MAX.

The caller passing an unsigned long and the 1st param also being (array
of) unsigned long, change the 2nd param accordingly while adding the
sole necessary cast. This was the original form of the function anyway.

Fixes: 5644ce014223 ("x86emul: relax asm() constraints")
Oss-fuzz: 70923
Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/xen/arch/x86/x86_emulate/x86_emulate.c
+++ b/xen/arch/x86/x86_emulate/x86_emulate.c
@@ -607,9 +607,9 @@ static bool div_dbl(unsigned long u[2],
  * NB. We don't use idiv directly as it's moderately hard to work out
  *     ahead of time whether it will #DE, which we cannot allow to happen.
  */
-static bool idiv_dbl(unsigned long u[2], long v)
+static bool idiv_dbl(unsigned long u[2], unsigned long v)
 {
-    bool negu = (long)u[1] < 0, negv = v < 0;
+    bool negu = (long)u[1] < 0, negv = (long)v < 0;
 
     /* u = abs(u) */
     if ( negu )

On Mon, 5 Aug 2024, Jan Beulich wrote:
> -LONG_MIN cannot be represented in a long and hence is UB, for being one
> larger than LONG_MAX.
> 
> The caller passing an unsigned long and the 1st param also being (array
> of) unsigned long, change the 2nd param accordingly while adding the
> sole necessary cast. This was the original form of the function anyway.
> 
> Fixes: 5644ce014223 ("x86emul: relax asm() constraints")
> Oss-fuzz: 70923
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> --- a/xen/arch/x86/x86_emulate/x86_emulate.c
> +++ b/xen/arch/x86/x86_emulate/x86_emulate.c
> @@ -607,9 +607,9 @@ static bool div_dbl(unsigned long u[2],
>   * NB. We don't use idiv directly as it's moderately hard to work out
>   *     ahead of time whether it will #DE, which we cannot allow to happen.
>   */
> -static bool idiv_dbl(unsigned long u[2], long v)
> +static bool idiv_dbl(unsigned long u[2], unsigned long v)
>  {
> -    bool negu = (long)u[1] < 0, negv = v < 0;
> +    bool negu = (long)u[1] < 0, negv = (long)v < 0;
>  
>      /* u = abs(u) */
>      if ( negu )
>