drm/xe: fix devcoredump chunk alignmnent calculation

[PATCH] drm/xe: fix devcoredump chunk alignmnent calculation

Posted by Arnd Bergmann 9 months, 2 weeks ago

From: Arnd Bergmann <arnd@arndb.de>

The device core dumps are copied in 1.5GB chunks, which leads to a
link-time error on 32-bit builds because of the 64-bit division not
getting trivially turned into mask and shift operations:

ERROR: modpost: "__moddi3" [drivers/gpu/drm/xe/xe.ko] undefined!

On top of this, I noticed that the ALIGN_DOWN() usage here cannot
work because that is only defined for power-of-two alignments.
Change ALIGN_DOWN into an explicit div_u64_rem() that avoids the
link error and hopefully produces the right results.

Doing a 1.5GB kvmalloc() does seem a bit suspicious as well, e.g.
this will clearly fail on any 32-bit platform and is also likely
to run out of memory on 64-bit systems under memory pressure, so
using a much smaller power-of-two chunk size might be a good idea
instead.

Fixes: c4a2e5f865b7 ("drm/xe: Add devcoredump chunking")
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
---
Please test this with multi-gigabyte buffers, the original code
was clearly not right, but I don't trust my version either.
---
 drivers/gpu/drm/xe/xe_devcoredump.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/drivers/gpu/drm/xe/xe_devcoredump.c b/drivers/gpu/drm/xe/xe_devcoredump.c
index a9e618abf8ac..4eb70e2d9f68 100644
--- a/drivers/gpu/drm/xe/xe_devcoredump.c
+++ b/drivers/gpu/drm/xe/xe_devcoredump.c
@@ -177,6 +177,7 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
 	struct xe_devcoredump *coredump = data;
 	struct xe_devcoredump_snapshot *ss;
 	ssize_t byte_copied;
+	u32 chunk_offset;
 
 	if (!coredump)
 		return -ENODEV;
@@ -203,8 +204,9 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
 
 	if (offset >= ss->read.chunk_position + XE_DEVCOREDUMP_CHUNK_MAX ||
 	    offset < ss->read.chunk_position) {
-		ss->read.chunk_position =
-			ALIGN_DOWN(offset, XE_DEVCOREDUMP_CHUNK_MAX);
+		ss->read.chunk_position = div_u64_rem(offset,
+			XE_DEVCOREDUMP_CHUNK_MAX, &chunk_offset)
+			* XE_DEVCOREDUMP_CHUNK_MAX;
 
 		__xe_devcoredump_read(ss->read.buffer,
 				      XE_DEVCOREDUMP_CHUNK_MAX,
@@ -213,8 +215,7 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
 
 	byte_copied = count < ss->read.size - offset ? count :
 		ss->read.size - offset;
-	memcpy(buffer, ss->read.buffer +
-	       (offset % XE_DEVCOREDUMP_CHUNK_MAX), byte_copied);
+	memcpy(buffer, ss->read.buffer + chunk_offset, byte_copied);
 
 	mutex_unlock(&coredump->lock);
 
-- 
2.39.5

Re: [PATCH] drm/xe: fix devcoredump chunk alignmnent calculation

Posted by Matthew Brost 9 months, 2 weeks ago

On Tue, Apr 29, 2025 at 09:34:00AM +0200, Arnd Bergmann wrote:
> From: Arnd Bergmann <arnd@arndb.de>
> 
> The device core dumps are copied in 1.5GB chunks, which leads to a
> link-time error on 32-bit builds because of the 64-bit division not
> getting trivially turned into mask and shift operations:
> 
> ERROR: modpost: "__moddi3" [drivers/gpu/drm/xe/xe.ko] undefined!
> 
> On top of this, I noticed that the ALIGN_DOWN() usage here cannot
> work because that is only defined for power-of-two alignments.
> Change ALIGN_DOWN into an explicit div_u64_rem() that avoids the
> link error and hopefully produces the right results.
> 
> Doing a 1.5GB kvmalloc() does seem a bit suspicious as well, e.g.
> this will clearly fail on any 32-bit platform and is also likely
> to run out of memory on 64-bit systems under memory pressure, so
> using a much smaller power-of-two chunk size might be a good idea
> instead.
> 
> Fixes: c4a2e5f865b7 ("drm/xe: Add devcoredump chunking")
> Signed-off-by: Arnd Bergmann <arnd@arndb.de>

Thanks for the fix, I had similar one [1] but I missed issue with
ALIGN_DOWN.

[1] https://patchwork.freedesktop.org/series/148301/

> ---
> Please test this with multi-gigabyte buffers, the original code
> was clearly not right, but I don't trust my version either.

This was tested on 64-bit only. I do see an issue with this version
though. Inline below.

> ---
>  drivers/gpu/drm/xe/xe_devcoredump.c | 9 +++++----
>  1 file changed, 5 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/gpu/drm/xe/xe_devcoredump.c b/drivers/gpu/drm/xe/xe_devcoredump.c
> index a9e618abf8ac..4eb70e2d9f68 100644
> --- a/drivers/gpu/drm/xe/xe_devcoredump.c
> +++ b/drivers/gpu/drm/xe/xe_devcoredump.c
> @@ -177,6 +177,7 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
>  	struct xe_devcoredump *coredump = data;
>  	struct xe_devcoredump_snapshot *ss;
>  	ssize_t byte_copied;
> +	u32 chunk_offset;
>  
>  	if (!coredump)
>  		return -ENODEV;
> @@ -203,8 +204,9 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
>  
>  	if (offset >= ss->read.chunk_position + XE_DEVCOREDUMP_CHUNK_MAX ||
>  	    offset < ss->read.chunk_position) {
> -		ss->read.chunk_position =
> -			ALIGN_DOWN(offset, XE_DEVCOREDUMP_CHUNK_MAX);
> +		ss->read.chunk_position = div_u64_rem(offset,
> +			XE_DEVCOREDUMP_CHUNK_MAX, &chunk_offset)
> +			* XE_DEVCOREDUMP_CHUNK_MAX;
>  
>  		__xe_devcoredump_read(ss->read.buffer,
>  				      XE_DEVCOREDUMP_CHUNK_MAX,
> @@ -213,8 +215,7 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
>  
>  	byte_copied = count < ss->read.size - offset ? count :
>  		ss->read.size - offset;
> -	memcpy(buffer, ss->read.buffer +
> -	       (offset % XE_DEVCOREDUMP_CHUNK_MAX), byte_copied);
> +	memcpy(buffer, ss->read.buffer + chunk_offset, byte_copied);

chunk_offset is unset unless a new devcoredump is read which is every
1.5 GB. You will need always call div_u64_rem outside of the above if
statement.

Matt

>  
>  	mutex_unlock(&coredump->lock);
>  
> -- 
> 2.39.5
>

Re: [PATCH] drm/xe: fix devcoredump chunk alignmnent calculation

Posted by Matthew Brost 9 months, 2 weeks ago

On Tue, Apr 29, 2025 at 10:51:18AM -0700, Matthew Brost wrote:
> On Tue, Apr 29, 2025 at 09:34:00AM +0200, Arnd Bergmann wrote:
> > From: Arnd Bergmann <arnd@arndb.de>
> > 
> > The device core dumps are copied in 1.5GB chunks, which leads to a
> > link-time error on 32-bit builds because of the 64-bit division not
> > getting trivially turned into mask and shift operations:
> > 
> > ERROR: modpost: "__moddi3" [drivers/gpu/drm/xe/xe.ko] undefined!
> > 
> > On top of this, I noticed that the ALIGN_DOWN() usage here cannot
> > work because that is only defined for power-of-two alignments.
> > Change ALIGN_DOWN into an explicit div_u64_rem() that avoids the
> > link error and hopefully produces the right results.
> > 
> > Doing a 1.5GB kvmalloc() does seem a bit suspicious as well, e.g.
> > this will clearly fail on any 32-bit platform and is also likely
> > to run out of memory on 64-bit systems under memory pressure, so
> > using a much smaller power-of-two chunk size might be a good idea
> > instead.
> > 
> > Fixes: c4a2e5f865b7 ("drm/xe: Add devcoredump chunking")
> > Signed-off-by: Arnd Bergmann <arnd@arndb.de>
> 
> Thanks for the fix, I had similar one [1] but I missed issue with
> ALIGN_DOWN.
> 
> [1] https://patchwork.freedesktop.org/series/148301/
> 
> > ---
> > Please test this with multi-gigabyte buffers, the original code
> > was clearly not right, but I don't trust my version either.
> 
> This was tested on 64-bit only. I do see an issue with this version
> though. Inline below.
> 

Oh, yea wrt the ALIGN_DOWN we'd be reading the wrong data. We have WIP
to replay hangs on the GPU and this would break the tool. Good catch.

Matt 

> > ---
> >  drivers/gpu/drm/xe/xe_devcoredump.c | 9 +++++----
> >  1 file changed, 5 insertions(+), 4 deletions(-)
> > 
> > diff --git a/drivers/gpu/drm/xe/xe_devcoredump.c b/drivers/gpu/drm/xe/xe_devcoredump.c
> > index a9e618abf8ac..4eb70e2d9f68 100644
> > --- a/drivers/gpu/drm/xe/xe_devcoredump.c
> > +++ b/drivers/gpu/drm/xe/xe_devcoredump.c
> > @@ -177,6 +177,7 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
> >  	struct xe_devcoredump *coredump = data;
> >  	struct xe_devcoredump_snapshot *ss;
> >  	ssize_t byte_copied;
> > +	u32 chunk_offset;
> >  
> >  	if (!coredump)
> >  		return -ENODEV;
> > @@ -203,8 +204,9 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
> >  
> >  	if (offset >= ss->read.chunk_position + XE_DEVCOREDUMP_CHUNK_MAX ||
> >  	    offset < ss->read.chunk_position) {
> > -		ss->read.chunk_position =
> > -			ALIGN_DOWN(offset, XE_DEVCOREDUMP_CHUNK_MAX);
> > +		ss->read.chunk_position = div_u64_rem(offset,
> > +			XE_DEVCOREDUMP_CHUNK_MAX, &chunk_offset)
> > +			* XE_DEVCOREDUMP_CHUNK_MAX;
> >  
> >  		__xe_devcoredump_read(ss->read.buffer,
> >  				      XE_DEVCOREDUMP_CHUNK_MAX,
> > @@ -213,8 +215,7 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
> >  
> >  	byte_copied = count < ss->read.size - offset ? count :
> >  		ss->read.size - offset;
> > -	memcpy(buffer, ss->read.buffer +
> > -	       (offset % XE_DEVCOREDUMP_CHUNK_MAX), byte_copied);
> > +	memcpy(buffer, ss->read.buffer + chunk_offset, byte_copied);
> 
> chunk_offset is unset unless a new devcoredump is read which is every
> 1.5 GB. You will need always call div_u64_rem outside of the above if
> statement.
> 
> Matt
> 
> >  
> >  	mutex_unlock(&coredump->lock);
> >  
> > -- 
> > 2.39.5
> >