xen/arch/arm/domain_build.c | 32 +++++++++++++--------- xen/arch/arm/platform.c | 5 ++++ xen/arch/arm/platforms/brcm-raspberry-pi.c | 1 + xen/include/asm-arm/mm.h | 2 ++ xen/include/asm-arm/numa.h | 5 ---- xen/include/asm-arm/platform.h | 2 ++ 6 files changed, 29 insertions(+), 18 deletions(-)
From: Julien Grall <jgrall@amazon.com>
Hi all,
At the moment, a user who wants to boot Xen on the Raspberry Pi 4 can
only use the first GB of memory.
This is because several devices cannot DMA above 1GB but Xen doesn't
necessarily allocate memory for Dom0 below 1GB.
This small series is trying to address the problem by allowing a
platform to restrict where Dom0 banks are allocated.
This is also a candidate for Xen 4.14. Without it, a user will not be
able to use all the RAM on the Raspberry Pi 4.
This series has only be slighlty tested. I would appreciate more test on
the Rasbperry Pi 4 to confirm this removing the restriction.
Cheers,
Cc: paul@xen.org
Julien Grall (3):
xen/arm: Allow a platform to override the DMA width
xen/arm: Take into account the DMA width when allocating Dom0 memory
banks
xen/arm: plat: Allocate as much as possible memory below 1GB for dom0
for RPI
xen/arch/arm/domain_build.c | 32 +++++++++++++---------
xen/arch/arm/platform.c | 5 ++++
xen/arch/arm/platforms/brcm-raspberry-pi.c | 1 +
xen/include/asm-arm/mm.h | 2 ++
xen/include/asm-arm/numa.h | 5 ----
xen/include/asm-arm/platform.h | 2 ++
6 files changed, 29 insertions(+), 18 deletions(-)
--
2.17.1
On Mon, 18 May 2020, Julien Grall wrote: > From: Julien Grall <jgrall@amazon.com> > > Hi all, > > At the moment, a user who wants to boot Xen on the Raspberry Pi 4 can > only use the first GB of memory. > > This is because several devices cannot DMA above 1GB but Xen doesn't > necessarily allocate memory for Dom0 below 1GB. > > This small series is trying to address the problem by allowing a > platform to restrict where Dom0 banks are allocated. > > This is also a candidate for Xen 4.14. Without it, a user will not be > able to use all the RAM on the Raspberry Pi 4. The series looks good to me aside from the couple of minor issues being discussed > This series has only be slighlty tested. I would appreciate more test on > the Rasbperry Pi 4 to confirm this removing the restriction. > > Cheers, > > Cc: paul@xen.org > > Julien Grall (3): > xen/arm: Allow a platform to override the DMA width > xen/arm: Take into account the DMA width when allocating Dom0 memory > banks > xen/arm: plat: Allocate as much as possible memory below 1GB for dom0 > for RPI > > xen/arch/arm/domain_build.c | 32 +++++++++++++--------- > xen/arch/arm/platform.c | 5 ++++ > xen/arch/arm/platforms/brcm-raspberry-pi.c | 1 + > xen/include/asm-arm/mm.h | 2 ++ > xen/include/asm-arm/numa.h | 5 ---- > xen/include/asm-arm/platform.h | 2 ++ > 6 files changed, 29 insertions(+), 18 deletions(-) > > -- > 2.17.1 >
Hi Stefano, On 20/05/2020 23:13, Stefano Stabellini wrote: > On Mon, 18 May 2020, Julien Grall wrote: >> From: Julien Grall <jgrall@amazon.com> >> >> Hi all, >> >> At the moment, a user who wants to boot Xen on the Raspberry Pi 4 can >> only use the first GB of memory. >> >> This is because several devices cannot DMA above 1GB but Xen doesn't >> necessarily allocate memory for Dom0 below 1GB. >> >> This small series is trying to address the problem by allowing a >> platform to restrict where Dom0 banks are allocated. >> >> This is also a candidate for Xen 4.14. Without it, a user will not be >> able to use all the RAM on the Raspberry Pi 4. > > The series looks good to me aside from the couple of minor issues being > discussed Thanks, I have sent the v2 yesterday but forgot to call add_maintainers.pl ([1]). Do you want me to resend it with you CCed? Cheers, [1] <20200519172028.31169-1-julien@xen.org> -- Julien Grall
On Thu, 21 May 2020, Julien Grall wrote: > Hi Stefano, > > On 20/05/2020 23:13, Stefano Stabellini wrote: > > On Mon, 18 May 2020, Julien Grall wrote: > > > From: Julien Grall <jgrall@amazon.com> > > > > > > Hi all, > > > > > > At the moment, a user who wants to boot Xen on the Raspberry Pi 4 can > > > only use the first GB of memory. > > > > > > This is because several devices cannot DMA above 1GB but Xen doesn't > > > necessarily allocate memory for Dom0 below 1GB. > > > > > > This small series is trying to address the problem by allowing a > > > platform to restrict where Dom0 banks are allocated. > > > > > > This is also a candidate for Xen 4.14. Without it, a user will not be > > > able to use all the RAM on the Raspberry Pi 4. > > > > The series looks good to me aside from the couple of minor issues being > > discussed > > Thanks, I have sent the v2 yesterday but forgot to call add_maintainers.pl > ([1]). Do you want me to resend it with you CCed? > > Cheers, > > [1] <20200519172028.31169-1-julien@xen.org> No worries I found them now
On Mon, May 18, 2020 at 5:32 AM Julien Grall <julien@xen.org> wrote: > > From: Julien Grall <jgrall@amazon.com> > > Hi all, > > At the moment, a user who wants to boot Xen on the Raspberry Pi 4 can > only use the first GB of memory. > > This is because several devices cannot DMA above 1GB but Xen doesn't > necessarily allocate memory for Dom0 below 1GB. > > This small series is trying to address the problem by allowing a > platform to restrict where Dom0 banks are allocated. > > This is also a candidate for Xen 4.14. Without it, a user will not be > able to use all the RAM on the Raspberry Pi 4. > > This series has only be slighlty tested. I would appreciate more test on > the Rasbperry Pi 4 to confirm this removing the restriction. Hi Julien, could you post a git branch somewhere? I can try this on my rpi4 that already runs 4.13. Thanks, Tamas
On 19/05/2020 04:08, Tamas K Lengyel wrote: > On Mon, May 18, 2020 at 5:32 AM Julien Grall <julien@xen.org> wrote: >> >> From: Julien Grall <jgrall@amazon.com> >> >> Hi all, >> >> At the moment, a user who wants to boot Xen on the Raspberry Pi 4 can >> only use the first GB of memory. >> >> This is because several devices cannot DMA above 1GB but Xen doesn't >> necessarily allocate memory for Dom0 below 1GB. >> >> This small series is trying to address the problem by allowing a >> platform to restrict where Dom0 banks are allocated. >> >> This is also a candidate for Xen 4.14. Without it, a user will not be >> able to use all the RAM on the Raspberry Pi 4. >> >> This series has only be slighlty tested. I would appreciate more test on >> the Rasbperry Pi 4 to confirm this removing the restriction. > > Hi Julien, Hi, > could you post a git branch somewhere? I can try this on my rpi4 that > already runs 4.13. I have pushed a branch based on unstable and the v2 of the series: git://xenbits.xen.org/people/julieng/xen-unstable.git branch arm-dma/v2 Thank you in advance for the testing! Cheers, -- Julien Grall
On Tue, May 19, 2020 at 11:23 AM Julien Grall <julien@xen.org> wrote: > > > > On 19/05/2020 04:08, Tamas K Lengyel wrote: > > On Mon, May 18, 2020 at 5:32 AM Julien Grall <julien@xen.org> wrote: > >> > >> From: Julien Grall <jgrall@amazon.com> > >> > >> Hi all, > >> > >> At the moment, a user who wants to boot Xen on the Raspberry Pi 4 can > >> only use the first GB of memory. > >> > >> This is because several devices cannot DMA above 1GB but Xen doesn't > >> necessarily allocate memory for Dom0 below 1GB. > >> > >> This small series is trying to address the problem by allowing a > >> platform to restrict where Dom0 banks are allocated. > >> > >> This is also a candidate for Xen 4.14. Without it, a user will not be > >> able to use all the RAM on the Raspberry Pi 4. > >> > >> This series has only be slighlty tested. I would appreciate more test on > >> the Rasbperry Pi 4 to confirm this removing the restriction. > > > > Hi Julien, > > Hi, > > > could you post a git branch somewhere? I can try this on my rpi4 that > > already runs 4.13. > > I have pushed a branch based on unstable and the v2 of the series: > > git://xenbits.xen.org/people/julieng/xen-unstable.git > > branch arm-dma/v2 > I've updated my image I built with https://github.com/tklengyel/xen-rpi4-builder a while ago and I've defined 2048m as total_mem and Xen seems to be booting fine and passes execution to dom0. With 512m being set as the Xen cmdline for dom0_mem it was working. When I increased the mem for dom0 the boot is now stuck at: [ 1.427788] of_cfs_init [ 1.429667] of_cfs_init: OK [ 1.432561] clk: Not disabling unused clocks [ 1.437239] Waiting for root device /dev/mmcblk0p2... [ 1.451599] mmc1: queuing unknown CIS tuple 0x80 (2 bytes) [ 1.458156] mmc1: queuing unknown CIS tuple 0x80 (3 bytes) [ 1.464729] mmc1: queuing unknown CIS tuple 0x80 (3 bytes) [ 1.472804] mmc1: queuing unknown CIS tuple 0x80 (7 bytes) [ 1.479370] mmc1: queuing unknown CIS tuple 0x80 (3 bytes) [ 1.546902] random: fast init done [ 1.564590] mmc1: new high speed SDIO card at address 0001 Could this be because the DTB I compiled from a fresh checkout of https://github.com/raspberrypi/linux.git branch rpi-4.19.y whereas the kernel itself is from a checkout ~5 months ago? I guess that must be the cause because even if I decrease the dom0_mem to 512m it still gets stuck at the same spot whereas it was booting fine before. Tamas
On Tue, 19 May 2020, Tamas K Lengyel wrote: > On Tue, May 19, 2020 at 11:23 AM Julien Grall <julien@xen.org> wrote: > > > > > > > > On 19/05/2020 04:08, Tamas K Lengyel wrote: > > > On Mon, May 18, 2020 at 5:32 AM Julien Grall <julien@xen.org> wrote: > > >> > > >> From: Julien Grall <jgrall@amazon.com> > > >> > > >> Hi all, > > >> > > >> At the moment, a user who wants to boot Xen on the Raspberry Pi 4 can > > >> only use the first GB of memory. > > >> > > >> This is because several devices cannot DMA above 1GB but Xen doesn't > > >> necessarily allocate memory for Dom0 below 1GB. > > >> > > >> This small series is trying to address the problem by allowing a > > >> platform to restrict where Dom0 banks are allocated. > > >> > > >> This is also a candidate for Xen 4.14. Without it, a user will not be > > >> able to use all the RAM on the Raspberry Pi 4. > > >> > > >> This series has only be slighlty tested. I would appreciate more test on > > >> the Rasbperry Pi 4 to confirm this removing the restriction. > > > > > > Hi Julien, > > > > Hi, > > > > > could you post a git branch somewhere? I can try this on my rpi4 that > > > already runs 4.13. > > > > I have pushed a branch based on unstable and the v2 of the series: > > > > git://xenbits.xen.org/people/julieng/xen-unstable.git > > > > branch arm-dma/v2 > > > > I've updated my image I built with > https://github.com/tklengyel/xen-rpi4-builder a while ago and I've > defined 2048m as total_mem and Xen seems to be booting fine and passes > execution to dom0. With 512m being set as the Xen cmdline for dom0_mem > it was working. When I increased the mem for dom0 the boot is now > stuck at: > > [ 1.427788] of_cfs_init > [ 1.429667] of_cfs_init: OK > [ 1.432561] clk: Not disabling unused clocks > [ 1.437239] Waiting for root device /dev/mmcblk0p2... > [ 1.451599] mmc1: queuing unknown CIS tuple 0x80 (2 bytes) > [ 1.458156] mmc1: queuing unknown CIS tuple 0x80 (3 bytes) > [ 1.464729] mmc1: queuing unknown CIS tuple 0x80 (3 bytes) > [ 1.472804] mmc1: queuing unknown CIS tuple 0x80 (7 bytes) > [ 1.479370] mmc1: queuing unknown CIS tuple 0x80 (3 bytes) > [ 1.546902] random: fast init done > [ 1.564590] mmc1: new high speed SDIO card at address 0001 > > Could this be because the DTB I compiled from a fresh checkout of > https://github.com/raspberrypi/linux.git branch rpi-4.19.y whereas the > kernel itself is from a checkout ~5 months ago? I guess that must be > the cause because even if I decrease the dom0_mem to 512m it still > gets stuck at the same spot whereas it was booting fine before. Just so that you are aware, there is a known issue with setting dom0_mem greater than 512M. I have a WIP patch to fix it in Linux that I plan to send to the list soon.
On Tue, May 19, 2020 at 4:44 PM Tamas K Lengyel <tamas.k.lengyel@gmail.com> wrote: > > On Tue, May 19, 2020 at 11:23 AM Julien Grall <julien@xen.org> wrote: > > > > > > > > On 19/05/2020 04:08, Tamas K Lengyel wrote: > > > On Mon, May 18, 2020 at 5:32 AM Julien Grall <julien@xen.org> wrote: > > >> > > >> From: Julien Grall <jgrall@amazon.com> > > >> > > >> Hi all, > > >> > > >> At the moment, a user who wants to boot Xen on the Raspberry Pi 4 can > > >> only use the first GB of memory. > > >> > > >> This is because several devices cannot DMA above 1GB but Xen doesn't > > >> necessarily allocate memory for Dom0 below 1GB. > > >> > > >> This small series is trying to address the problem by allowing a > > >> platform to restrict where Dom0 banks are allocated. > > >> > > >> This is also a candidate for Xen 4.14. Without it, a user will not be > > >> able to use all the RAM on the Raspberry Pi 4. > > >> > > >> This series has only be slighlty tested. I would appreciate more test on > > >> the Rasbperry Pi 4 to confirm this removing the restriction. > > > > > > Hi Julien, > > > > Hi, > > > > > could you post a git branch somewhere? I can try this on my rpi4 that > > > already runs 4.13. > > > > I have pushed a branch based on unstable and the v2 of the series: > > > > git://xenbits.xen.org/people/julieng/xen-unstable.git > > > > branch arm-dma/v2 > > > > I've updated my image I built with > https://github.com/tklengyel/xen-rpi4-builder a while ago and I've > defined 2048m as total_mem and Xen seems to be booting fine and passes > execution to dom0. With 512m being set as the Xen cmdline for dom0_mem > it was working. When I increased the mem for dom0 the boot is now > stuck at: > > [ 1.427788] of_cfs_init > [ 1.429667] of_cfs_init: OK > [ 1.432561] clk: Not disabling unused clocks > [ 1.437239] Waiting for root device /dev/mmcblk0p2... > [ 1.451599] mmc1: queuing unknown CIS tuple 0x80 (2 bytes) > [ 1.458156] mmc1: queuing unknown CIS tuple 0x80 (3 bytes) > [ 1.464729] mmc1: queuing unknown CIS tuple 0x80 (3 bytes) > [ 1.472804] mmc1: queuing unknown CIS tuple 0x80 (7 bytes) > [ 1.479370] mmc1: queuing unknown CIS tuple 0x80 (3 bytes) > [ 1.546902] random: fast init done > [ 1.564590] mmc1: new high speed SDIO card at address 0001 > > Could this be because the DTB I compiled from a fresh checkout of > https://github.com/raspberrypi/linux.git branch rpi-4.19.y whereas the > kernel itself is from a checkout ~5 months ago? I guess that must be > the cause because even if I decrease the dom0_mem to 512m it still > gets stuck at the same spot whereas it was booting fine before. Stefano and I are testing the fix right now -- for now just set your Dom0 mem to less than 512m. Thanks, Roman.
On Tue, May 19, 2020 at 5:50 PM Roman Shaposhnik <roman@zededa.com> wrote: > > On Tue, May 19, 2020 at 4:44 PM Tamas K Lengyel > <tamas.k.lengyel@gmail.com> wrote: > > > > On Tue, May 19, 2020 at 11:23 AM Julien Grall <julien@xen.org> wrote: > > > > > > > > > > > > On 19/05/2020 04:08, Tamas K Lengyel wrote: > > > > On Mon, May 18, 2020 at 5:32 AM Julien Grall <julien@xen.org> wrote: > > > >> > > > >> From: Julien Grall <jgrall@amazon.com> > > > >> > > > >> Hi all, > > > >> > > > >> At the moment, a user who wants to boot Xen on the Raspberry Pi 4 can > > > >> only use the first GB of memory. > > > >> > > > >> This is because several devices cannot DMA above 1GB but Xen doesn't > > > >> necessarily allocate memory for Dom0 below 1GB. > > > >> > > > >> This small series is trying to address the problem by allowing a > > > >> platform to restrict where Dom0 banks are allocated. > > > >> > > > >> This is also a candidate for Xen 4.14. Without it, a user will not be > > > >> able to use all the RAM on the Raspberry Pi 4. > > > >> > > > >> This series has only be slighlty tested. I would appreciate more test on > > > >> the Rasbperry Pi 4 to confirm this removing the restriction. > > > > > > > > Hi Julien, > > > > > > Hi, > > > > > > > could you post a git branch somewhere? I can try this on my rpi4 that > > > > already runs 4.13. > > > > > > I have pushed a branch based on unstable and the v2 of the series: > > > > > > git://xenbits.xen.org/people/julieng/xen-unstable.git > > > > > > branch arm-dma/v2 > > > > > > > I've updated my image I built with > > https://github.com/tklengyel/xen-rpi4-builder a while ago and I've > > defined 2048m as total_mem and Xen seems to be booting fine and passes > > execution to dom0. With 512m being set as the Xen cmdline for dom0_mem > > it was working. When I increased the mem for dom0 the boot is now > > stuck at: > > > > [ 1.427788] of_cfs_init > > [ 1.429667] of_cfs_init: OK > > [ 1.432561] clk: Not disabling unused clocks > > [ 1.437239] Waiting for root device /dev/mmcblk0p2... > > [ 1.451599] mmc1: queuing unknown CIS tuple 0x80 (2 bytes) > > [ 1.458156] mmc1: queuing unknown CIS tuple 0x80 (3 bytes) > > [ 1.464729] mmc1: queuing unknown CIS tuple 0x80 (3 bytes) > > [ 1.472804] mmc1: queuing unknown CIS tuple 0x80 (7 bytes) > > [ 1.479370] mmc1: queuing unknown CIS tuple 0x80 (3 bytes) > > [ 1.546902] random: fast init done > > [ 1.564590] mmc1: new high speed SDIO card at address 0001 > > > > Could this be because the DTB I compiled from a fresh checkout of > > https://github.com/raspberrypi/linux.git branch rpi-4.19.y whereas the > > kernel itself is from a checkout ~5 months ago? I guess that must be > > the cause because even if I decrease the dom0_mem to 512m it still > > gets stuck at the same spot whereas it was booting fine before. > > Stefano and I are testing the fix right now -- for now just set your > Dom0 mem to less than 512m. Actually seems to work after I recompiled the kernel and reinstalled all kernel modules. Xen boots with 4gb RAM and dom0 boots with 2g: xl info: ... total_memory : 3956 free_memory : 1842 cat /proc/meminfo MemTotal: 1963844 kB I get an emergency shell during boot on the console complaining about xenbr0 not coming up but if I just hit continue it boots fine and the network is up. So AFAICT things are good. Cheers, Tamas
On Tue, May 19, 2020, 7:15 PM Tamas K Lengyel <tamas.k.lengyel@gmail.com> wrote: > On Tue, May 19, 2020 at 5:50 PM Roman Shaposhnik <roman@zededa.com> wrote: > > > > On Tue, May 19, 2020 at 4:44 PM Tamas K Lengyel > > <tamas.k.lengyel@gmail.com> wrote: > > > > > > On Tue, May 19, 2020 at 11:23 AM Julien Grall <julien@xen.org> wrote: > > > > > > > > > > > > > > > > On 19/05/2020 04:08, Tamas K Lengyel wrote: > > > > > On Mon, May 18, 2020 at 5:32 AM Julien Grall <julien@xen.org> > wrote: > > > > >> > > > > >> From: Julien Grall <jgrall@amazon.com> > > > > >> > > > > >> Hi all, > > > > >> > > > > >> At the moment, a user who wants to boot Xen on the Raspberry Pi 4 > can > > > > >> only use the first GB of memory. > > > > >> > > > > >> This is because several devices cannot DMA above 1GB but Xen > doesn't > > > > >> necessarily allocate memory for Dom0 below 1GB. > > > > >> > > > > >> This small series is trying to address the problem by allowing a > > > > >> platform to restrict where Dom0 banks are allocated. > > > > >> > > > > >> This is also a candidate for Xen 4.14. Without it, a user will > not be > > > > >> able to use all the RAM on the Raspberry Pi 4. > > > > >> > > > > >> This series has only be slighlty tested. I would appreciate more > test on > > > > >> the Rasbperry Pi 4 to confirm this removing the restriction. > > > > > > > > > > Hi Julien, > > > > > > > > Hi, > > > > > > > > > could you post a git branch somewhere? I can try this on my rpi4 > that > > > > > already runs 4.13. > > > > > > > > I have pushed a branch based on unstable and the v2 of the series: > > > > > > > > git://xenbits.xen.org/people/julieng/xen-unstable.git > > > > > > > > branch arm-dma/v2 > > > > > > > > > > I've updated my image I built with > > > https://github.com/tklengyel/xen-rpi4-builder a while ago and I've > > > defined 2048m as total_mem and Xen seems to be booting fine and passes > > > execution to dom0. With 512m being set as the Xen cmdline for dom0_mem > > > it was working. When I increased the mem for dom0 the boot is now > > > stuck at: > > > > > > [ 1.427788] of_cfs_init > > > [ 1.429667] of_cfs_init: OK > > > [ 1.432561] clk: Not disabling unused clocks > > > [ 1.437239] Waiting for root device /dev/mmcblk0p2... > > > [ 1.451599] mmc1: queuing unknown CIS tuple 0x80 (2 bytes) > > > [ 1.458156] mmc1: queuing unknown CIS tuple 0x80 (3 bytes) > > > [ 1.464729] mmc1: queuing unknown CIS tuple 0x80 (3 bytes) > > > [ 1.472804] mmc1: queuing unknown CIS tuple 0x80 (7 bytes) > > > [ 1.479370] mmc1: queuing unknown CIS tuple 0x80 (3 bytes) > > > [ 1.546902] random: fast init done > > > [ 1.564590] mmc1: new high speed SDIO card at address 0001 > > > > > > Could this be because the DTB I compiled from a fresh checkout of > > > https://github.com/raspberrypi/linux.git branch rpi-4.19.y whereas the > > > kernel itself is from a checkout ~5 months ago? I guess that must be > > > the cause because even if I decrease the dom0_mem to 512m it still > > > gets stuck at the same spot whereas it was booting fine before. > > > > Stefano and I are testing the fix right now -- for now just set your > > Dom0 mem to less than 512m. > > Actually seems to work after I recompiled the kernel and reinstalled > all kernel modules. Xen boots with 4gb RAM and dom0 boots with 2g: > > xl info: > ... > total_memory : 3956 > free_memory : 1842 > > cat /proc/meminfo > MemTotal: 1963844 kB > > I get an emergency shell during boot on the console complaining about > xenbr0 not coming up but if I just hit continue it boots fine and the > network is up. So AFAICT things are good. > What exact version of the kernel are you using and what did you build it from? FWIW: 5.6.x clearly has an issue with DMA. Thanks, Roman. >
On Tue, May 19, 2020 at 8:28 PM Roman Shaposhnik <roman@zededa.com> wrote: > > On Tue, May 19, 2020, 7:15 PM Tamas K Lengyel <tamas.k.lengyel@gmail.com> wrote: >> >> On Tue, May 19, 2020 at 5:50 PM Roman Shaposhnik <roman@zededa.com> wrote: >> > >> > On Tue, May 19, 2020 at 4:44 PM Tamas K Lengyel >> > <tamas.k.lengyel@gmail.com> wrote: >> > > >> > > On Tue, May 19, 2020 at 11:23 AM Julien Grall <julien@xen.org> wrote: >> > > > >> > > > >> > > > >> > > > On 19/05/2020 04:08, Tamas K Lengyel wrote: >> > > > > On Mon, May 18, 2020 at 5:32 AM Julien Grall <julien@xen.org> wrote: >> > > > >> >> > > > >> From: Julien Grall <jgrall@amazon.com> >> > > > >> >> > > > >> Hi all, >> > > > >> >> > > > >> At the moment, a user who wants to boot Xen on the Raspberry Pi 4 can >> > > > >> only use the first GB of memory. >> > > > >> >> > > > >> This is because several devices cannot DMA above 1GB but Xen doesn't >> > > > >> necessarily allocate memory for Dom0 below 1GB. >> > > > >> >> > > > >> This small series is trying to address the problem by allowing a >> > > > >> platform to restrict where Dom0 banks are allocated. >> > > > >> >> > > > >> This is also a candidate for Xen 4.14. Without it, a user will not be >> > > > >> able to use all the RAM on the Raspberry Pi 4. >> > > > >> >> > > > >> This series has only be slighlty tested. I would appreciate more test on >> > > > >> the Rasbperry Pi 4 to confirm this removing the restriction. >> > > > > >> > > > > Hi Julien, >> > > > >> > > > Hi, >> > > > >> > > > > could you post a git branch somewhere? I can try this on my rpi4 that >> > > > > already runs 4.13. >> > > > >> > > > I have pushed a branch based on unstable and the v2 of the series: >> > > > >> > > > git://xenbits.xen.org/people/julieng/xen-unstable.git >> > > > >> > > > branch arm-dma/v2 >> > > > >> > > >> > > I've updated my image I built with >> > > https://github.com/tklengyel/xen-rpi4-builder a while ago and I've >> > > defined 2048m as total_mem and Xen seems to be booting fine and passes >> > > execution to dom0. With 512m being set as the Xen cmdline for dom0_mem >> > > it was working. When I increased the mem for dom0 the boot is now >> > > stuck at: >> > > >> > > [ 1.427788] of_cfs_init >> > > [ 1.429667] of_cfs_init: OK >> > > [ 1.432561] clk: Not disabling unused clocks >> > > [ 1.437239] Waiting for root device /dev/mmcblk0p2... >> > > [ 1.451599] mmc1: queuing unknown CIS tuple 0x80 (2 bytes) >> > > [ 1.458156] mmc1: queuing unknown CIS tuple 0x80 (3 bytes) >> > > [ 1.464729] mmc1: queuing unknown CIS tuple 0x80 (3 bytes) >> > > [ 1.472804] mmc1: queuing unknown CIS tuple 0x80 (7 bytes) >> > > [ 1.479370] mmc1: queuing unknown CIS tuple 0x80 (3 bytes) >> > > [ 1.546902] random: fast init done >> > > [ 1.564590] mmc1: new high speed SDIO card at address 0001 >> > > >> > > Could this be because the DTB I compiled from a fresh checkout of >> > > https://github.com/raspberrypi/linux.git branch rpi-4.19.y whereas the >> > > kernel itself is from a checkout ~5 months ago? I guess that must be >> > > the cause because even if I decrease the dom0_mem to 512m it still >> > > gets stuck at the same spot whereas it was booting fine before. >> > >> > Stefano and I are testing the fix right now -- for now just set your >> > Dom0 mem to less than 512m. >> >> Actually seems to work after I recompiled the kernel and reinstalled >> all kernel modules. Xen boots with 4gb RAM and dom0 boots with 2g: >> >> xl info: >> ... >> total_memory : 3956 >> free_memory : 1842 >> >> cat /proc/meminfo >> MemTotal: 1963844 kB >> >> I get an emergency shell during boot on the console complaining about >> xenbr0 not coming up but if I just hit continue it boots fine and the >> network is up. So AFAICT things are good. > > > What exact version of the kernel are you using and what did you build it from? > > FWIW: 5.6.x clearly has an issue with DMA. As I said above: https://github.com/raspberrypi/linux.git branch rpi-4.19.y, I applied the Linux patches from the xen-rpi4-builder repo, just changing the dom0_mem option in patch 1. I reverted the xen-rpi4-builder a couple revisions as to not build using the DTB overlay. Tamas
© 2016 - 2024 Red Hat, Inc.