From: Bin Meng <bin.meng@windriver.com>

Update virt and sifive_u machines to use the opensbi fw_dynamic bios
image built for the generic FDT platform.

Remove the out-of-date no longer used bios images.

Signed-off-by: Bin Meng <bin.meng@windriver.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>

---

Changes in v4:
- Remove old binaries in the Makefile for `make install` bisection

Changes in v3:
- Change to fw_dynamic.bin for virt & sifive_u

 Makefile                                       |   4 +---
 hw/riscv/sifive_u.c                            |   4 ++--
 hw/riscv/virt.c                                |   4 ++--
 pc-bios/opensbi-riscv32-generic-fw_dynamic.bin | Bin 0 -> 62144 bytes
 pc-bios/opensbi-riscv32-sifive_u-fw_jump.bin   | Bin 49520 -> 0 bytes
 pc-bios/opensbi-riscv32-virt-fw_jump.bin       | Bin 49504 -> 0 bytes
 pc-bios/opensbi-riscv64-generic-fw_dynamic.bin | Bin 0 -> 70792 bytes
 pc-bios/opensbi-riscv64-sifive_u-fw_jump.bin   | Bin 57936 -> 0 bytes
 pc-bios/opensbi-riscv64-virt-fw_jump.bin       | Bin 57920 -> 0 bytes
 9 files changed, 5 insertions(+), 7 deletions(-)
 create mode 100644 pc-bios/opensbi-riscv32-generic-fw_dynamic.bin
 delete mode 100644 pc-bios/opensbi-riscv32-sifive_u-fw_jump.bin
 delete mode 100644 pc-bios/opensbi-riscv32-virt-fw_jump.bin
 create mode 100644 pc-bios/opensbi-riscv64-generic-fw_dynamic.bin
 delete mode 100644 pc-bios/opensbi-riscv64-sifive_u-fw_jump.bin
 delete mode 100644 pc-bios/opensbi-riscv64-virt-fw_jump.bin

diff --git a/Makefile b/Makefile
index b1b8a5a..f06b3ae 100644
--- a/Makefile
+++ b/Makefile
@@ -840,9 +840,7 @@ palcode-clipper \
 u-boot.e500 u-boot-sam460-20100605.bin \
 qemu_vga.ndrv \
 edk2-licenses.txt \
-hppa-firmware.img \
-opensbi-riscv32-sifive_u-fw_jump.bin opensbi-riscv32-virt-fw_jump.bin \
-opensbi-riscv64-sifive_u-fw_jump.bin opensbi-riscv64-virt-fw_jump.bin
+hppa-firmware.img
 
 
 DESCS=50-edk2-i386-secure.json 50-edk2-x86_64-secure.json \
diff --git a/hw/riscv/sifive_u.c b/hw/riscv/sifive_u.c
index b434fd3..dc46f64 100644
--- a/hw/riscv/sifive_u.c
+++ b/hw/riscv/sifive_u.c
@@ -60,9 +60,9 @@
 #include <libfdt.h>
 
 #if defined(TARGET_RISCV32)
-# define BIOS_FILENAME "opensbi-riscv32-sifive_u-fw_jump.bin"
+# define BIOS_FILENAME "opensbi-riscv32-generic-fw_dynamic.bin"
 #else
-# define BIOS_FILENAME "opensbi-riscv64-sifive_u-fw_jump.bin"
+# define BIOS_FILENAME "opensbi-riscv64-generic-fw_dynamic.bin"
 #endif
 
 static const struct MemmapEntry {
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index 85e17fe..5ca49c5 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -43,9 +43,9 @@
 #include "hw/pci-host/gpex.h"
 
 #if defined(TARGET_RISCV32)
-# define BIOS_FILENAME "opensbi-riscv32-virt-fw_jump.bin"
+# define BIOS_FILENAME "opensbi-riscv32-generic-fw_dynamic.bin"
 #else
-# define BIOS_FILENAME "opensbi-riscv64-virt-fw_jump.bin"
+# define BIOS_FILENAME "opensbi-riscv64-generic-fw_dynamic.bin"
 #endif
 
 static const struct MemmapEntry {
diff --git a/pc-bios/opensbi-riscv32-generic-fw_dynamic.bin b/pc-bios/opensbi-riscv32-generic-fw_dynamic.bin
new file mode 100644
index 0000000..23b4dfb
Binary files /dev/null and b/pc-bios/opensbi-riscv32-generic-fw_dynamic.bin differ
diff --git a/pc-bios/opensbi-riscv32-sifive_u-fw_jump.bin b/pc-bios/opensbi-riscv32-sifive_u-fw_jump.bin
deleted file mode 100644
index 3e0da54..0000000
Binary files a/pc-bios/opensbi-riscv32-sifive_u-fw_jump.bin and /dev/null differ
diff --git a/pc-bios/opensbi-riscv32-virt-fw_jump.bin b/pc-bios/opensbi-riscv32-virt-fw_jump.bin
deleted file mode 100644
index bc56ed6..0000000
Binary files a/pc-bios/opensbi-riscv32-virt-fw_jump.bin and /dev/null differ
diff --git a/pc-bios/opensbi-riscv64-generic-fw_dynamic.bin b/pc-bios/opensbi-riscv64-generic-fw_dynamic.bin
new file mode 100644
index 0000000..16c0cf4
Binary files /dev/null and b/pc-bios/opensbi-riscv64-generic-fw_dynamic.bin differ
diff --git a/pc-bios/opensbi-riscv64-sifive_u-fw_jump.bin b/pc-bios/opensbi-riscv64-sifive_u-fw_jump.bin
deleted file mode 100644
index 1acee86..0000000
Binary files a/pc-bios/opensbi-riscv64-sifive_u-fw_jump.bin and /dev/null differ
diff --git a/pc-bios/opensbi-riscv64-virt-fw_jump.bin b/pc-bios/opensbi-riscv64-virt-fw_jump.bin
deleted file mode 100644
index c62f2b4..0000000
Binary files a/pc-bios/opensbi-riscv64-virt-fw_jump.bin and /dev/null differ
-- 
2.7.4

On Thu, Jul 9, 2020 at 10:07 PM Bin Meng <bmeng.cn@gmail.com> wrote:
>
> From: Bin Meng <bin.meng@windriver.com>
>
> Update virt and sifive_u machines to use the opensbi fw_dynamic bios
> image built for the generic FDT platform.
>
> Remove the out-of-date no longer used bios images.
>
> Signed-off-by: Bin Meng <bin.meng@windriver.com>
> Reviewed-by: Anup Patel <anup@brainfault.org>
> Reviewed-by: Alistair Francis <alistair.francis@wdc.com>

This patch seems to break 32-bit Linux boots on the sifive_u and virt machines.

Alistair

>
> ---
>
> Changes in v4:
> - Remove old binaries in the Makefile for `make install` bisection
>
> Changes in v3:
> - Change to fw_dynamic.bin for virt & sifive_u
>
>  Makefile                                       |   4 +---
>  hw/riscv/sifive_u.c                            |   4 ++--
>  hw/riscv/virt.c                                |   4 ++--
>  pc-bios/opensbi-riscv32-generic-fw_dynamic.bin | Bin 0 -> 62144 bytes
>  pc-bios/opensbi-riscv32-sifive_u-fw_jump.bin   | Bin 49520 -> 0 bytes
>  pc-bios/opensbi-riscv32-virt-fw_jump.bin       | Bin 49504 -> 0 bytes
>  pc-bios/opensbi-riscv64-generic-fw_dynamic.bin | Bin 0 -> 70792 bytes
>  pc-bios/opensbi-riscv64-sifive_u-fw_jump.bin   | Bin 57936 -> 0 bytes
>  pc-bios/opensbi-riscv64-virt-fw_jump.bin       | Bin 57920 -> 0 bytes
>  9 files changed, 5 insertions(+), 7 deletions(-)
>  create mode 100644 pc-bios/opensbi-riscv32-generic-fw_dynamic.bin
>  delete mode 100644 pc-bios/opensbi-riscv32-sifive_u-fw_jump.bin
>  delete mode 100644 pc-bios/opensbi-riscv32-virt-fw_jump.bin
>  create mode 100644 pc-bios/opensbi-riscv64-generic-fw_dynamic.bin
>  delete mode 100644 pc-bios/opensbi-riscv64-sifive_u-fw_jump.bin
>  delete mode 100644 pc-bios/opensbi-riscv64-virt-fw_jump.bin
>
> diff --git a/Makefile b/Makefile
> index b1b8a5a..f06b3ae 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -840,9 +840,7 @@ palcode-clipper \
>  u-boot.e500 u-boot-sam460-20100605.bin \
>  qemu_vga.ndrv \
>  edk2-licenses.txt \
> -hppa-firmware.img \
> -opensbi-riscv32-sifive_u-fw_jump.bin opensbi-riscv32-virt-fw_jump.bin \
> -opensbi-riscv64-sifive_u-fw_jump.bin opensbi-riscv64-virt-fw_jump.bin
> +hppa-firmware.img
>
>
>  DESCS=50-edk2-i386-secure.json 50-edk2-x86_64-secure.json \
> diff --git a/hw/riscv/sifive_u.c b/hw/riscv/sifive_u.c
> index b434fd3..dc46f64 100644
> --- a/hw/riscv/sifive_u.c
> +++ b/hw/riscv/sifive_u.c
> @@ -60,9 +60,9 @@
>  #include <libfdt.h>
>
>  #if defined(TARGET_RISCV32)
> -# define BIOS_FILENAME "opensbi-riscv32-sifive_u-fw_jump.bin"
> +# define BIOS_FILENAME "opensbi-riscv32-generic-fw_dynamic.bin"
>  #else
> -# define BIOS_FILENAME "opensbi-riscv64-sifive_u-fw_jump.bin"
> +# define BIOS_FILENAME "opensbi-riscv64-generic-fw_dynamic.bin"
>  #endif
>
>  static const struct MemmapEntry {
> diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
> index 85e17fe..5ca49c5 100644
> --- a/hw/riscv/virt.c
> +++ b/hw/riscv/virt.c
> @@ -43,9 +43,9 @@
>  #include "hw/pci-host/gpex.h"
>
>  #if defined(TARGET_RISCV32)
> -# define BIOS_FILENAME "opensbi-riscv32-virt-fw_jump.bin"
> +# define BIOS_FILENAME "opensbi-riscv32-generic-fw_dynamic.bin"
>  #else
> -# define BIOS_FILENAME "opensbi-riscv64-virt-fw_jump.bin"
> +# define BIOS_FILENAME "opensbi-riscv64-generic-fw_dynamic.bin"
>  #endif
>
>  static const struct MemmapEntry {
> diff --git a/pc-bios/opensbi-riscv32-generic-fw_dynamic.bin b/pc-bios/opensbi-riscv32-generic-fw_dynamic.bin
> new file mode 100644
> index 0000000..23b4dfb
> Binary files /dev/null and b/pc-bios/opensbi-riscv32-generic-fw_dynamic.bin differ
> diff --git a/pc-bios/opensbi-riscv32-sifive_u-fw_jump.bin b/pc-bios/opensbi-riscv32-sifive_u-fw_jump.bin
> deleted file mode 100644
> index 3e0da54..0000000
> Binary files a/pc-bios/opensbi-riscv32-sifive_u-fw_jump.bin and /dev/null differ
> diff --git a/pc-bios/opensbi-riscv32-virt-fw_jump.bin b/pc-bios/opensbi-riscv32-virt-fw_jump.bin
> deleted file mode 100644
> index bc56ed6..0000000
> Binary files a/pc-bios/opensbi-riscv32-virt-fw_jump.bin and /dev/null differ
> diff --git a/pc-bios/opensbi-riscv64-generic-fw_dynamic.bin b/pc-bios/opensbi-riscv64-generic-fw_dynamic.bin
> new file mode 100644
> index 0000000..16c0cf4
> Binary files /dev/null and b/pc-bios/opensbi-riscv64-generic-fw_dynamic.bin differ
> diff --git a/pc-bios/opensbi-riscv64-sifive_u-fw_jump.bin b/pc-bios/opensbi-riscv64-sifive_u-fw_jump.bin
> deleted file mode 100644
> index 1acee86..0000000
> Binary files a/pc-bios/opensbi-riscv64-sifive_u-fw_jump.bin and /dev/null differ
> diff --git a/pc-bios/opensbi-riscv64-virt-fw_jump.bin b/pc-bios/opensbi-riscv64-virt-fw_jump.bin
> deleted file mode 100644
> index c62f2b4..0000000
> Binary files a/pc-bios/opensbi-riscv64-virt-fw_jump.bin and /dev/null differ
> --
> 2.7.4
>
>

On Sun, Jul 12, 2020 at 1:34 AM Alistair Francis <alistair23@gmail.com> wrote:
>
> On Thu, Jul 9, 2020 at 10:07 PM Bin Meng <bmeng.cn@gmail.com> wrote:
> >
> > From: Bin Meng <bin.meng@windriver.com>
> >
> > Update virt and sifive_u machines to use the opensbi fw_dynamic bios
> > image built for the generic FDT platform.
> >
> > Remove the out-of-date no longer used bios images.
> >
> > Signed-off-by: Bin Meng <bin.meng@windriver.com>
> > Reviewed-by: Anup Patel <anup@brainfault.org>
> > Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
>
> This patch seems to break 32-bit Linux boots on the sifive_u and virt machines.
>

It looks only Linux boot on sifive_u is broken. On our side, we have
been using VxWorks to test 32-bit OpenSBI on sifive_u so this issue
gets unnoticed. I will take a look.

Regards,
Bin

Hi Alistair,

On Mon, Jul 13, 2020 at 9:53 AM Bin Meng <bmeng.cn@gmail.com> wrote:
>
> On Sun, Jul 12, 2020 at 1:34 AM Alistair Francis <alistair23@gmail.com> wrote:
> >
> > On Thu, Jul 9, 2020 at 10:07 PM Bin Meng <bmeng.cn@gmail.com> wrote:
> > >
> > > From: Bin Meng <bin.meng@windriver.com>
> > >
> > > Update virt and sifive_u machines to use the opensbi fw_dynamic bios
> > > image built for the generic FDT platform.
> > >
> > > Remove the out-of-date no longer used bios images.
> > >
> > > Signed-off-by: Bin Meng <bin.meng@windriver.com>
> > > Reviewed-by: Anup Patel <anup@brainfault.org>
> > > Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
> >
> > This patch seems to break 32-bit Linux boots on the sifive_u and virt machines.
> >
>
> It looks only Linux boot on sifive_u is broken. On our side, we have
> been using VxWorks to test 32-bit OpenSBI on sifive_u so this issue
> gets unnoticed. I will take a look.

I've figured out the issue of 32-bit Linux booting failure on
sifive_u. A patch has been sent to Linux upstream:
http://lists.infradead.org/pipermail/linux-riscv/2020-July/001213.html

Regards,
Bin

On Wed, Jul 15, 2020 at 9:55 PM Bin Meng <bmeng.cn@gmail.com> wrote:
>
> Hi Alistair,
>
> On Mon, Jul 13, 2020 at 9:53 AM Bin Meng <bmeng.cn@gmail.com> wrote:
> >
> > On Sun, Jul 12, 2020 at 1:34 AM Alistair Francis <alistair23@gmail.com> wrote:
> > >
> > > On Thu, Jul 9, 2020 at 10:07 PM Bin Meng <bmeng.cn@gmail.com> wrote:
> > > >
> > > > From: Bin Meng <bin.meng@windriver.com>
> > > >
> > > > Update virt and sifive_u machines to use the opensbi fw_dynamic bios
> > > > image built for the generic FDT platform.
> > > >
> > > > Remove the out-of-date no longer used bios images.
> > > >
> > > > Signed-off-by: Bin Meng <bin.meng@windriver.com>
> > > > Reviewed-by: Anup Patel <anup@brainfault.org>
> > > > Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
> > >
> > > This patch seems to break 32-bit Linux boots on the sifive_u and virt machines.
> > >
> >
> > It looks only Linux boot on sifive_u is broken. On our side, we have
> > been using VxWorks to test 32-bit OpenSBI on sifive_u so this issue
> > gets unnoticed. I will take a look.
>
> I've figured out the issue of 32-bit Linux booting failure on
> sifive_u. A patch has been sent to Linux upstream:
> http://lists.infradead.org/pipermail/linux-riscv/2020-July/001213.html

Thanks for that. What change in QEMU causes this failure though?

There are lots of people not running the latest Linux from master that
this will cause breakages for.

Alistair

>
> Regards,
> Bin

Hi Alistair,

On Tue, Jul 28, 2020 at 11:39 PM Alistair Francis <alistair23@gmail.com> wrote:
>
> On Wed, Jul 15, 2020 at 9:55 PM Bin Meng <bmeng.cn@gmail.com> wrote:
> >
> > Hi Alistair,
> >
> > On Mon, Jul 13, 2020 at 9:53 AM Bin Meng <bmeng.cn@gmail.com> wrote:
> > >
> > > On Sun, Jul 12, 2020 at 1:34 AM Alistair Francis <alistair23@gmail.com> wrote:
> > > >
> > > > On Thu, Jul 9, 2020 at 10:07 PM Bin Meng <bmeng.cn@gmail.com> wrote:
> > > > >
> > > > > From: Bin Meng <bin.meng@windriver.com>
> > > > >
> > > > > Update virt and sifive_u machines to use the opensbi fw_dynamic bios
> > > > > image built for the generic FDT platform.
> > > > >
> > > > > Remove the out-of-date no longer used bios images.
> > > > >
> > > > > Signed-off-by: Bin Meng <bin.meng@windriver.com>
> > > > > Reviewed-by: Anup Patel <anup@brainfault.org>
> > > > > Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
> > > >
> > > > This patch seems to break 32-bit Linux boots on the sifive_u and virt machines.
> > > >
> > >
> > > It looks only Linux boot on sifive_u is broken. On our side, we have
> > > been using VxWorks to test 32-bit OpenSBI on sifive_u so this issue
> > > gets unnoticed. I will take a look.
> >
> > I've figured out the issue of 32-bit Linux booting failure on
> > sifive_u. A patch has been sent to Linux upstream:
> > http://lists.infradead.org/pipermail/linux-riscv/2020-July/001213.html
>
> Thanks for that. What change in QEMU causes this failure though?
>

There is nothing wrong in QEMU.

> There are lots of people not running the latest Linux from master that
> this will cause breakages for.

It's just that the 32-bit Linux defconfig has never been validated by
people with 'sifive_u' machine. I bet people only validated the 32-bit
kernel with the 'virt' machine.

Regards,
Bin

On Tue, Jul 28, 2020 at 8:46 AM Bin Meng <bmeng.cn@gmail.com> wrote:
>
> Hi Alistair,
>
> On Tue, Jul 28, 2020 at 11:39 PM Alistair Francis <alistair23@gmail.com> wrote:
> >
> > On Wed, Jul 15, 2020 at 9:55 PM Bin Meng <bmeng.cn@gmail.com> wrote:
> > >
> > > Hi Alistair,
> > >
> > > On Mon, Jul 13, 2020 at 9:53 AM Bin Meng <bmeng.cn@gmail.com> wrote:
> > > >
> > > > On Sun, Jul 12, 2020 at 1:34 AM Alistair Francis <alistair23@gmail.com> wrote:
> > > > >
> > > > > On Thu, Jul 9, 2020 at 10:07 PM Bin Meng <bmeng.cn@gmail.com> wrote:
> > > > > >
> > > > > > From: Bin Meng <bin.meng@windriver.com>
> > > > > >
> > > > > > Update virt and sifive_u machines to use the opensbi fw_dynamic bios
> > > > > > image built for the generic FDT platform.
> > > > > >
> > > > > > Remove the out-of-date no longer used bios images.
> > > > > >
> > > > > > Signed-off-by: Bin Meng <bin.meng@windriver.com>
> > > > > > Reviewed-by: Anup Patel <anup@brainfault.org>
> > > > > > Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
> > > > >
> > > > > This patch seems to break 32-bit Linux boots on the sifive_u and virt machines.
> > > > >
> > > >
> > > > It looks only Linux boot on sifive_u is broken. On our side, we have
> > > > been using VxWorks to test 32-bit OpenSBI on sifive_u so this issue
> > > > gets unnoticed. I will take a look.
> > >
> > > I've figured out the issue of 32-bit Linux booting failure on
> > > sifive_u. A patch has been sent to Linux upstream:
> > > http://lists.infradead.org/pipermail/linux-riscv/2020-July/001213.html
> >
> > Thanks for that. What change in QEMU causes this failure though?
> >
>
> There is nothing wrong in QEMU.

There is. This patch causes a regression for 32-bit Linux boot on the
sifive_u. Your v5 has not addressed this.

With this patch, the Linux boot stops here:

OpenSBI v0.8
   ____                    _____ ____ _____
  / __ \                  / ____|  _ \_   _|
 | |  | |_ __   ___ _ __ | (___ | |_) || |
 | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
 | |__| | |_) |  __/ | | |____) | |_) || |_
  \____/| .__/ \___|_| |_|_____/|____/_____|
        | |
        |_|

Platform Name       : SiFive HiFive Unleashed A00
Platform Features   : timer,mfdeleg
Platform HART Count : 4
Boot HART ID        : 3
Boot HART ISA       : rv64imafdcsu
BOOT HART Features  : pmp,scounteren,mcounteren
BOOT HART PMP Count : 16
Firmware Base       : 0x80000000
Firmware Size       : 116 KB
Runtime SBI Version : 0.2

MIDELEG : 0x0000000000000222
MEDELEG : 0x000000000000b109
PMP0    : 0x0000000080000000-0x000000008001ffff (A)
PMP1    : 0x0000000000000000-0xffffffffffffffff (A,R,W,X)
[    0.000000] OF: fdt: Ignoring memory range 0x80000000 - 0x80200000
[    0.000000] Linux version 5.3.0 (oe-user@oe-host) (gcc version
9.2.0 (GCC)) #1 SMP Thu Sep 19 18:34:52 UTC 2019
[    0.000000] earlycon: sbi0 at I/O port 0x0 (options '')
[    0.000000] printk: bootconsole [sbi0] enabled
[    0.000000] initrd not found or empty - disabling initrd
[    0.000000] Zone ranges:
[    0.000000]   DMA32    [mem 0x0000000080200000-0x00000000bfffffff]
[    0.000000]   Normal   empty
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000080200000-0x00000000bfffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000080200000-0x00000000bfffffff]
[    0.000000] OF: fdt: Invalid device tree blob header
[    0.000000] software IO TLB: mapped [mem 0xbb1fe000-0xbf1fe000] (64MB)

Without this patch I can boot all the way to looking for a rootFS.

Please don't send new versions of patches without addresses regressions.

Alistair

>
> > There are lots of people not running the latest Linux from master that
> > this will cause breakages for.
>
> It's just that the 32-bit Linux defconfig has never been validated by
> people with 'sifive_u' machine. I bet people only validated the 32-bit
> kernel with the 'virt' machine.
>
> Regards,
> Bin

Hi Alistair,

On Wed, Jul 29, 2020 at 2:26 AM Alistair Francis <alistair23@gmail.com> wrote:
>
> On Tue, Jul 28, 2020 at 8:46 AM Bin Meng <bmeng.cn@gmail.com> wrote:
> >
> > Hi Alistair,
> >
> > On Tue, Jul 28, 2020 at 11:39 PM Alistair Francis <alistair23@gmail.com> wrote:
> > >
> > > On Wed, Jul 15, 2020 at 9:55 PM Bin Meng <bmeng.cn@gmail.com> wrote:
> > > >
> > > > Hi Alistair,
> > > >
> > > > On Mon, Jul 13, 2020 at 9:53 AM Bin Meng <bmeng.cn@gmail.com> wrote:
> > > > >
> > > > > On Sun, Jul 12, 2020 at 1:34 AM Alistair Francis <alistair23@gmail.com> wrote:
> > > > > >
> > > > > > On Thu, Jul 9, 2020 at 10:07 PM Bin Meng <bmeng.cn@gmail.com> wrote:
> > > > > > >
> > > > > > > From: Bin Meng <bin.meng@windriver.com>
> > > > > > >
> > > > > > > Update virt and sifive_u machines to use the opensbi fw_dynamic bios
> > > > > > > image built for the generic FDT platform.
> > > > > > >
> > > > > > > Remove the out-of-date no longer used bios images.
> > > > > > >
> > > > > > > Signed-off-by: Bin Meng <bin.meng@windriver.com>
> > > > > > > Reviewed-by: Anup Patel <anup@brainfault.org>
> > > > > > > Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
> > > > > >
> > > > > > This patch seems to break 32-bit Linux boots on the sifive_u and virt machines.
> > > > > >
> > > > >
> > > > > It looks only Linux boot on sifive_u is broken. On our side, we have
> > > > > been using VxWorks to test 32-bit OpenSBI on sifive_u so this issue
> > > > > gets unnoticed. I will take a look.
> > > >
> > > > I've figured out the issue of 32-bit Linux booting failure on
> > > > sifive_u. A patch has been sent to Linux upstream:
> > > > http://lists.infradead.org/pipermail/linux-riscv/2020-July/001213.html
> > >
> > > Thanks for that. What change in QEMU causes this failure though?
> > >
> >
> > There is nothing wrong in QEMU.
>
> There is. This patch causes a regression for 32-bit Linux boot on the
> sifive_u. Your v5 has not addressed this.

The 32-bit Linux boot failure was fixed by:
http://lists.infradead.org/pipermail/linux-riscv/2020-July/001213.html

What additional issue did you see?

>
> With this patch, the Linux boot stops here:
>
> OpenSBI v0.8
>    ____                    _____ ____ _____
>   / __ \                  / ____|  _ \_   _|
>  | |  | |_ __   ___ _ __ | (___ | |_) || |
>  | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
>  | |__| | |_) |  __/ | | |____) | |_) || |_
>   \____/| .__/ \___|_| |_|_____/|____/_____|
>         | |
>         |_|
>
> Platform Name       : SiFive HiFive Unleashed A00
> Platform Features   : timer,mfdeleg
> Platform HART Count : 4
> Boot HART ID        : 3
> Boot HART ISA       : rv64imafdcsu

This is a 64-bit hardware.

> BOOT HART Features  : pmp,scounteren,mcounteren
> BOOT HART PMP Count : 16
> Firmware Base       : 0x80000000
> Firmware Size       : 116 KB
> Runtime SBI Version : 0.2
>
> MIDELEG : 0x0000000000000222
> MEDELEG : 0x000000000000b109
> PMP0    : 0x0000000080000000-0x000000008001ffff (A)
> PMP1    : 0x0000000000000000-0xffffffffffffffff (A,R,W,X)
> [    0.000000] OF: fdt: Ignoring memory range 0x80000000 - 0x80200000
> [    0.000000] Linux version 5.3.0 (oe-user@oe-host) (gcc version

It seems that you are using quite an old kernel. Can you please try
the latest version?

> 9.2.0 (GCC)) #1 SMP Thu Sep 19 18:34:52 UTC 2019
> [    0.000000] earlycon: sbi0 at I/O port 0x0 (options '')
> [    0.000000] printk: bootconsole [sbi0] enabled
> [    0.000000] initrd not found or empty - disabling initrd
> [    0.000000] Zone ranges:
> [    0.000000]   DMA32    [mem 0x0000000080200000-0x00000000bfffffff]
> [    0.000000]   Normal   empty
> [    0.000000] Movable zone start for each node
> [    0.000000] Early memory node ranges
> [    0.000000]   node   0: [mem 0x0000000080200000-0x00000000bfffffff]
> [    0.000000] Initmem setup node 0 [mem 0x0000000080200000-0x00000000bfffffff]
> [    0.000000] OF: fdt: Invalid device tree blob header
> [    0.000000] software IO TLB: mapped [mem 0xbb1fe000-0xbf1fe000] (64MB)
>
> Without this patch I can boot all the way to looking for a rootFS.
>
> Please don't send new versions of patches without addresses regressions.

The patches were sent after addressing all regressions you reported
(well the 32-bit Linux booting issue is actually not a QEMU
regression, but one that exists in the Linux kernel side for a long
time).

I just tested 64-bit Linux boot on both virt and sifive_u, and they
both can boot all the way to looking for a root fs.

Regards,
Bin

On Tue, Jul 28, 2020 at 9:51 PM Bin Meng <bmeng.cn@gmail.com> wrote:
>
> Hi Alistair,
>
> On Wed, Jul 29, 2020 at 2:26 AM Alistair Francis <alistair23@gmail.com> wrote:
> >
> > On Tue, Jul 28, 2020 at 8:46 AM Bin Meng <bmeng.cn@gmail.com> wrote:
> > >
> > > Hi Alistair,
> > >
> > > On Tue, Jul 28, 2020 at 11:39 PM Alistair Francis <alistair23@gmail.com> wrote:
> > > >
> > > > On Wed, Jul 15, 2020 at 9:55 PM Bin Meng <bmeng.cn@gmail.com> wrote:
> > > > >
> > > > > Hi Alistair,
> > > > >
> > > > > On Mon, Jul 13, 2020 at 9:53 AM Bin Meng <bmeng.cn@gmail.com> wrote:
> > > > > >
> > > > > > On Sun, Jul 12, 2020 at 1:34 AM Alistair Francis <alistair23@gmail.com> wrote:
> > > > > > >
> > > > > > > On Thu, Jul 9, 2020 at 10:07 PM Bin Meng <bmeng.cn@gmail.com> wrote:
> > > > > > > >
> > > > > > > > From: Bin Meng <bin.meng@windriver.com>
> > > > > > > >
> > > > > > > > Update virt and sifive_u machines to use the opensbi fw_dynamic bios
> > > > > > > > image built for the generic FDT platform.
> > > > > > > >
> > > > > > > > Remove the out-of-date no longer used bios images.
> > > > > > > >
> > > > > > > > Signed-off-by: Bin Meng <bin.meng@windriver.com>
> > > > > > > > Reviewed-by: Anup Patel <anup@brainfault.org>
> > > > > > > > Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
> > > > > > >
> > > > > > > This patch seems to break 32-bit Linux boots on the sifive_u and virt machines.
> > > > > > >
> > > > > >
> > > > > > It looks only Linux boot on sifive_u is broken. On our side, we have
> > > > > > been using VxWorks to test 32-bit OpenSBI on sifive_u so this issue
> > > > > > gets unnoticed. I will take a look.
> > > > >
> > > > > I've figured out the issue of 32-bit Linux booting failure on
> > > > > sifive_u. A patch has been sent to Linux upstream:
> > > > > http://lists.infradead.org/pipermail/linux-riscv/2020-July/001213.html
> > > >
> > > > Thanks for that. What change in QEMU causes this failure though?
> > > >
> > >
> > > There is nothing wrong in QEMU.
> >
> > There is. This patch causes a regression for 32-bit Linux boot on the
> > sifive_u. Your v5 has not addressed this.
>
> The 32-bit Linux boot failure was fixed by:
> http://lists.infradead.org/pipermail/linux-riscv/2020-July/001213.html
>
> What additional issue did you see?
>
> >
> > With this patch, the Linux boot stops here:
> >
> > OpenSBI v0.8
> >    ____                    _____ ____ _____
> >   / __ \                  / ____|  _ \_   _|
> >  | |  | |_ __   ___ _ __ | (___ | |_) || |
> >  | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
> >  | |__| | |_) |  __/ | | |____) | |_) || |_
> >   \____/| .__/ \___|_| |_|_____/|____/_____|
> >         | |
> >         |_|
> >
> > Platform Name       : SiFive HiFive Unleashed A00
> > Platform Features   : timer,mfdeleg
> > Platform HART Count : 4
> > Boot HART ID        : 3
> > Boot HART ISA       : rv64imafdcsu
>
> This is a 64-bit hardware.

You are right. It's not 32-bit, that was my mistake. I'm used to my
first test being 32-bit, but in this case it's not.

It looks like this commit instead breaks the sifive_u for 64-bit with
the 5.3 kernel.

>
> > BOOT HART Features  : pmp,scounteren,mcounteren
> > BOOT HART PMP Count : 16
> > Firmware Base       : 0x80000000
> > Firmware Size       : 116 KB
> > Runtime SBI Version : 0.2
> >
> > MIDELEG : 0x0000000000000222
> > MEDELEG : 0x000000000000b109
> > PMP0    : 0x0000000080000000-0x000000008001ffff (A)
> > PMP1    : 0x0000000000000000-0xffffffffffffffff (A,R,W,X)
> > [    0.000000] OF: fdt: Ignoring memory range 0x80000000 - 0x80200000
> > [    0.000000] Linux version 5.3.0 (oe-user@oe-host) (gcc version
>
> It seems that you are using quite an old kernel. Can you please try
> the latest version?

It is an old kernel, but old kernels should still keep working (or we
should at least know why they don't)

>
> > 9.2.0 (GCC)) #1 SMP Thu Sep 19 18:34:52 UTC 2019
> > [    0.000000] earlycon: sbi0 at I/O port 0x0 (options '')
> > [    0.000000] printk: bootconsole [sbi0] enabled
> > [    0.000000] initrd not found or empty - disabling initrd
> > [    0.000000] Zone ranges:
> > [    0.000000]   DMA32    [mem 0x0000000080200000-0x00000000bfffffff]
> > [    0.000000]   Normal   empty
> > [    0.000000] Movable zone start for each node
> > [    0.000000] Early memory node ranges
> > [    0.000000]   node   0: [mem 0x0000000080200000-0x00000000bfffffff]
> > [    0.000000] Initmem setup node 0 [mem 0x0000000080200000-0x00000000bfffffff]
> > [    0.000000] OF: fdt: Invalid device tree blob header
> > [    0.000000] software IO TLB: mapped [mem 0xbb1fe000-0xbf1fe000] (64MB)
> >
> > Without this patch I can boot all the way to looking for a rootFS.
> >
> > Please don't send new versions of patches without addresses regressions.
>
> The patches were sent after addressing all regressions you reported
> (well the 32-bit Linux booting issue is actually not a QEMU
> regression, but one that exists in the Linux kernel side for a long
> time).

Yep, that is my mistake. Sorry about the confusion.

>
> I just tested 64-bit Linux boot on both virt and sifive_u, and they
> both can boot all the way to looking for a root fs.

Can you test with older kernels?

If we can't support older kernels with the default bios option we at
least need to know why and list that in the release notes.

Alistair

>
> Regards,
> Bin

Hi Alistair,

On Wed, Jul 29, 2020 at 1:05 PM Alistair Francis <alistair23@gmail.com> wrote:
>
> On Tue, Jul 28, 2020 at 9:51 PM Bin Meng <bmeng.cn@gmail.com> wrote:
> >
> > Hi Alistair,
> >
> > On Wed, Jul 29, 2020 at 2:26 AM Alistair Francis <alistair23@gmail.com> wrote:
> > >
> > > On Tue, Jul 28, 2020 at 8:46 AM Bin Meng <bmeng.cn@gmail.com> wrote:
> > > >
> > > > Hi Alistair,
> > > >
> > > > On Tue, Jul 28, 2020 at 11:39 PM Alistair Francis <alistair23@gmail.com> wrote:
> > > > >
> > > > > On Wed, Jul 15, 2020 at 9:55 PM Bin Meng <bmeng.cn@gmail.com> wrote:
> > > > > >
> > > > > > Hi Alistair,
> > > > > >
> > > > > > On Mon, Jul 13, 2020 at 9:53 AM Bin Meng <bmeng.cn@gmail.com> wrote:
> > > > > > >
> > > > > > > On Sun, Jul 12, 2020 at 1:34 AM Alistair Francis <alistair23@gmail.com> wrote:
> > > > > > > >
> > > > > > > > On Thu, Jul 9, 2020 at 10:07 PM Bin Meng <bmeng.cn@gmail.com> wrote:
> > > > > > > > >
> > > > > > > > > From: Bin Meng <bin.meng@windriver.com>
> > > > > > > > >
> > > > > > > > > Update virt and sifive_u machines to use the opensbi fw_dynamic bios
> > > > > > > > > image built for the generic FDT platform.
> > > > > > > > >
> > > > > > > > > Remove the out-of-date no longer used bios images.
> > > > > > > > >
> > > > > > > > > Signed-off-by: Bin Meng <bin.meng@windriver.com>
> > > > > > > > > Reviewed-by: Anup Patel <anup@brainfault.org>
> > > > > > > > > Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
> > > > > > > >
> > > > > > > > This patch seems to break 32-bit Linux boots on the sifive_u and virt machines.
> > > > > > > >
> > > > > > >
> > > > > > > It looks only Linux boot on sifive_u is broken. On our side, we have
> > > > > > > been using VxWorks to test 32-bit OpenSBI on sifive_u so this issue
> > > > > > > gets unnoticed. I will take a look.
> > > > > >
> > > > > > I've figured out the issue of 32-bit Linux booting failure on
> > > > > > sifive_u. A patch has been sent to Linux upstream:
> > > > > > http://lists.infradead.org/pipermail/linux-riscv/2020-July/001213.html
> > > > >
> > > > > Thanks for that. What change in QEMU causes this failure though?
> > > > >
> > > >
> > > > There is nothing wrong in QEMU.
> > >
> > > There is. This patch causes a regression for 32-bit Linux boot on the
> > > sifive_u. Your v5 has not addressed this.
> >
> > The 32-bit Linux boot failure was fixed by:
> > http://lists.infradead.org/pipermail/linux-riscv/2020-July/001213.html
> >
> > What additional issue did you see?
> >
> > >
> > > With this patch, the Linux boot stops here:
> > >
> > > OpenSBI v0.8
> > >    ____                    _____ ____ _____
> > >   / __ \                  / ____|  _ \_   _|
> > >  | |  | |_ __   ___ _ __ | (___ | |_) || |
> > >  | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
> > >  | |__| | |_) |  __/ | | |____) | |_) || |_
> > >   \____/| .__/ \___|_| |_|_____/|____/_____|
> > >         | |
> > >         |_|
> > >
> > > Platform Name       : SiFive HiFive Unleashed A00
> > > Platform Features   : timer,mfdeleg
> > > Platform HART Count : 4
> > > Boot HART ID        : 3
> > > Boot HART ISA       : rv64imafdcsu
> >
> > This is a 64-bit hardware.
>
> You are right. It's not 32-bit, that was my mistake. I'm used to my
> first test being 32-bit, but in this case it's not.
>
> It looks like this commit instead breaks the sifive_u for 64-bit with
> the 5.3 kernel.
>
> >
> > > BOOT HART Features  : pmp,scounteren,mcounteren
> > > BOOT HART PMP Count : 16
> > > Firmware Base       : 0x80000000
> > > Firmware Size       : 116 KB
> > > Runtime SBI Version : 0.2
> > >
> > > MIDELEG : 0x0000000000000222
> > > MEDELEG : 0x000000000000b109
> > > PMP0    : 0x0000000080000000-0x000000008001ffff (A)
> > > PMP1    : 0x0000000000000000-0xffffffffffffffff (A,R,W,X)
> > > [    0.000000] OF: fdt: Ignoring memory range 0x80000000 - 0x80200000
> > > [    0.000000] Linux version 5.3.0 (oe-user@oe-host) (gcc version
> >
> > It seems that you are using quite an old kernel. Can you please try
> > the latest version?
>
> It is an old kernel, but old kernels should still keep working (or we
> should at least know why they don't)
>
> >
> > > 9.2.0 (GCC)) #1 SMP Thu Sep 19 18:34:52 UTC 2019
> > > [    0.000000] earlycon: sbi0 at I/O port 0x0 (options '')
> > > [    0.000000] printk: bootconsole [sbi0] enabled
> > > [    0.000000] initrd not found or empty - disabling initrd
> > > [    0.000000] Zone ranges:
> > > [    0.000000]   DMA32    [mem 0x0000000080200000-0x00000000bfffffff]
> > > [    0.000000]   Normal   empty
> > > [    0.000000] Movable zone start for each node
> > > [    0.000000] Early memory node ranges
> > > [    0.000000]   node   0: [mem 0x0000000080200000-0x00000000bfffffff]
> > > [    0.000000] Initmem setup node 0 [mem 0x0000000080200000-0x00000000bfffffff]
> > > [    0.000000] OF: fdt: Invalid device tree blob header
> > > [    0.000000] software IO TLB: mapped [mem 0xbb1fe000-0xbf1fe000] (64MB)
> > >
> > > Without this patch I can boot all the way to looking for a rootFS.
> > >
> > > Please don't send new versions of patches without addresses regressions.
> >
> > The patches were sent after addressing all regressions you reported
> > (well the 32-bit Linux booting issue is actually not a QEMU
> > regression, but one that exists in the Linux kernel side for a long
> > time).
>
> Yep, that is my mistake. Sorry about the confusion.
>
> >
> > I just tested 64-bit Linux boot on both virt and sifive_u, and they
> > both can boot all the way to looking for a root fs.
>
> Can you test with older kernels?
>

OK I will investigate.

> If we can't support older kernels with the default bios option we at
> least need to know why and list that in the release notes.
>

Regards,
Bin

Hi Alistair,

On Wed, Jul 29, 2020 at 1:10 PM Bin Meng <bmeng.cn@gmail.com> wrote:
>
> Hi Alistair,
>
> On Wed, Jul 29, 2020 at 1:05 PM Alistair Francis <alistair23@gmail.com> wrote:
> >
> > On Tue, Jul 28, 2020 at 9:51 PM Bin Meng <bmeng.cn@gmail.com> wrote:
> > >
> > > Hi Alistair,
> > >
> > > On Wed, Jul 29, 2020 at 2:26 AM Alistair Francis <alistair23@gmail.com> wrote:
> > > >
> > > > On Tue, Jul 28, 2020 at 8:46 AM Bin Meng <bmeng.cn@gmail.com> wrote:
> > > > >
> > > > > Hi Alistair,
> > > > >
> > > > > On Tue, Jul 28, 2020 at 11:39 PM Alistair Francis <alistair23@gmail.com> wrote:
> > > > > >
> > > > > > On Wed, Jul 15, 2020 at 9:55 PM Bin Meng <bmeng.cn@gmail.com> wrote:
> > > > > > >
> > > > > > > Hi Alistair,
> > > > > > >
> > > > > > > On Mon, Jul 13, 2020 at 9:53 AM Bin Meng <bmeng.cn@gmail.com> wrote:
> > > > > > > >
> > > > > > > > On Sun, Jul 12, 2020 at 1:34 AM Alistair Francis <alistair23@gmail.com> wrote:
> > > > > > > > >
> > > > > > > > > On Thu, Jul 9, 2020 at 10:07 PM Bin Meng <bmeng.cn@gmail.com> wrote:
> > > > > > > > > >
> > > > > > > > > > From: Bin Meng <bin.meng@windriver.com>
> > > > > > > > > >
> > > > > > > > > > Update virt and sifive_u machines to use the opensbi fw_dynamic bios
> > > > > > > > > > image built for the generic FDT platform.
> > > > > > > > > >
> > > > > > > > > > Remove the out-of-date no longer used bios images.
> > > > > > > > > >
> > > > > > > > > > Signed-off-by: Bin Meng <bin.meng@windriver.com>
> > > > > > > > > > Reviewed-by: Anup Patel <anup@brainfault.org>
> > > > > > > > > > Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
> > > > > > > > >
> > > > > > > > > This patch seems to break 32-bit Linux boots on the sifive_u and virt machines.
> > > > > > > > >
> > > > > > > >
> > > > > > > > It looks only Linux boot on sifive_u is broken. On our side, we have
> > > > > > > > been using VxWorks to test 32-bit OpenSBI on sifive_u so this issue
> > > > > > > > gets unnoticed. I will take a look.
> > > > > > >
> > > > > > > I've figured out the issue of 32-bit Linux booting failure on
> > > > > > > sifive_u. A patch has been sent to Linux upstream:
> > > > > > > http://lists.infradead.org/pipermail/linux-riscv/2020-July/001213.html
> > > > > >
> > > > > > Thanks for that. What change in QEMU causes this failure though?
> > > > > >
> > > > >
> > > > > There is nothing wrong in QEMU.
> > > >
> > > > There is. This patch causes a regression for 32-bit Linux boot on the
> > > > sifive_u. Your v5 has not addressed this.
> > >
> > > The 32-bit Linux boot failure was fixed by:
> > > http://lists.infradead.org/pipermail/linux-riscv/2020-July/001213.html
> > >
> > > What additional issue did you see?
> > >
> > > >
> > > > With this patch, the Linux boot stops here:
> > > >
> > > > OpenSBI v0.8
> > > >    ____                    _____ ____ _____
> > > >   / __ \                  / ____|  _ \_   _|
> > > >  | |  | |_ __   ___ _ __ | (___ | |_) || |
> > > >  | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
> > > >  | |__| | |_) |  __/ | | |____) | |_) || |_
> > > >   \____/| .__/ \___|_| |_|_____/|____/_____|
> > > >         | |
> > > >         |_|
> > > >
> > > > Platform Name       : SiFive HiFive Unleashed A00
> > > > Platform Features   : timer,mfdeleg
> > > > Platform HART Count : 4
> > > > Boot HART ID        : 3
> > > > Boot HART ISA       : rv64imafdcsu
> > >
> > > This is a 64-bit hardware.
> >
> > You are right. It's not 32-bit, that was my mistake. I'm used to my
> > first test being 32-bit, but in this case it's not.
> >
> > It looks like this commit instead breaks the sifive_u for 64-bit with
> > the 5.3 kernel.
> >
> > >
> > > > BOOT HART Features  : pmp,scounteren,mcounteren
> > > > BOOT HART PMP Count : 16
> > > > Firmware Base       : 0x80000000
> > > > Firmware Size       : 116 KB
> > > > Runtime SBI Version : 0.2
> > > >
> > > > MIDELEG : 0x0000000000000222
> > > > MEDELEG : 0x000000000000b109
> > > > PMP0    : 0x0000000080000000-0x000000008001ffff (A)
> > > > PMP1    : 0x0000000000000000-0xffffffffffffffff (A,R,W,X)
> > > > [    0.000000] OF: fdt: Ignoring memory range 0x80000000 - 0x80200000
> > > > [    0.000000] Linux version 5.3.0 (oe-user@oe-host) (gcc version
> > >
> > > It seems that you are using quite an old kernel. Can you please try
> > > the latest version?
> >
> > It is an old kernel, but old kernels should still keep working (or we
> > should at least know why they don't)
> >
> > >
> > > > 9.2.0 (GCC)) #1 SMP Thu Sep 19 18:34:52 UTC 2019
> > > > [    0.000000] earlycon: sbi0 at I/O port 0x0 (options '')
> > > > [    0.000000] printk: bootconsole [sbi0] enabled
> > > > [    0.000000] initrd not found or empty - disabling initrd
> > > > [    0.000000] Zone ranges:
> > > > [    0.000000]   DMA32    [mem 0x0000000080200000-0x00000000bfffffff]
> > > > [    0.000000]   Normal   empty
> > > > [    0.000000] Movable zone start for each node
> > > > [    0.000000] Early memory node ranges
> > > > [    0.000000]   node   0: [mem 0x0000000080200000-0x00000000bfffffff]
> > > > [    0.000000] Initmem setup node 0 [mem 0x0000000080200000-0x00000000bfffffff]
> > > > [    0.000000] OF: fdt: Invalid device tree blob header
> > > > [    0.000000] software IO TLB: mapped [mem 0xbb1fe000-0xbf1fe000] (64MB)
> > > >
> > > > Without this patch I can boot all the way to looking for a rootFS.
> > > >
> > > > Please don't send new versions of patches without addresses regressions.
> > >
> > > The patches were sent after addressing all regressions you reported
> > > (well the 32-bit Linux booting issue is actually not a QEMU
> > > regression, but one that exists in the Linux kernel side for a long
> > > time).
> >
> > Yep, that is my mistake. Sorry about the confusion.
> >
> > >
> > > I just tested 64-bit Linux boot on both virt and sifive_u, and they
> > > both can boot all the way to looking for a root fs.
> >
> > Can you test with older kernels?
> >
>
> OK I will investigate.

I've located the commit in the newer Linux kernel that fixed the boot
failure of the 5.3 kernel. See below:

commit 922b0375fc93fb1a20c5617e37c389c26bbccb70
Author: Albert Ou <aou@eecs.berkeley.edu>
Date:   Fri Sep 27 16:14:18 2019 -0700

    riscv: Fix memblock reservation for device tree blob

    This fixes an error with how the FDT blob is reserved in memblock.
    An incorrect physical address calculation exposed the FDT header to
    unintended corruption, which typically manifested with of_fdt_raw_init()
    faulting during late boot after fdt_totalsize() returned a wrong value.
    Systems with smaller physical memory sizes more frequently trigger this
    issue, as the kernel is more likely to allocate from the DMA32 zone
    where bbl places the DTB after the kernel image.

    Commit 671f9a3e2e24 ("RISC-V: Setup initial page tables in two stages")
    changed the mapping of the DTB to reside in the fixmap area.
    Consequently, early_init_fdt_reserve_self() cannot be used anymore in
    setup_bootmem() since it relies on __pa() to derive a physical address,
    which does not work with dtb_early_va that is no longer a valid kernel
    logical address.

    The reserved[0x1] region shows the effect of the pointer underflow
    resulting from the __pa(initial_boot_params) offset subtraction:

    [    0.000000] MEMBLOCK configuration:
    [    0.000000]  memory size = 0x000000001fe00000 reserved size =
0x0000000000a2e514
    [    0.000000]  memory.cnt  = 0x1
    [    0.000000]  memory[0x0]
[0x0000000080200000-0x000000009fffffff], 0x000000001fe00000 bytes
flags: 0x0
    [    0.000000]  reserved.cnt  = 0x2
    [    0.000000]  reserved[0x0]
[0x0000000080200000-0x0000000080c2dfeb], 0x0000000000a2dfec bytes
flags: 0x0
    [    0.000000]  reserved[0x1]
[0xfffffff080100000-0xfffffff080100527], 0x0000000000000528 bytes
flags: 0x0

    With the fix applied:

    [    0.000000] MEMBLOCK configuration:
    [    0.000000]  memory size = 0x000000001fe00000 reserved size =
0x0000000000a2e514
    [    0.000000]  memory.cnt  = 0x1
    [    0.000000]  memory[0x0]
[0x0000000080200000-0x000000009fffffff], 0x000000001fe00000 bytes
flags: 0x0
    [    0.000000]  reserved.cnt  = 0x2
    [    0.000000]  reserved[0x0]
[0x0000000080200000-0x0000000080c2dfeb], 0x0000000000a2dfec bytes
flags: 0x0
    [    0.000000]  reserved[0x1]
[0x0000000080e00000-0x0000000080e00527], 0x0000000000000528 bytes
flags: 0x0

    Fixes: 671f9a3e2e24 ("RISC-V: Setup initial page tables in two stages")
    Signed-off-by: Albert Ou <aou@eecs.berkeley.edu>
    Tested-by: Bin Meng <bmeng.cn@gmail.com>
    Reviewed-by: Anup Patel <anup@brainfault.org>
    Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>

As the commit message says, this commit is a fix to commit
671f9a3e2e24 ("RISC-V: Setup initial page tables in two stages") which
is in the 5.3 kernel:

commit 671f9a3e2e24cdeb2d2856abee7422f093e23e29
Author: Anup Patel <Anup.Patel@wdc.com>
Date:   Fri Jun 28 13:36:21 2019 -0700

    RISC-V: Setup initial page tables in two stages

    Currently, the setup_vm() does initial page table setup in one-shot
    very early before enabling MMU. Due to this, the setup_vm() has to map
    all possible kernel virtual addresses since it does not know size and
    location of RAM. This means we have kernel mappings for non-existent
    RAM and any buggy driver (or kernel) code doing out-of-bound access
    to RAM will not fault and cause underterministic behaviour.

    Further, the setup_vm() creates PMD mappings (i.e. 2M mappings) for
    RV64 systems. This means for PAGE_OFFSET=0xffffffe000000000 (i.e.
    MAXPHYSMEM_128GB=y), the setup_vm() will require 129 pages (i.e.
    516 KB) of memory for initial page tables which is never freed. The
    memory required for initial page tables will further increase if
    we chose a lower value of PAGE_OFFSET (e.g. 0xffffff0000000000)

    This patch implements two-staged initial page table setup, as follows:
    1. Early (i.e. setup_vm()): This stage maps kernel image and DTB in
    a early page table (i.e. early_pg_dir). The early_pg_dir will be used
    only by boot HART so it can be freed as-part of init memory free-up.
    2. Final (i.e. setup_vm_final()): This stage maps all possible RAM
    banks in the final page table (i.e. swapper_pg_dir). The boot HART
    will start using swapper_pg_dir at the end of setup_vm_final(). All
    non-boot HARTs directly use the swapper_pg_dir created by boot HART.

    We have following advantages with this new approach:
    1. Kernel mappings for non-existent RAM don't exists anymore.
    2. Memory consumed by initial page tables is now indpendent of the
    chosen PAGE_OFFSET.
    3. Memory consumed by initial page tables on RV64 system is 2 pages
    (i.e. 8 KB) which has significantly reduced and these pages will be
    freed as-part of the init memory free-up.

    The patch also provides a foundation for implementing strict kernel
    mappings where we protect kernel text and rodata using PTE permissions.

    Suggested-by: Mike Rapoport <rppt@linux.ibm.com>
    Signed-off-by: Anup Patel <anup.patel@wdc.com>
    [paul.walmsley@sifive.com: updated to apply; fixed a checkpatch warning]
    Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>

I cherry-picked 922b0375fc93fb1a20c5617e37c389c26bbccb70 on top of a
5.3 kernel, and with that the kernel boots again with this patch set.

>
> > If we can't support older kernels with the default bios option we at
> > least need to know why and list that in the release notes.
> >

I will include the above findings in my commit message of this series,
mentioning that to test an older kernel like 5.3 version Linux,
922b0375fc93fb1a20c5617e37c389c26bbccb70 should be cherry-picked.

Regards,
Bin

On Sun, Aug 2, 2020 at 11:51 PM Bin Meng <bmeng.cn@gmail.com> wrote:
>
> Hi Alistair,
>
> On Wed, Jul 29, 2020 at 1:10 PM Bin Meng <bmeng.cn@gmail.com> wrote:
> >
> > Hi Alistair,
> >
> > On Wed, Jul 29, 2020 at 1:05 PM Alistair Francis <alistair23@gmail.com> wrote:
> > >
> > > On Tue, Jul 28, 2020 at 9:51 PM Bin Meng <bmeng.cn@gmail.com> wrote:
> > > >
> > > > Hi Alistair,
> > > >
> > > > On Wed, Jul 29, 2020 at 2:26 AM Alistair Francis <alistair23@gmail.com> wrote:
> > > > >
> > > > > On Tue, Jul 28, 2020 at 8:46 AM Bin Meng <bmeng.cn@gmail.com> wrote:
> > > > > >
> > > > > > Hi Alistair,
> > > > > >
> > > > > > On Tue, Jul 28, 2020 at 11:39 PM Alistair Francis <alistair23@gmail.com> wrote:
> > > > > > >
> > > > > > > On Wed, Jul 15, 2020 at 9:55 PM Bin Meng <bmeng.cn@gmail.com> wrote:
> > > > > > > >
> > > > > > > > Hi Alistair,
> > > > > > > >
> > > > > > > > On Mon, Jul 13, 2020 at 9:53 AM Bin Meng <bmeng.cn@gmail.com> wrote:
> > > > > > > > >
> > > > > > > > > On Sun, Jul 12, 2020 at 1:34 AM Alistair Francis <alistair23@gmail.com> wrote:
> > > > > > > > > >
> > > > > > > > > > On Thu, Jul 9, 2020 at 10:07 PM Bin Meng <bmeng.cn@gmail.com> wrote:
> > > > > > > > > > >
> > > > > > > > > > > From: Bin Meng <bin.meng@windriver.com>
> > > > > > > > > > >
> > > > > > > > > > > Update virt and sifive_u machines to use the opensbi fw_dynamic bios
> > > > > > > > > > > image built for the generic FDT platform.
> > > > > > > > > > >
> > > > > > > > > > > Remove the out-of-date no longer used bios images.
> > > > > > > > > > >
> > > > > > > > > > > Signed-off-by: Bin Meng <bin.meng@windriver.com>
> > > > > > > > > > > Reviewed-by: Anup Patel <anup@brainfault.org>
> > > > > > > > > > > Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
> > > > > > > > > >
> > > > > > > > > > This patch seems to break 32-bit Linux boots on the sifive_u and virt machines.
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > It looks only Linux boot on sifive_u is broken. On our side, we have
> > > > > > > > > been using VxWorks to test 32-bit OpenSBI on sifive_u so this issue
> > > > > > > > > gets unnoticed. I will take a look.
> > > > > > > >
> > > > > > > > I've figured out the issue of 32-bit Linux booting failure on
> > > > > > > > sifive_u. A patch has been sent to Linux upstream:
> > > > > > > > http://lists.infradead.org/pipermail/linux-riscv/2020-July/001213.html
> > > > > > >
> > > > > > > Thanks for that. What change in QEMU causes this failure though?
> > > > > > >
> > > > > >
> > > > > > There is nothing wrong in QEMU.
> > > > >
> > > > > There is. This patch causes a regression for 32-bit Linux boot on the
> > > > > sifive_u. Your v5 has not addressed this.
> > > >
> > > > The 32-bit Linux boot failure was fixed by:
> > > > http://lists.infradead.org/pipermail/linux-riscv/2020-July/001213.html
> > > >
> > > > What additional issue did you see?
> > > >
> > > > >
> > > > > With this patch, the Linux boot stops here:
> > > > >
> > > > > OpenSBI v0.8
> > > > >    ____                    _____ ____ _____
> > > > >   / __ \                  / ____|  _ \_   _|
> > > > >  | |  | |_ __   ___ _ __ | (___ | |_) || |
> > > > >  | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
> > > > >  | |__| | |_) |  __/ | | |____) | |_) || |_
> > > > >   \____/| .__/ \___|_| |_|_____/|____/_____|
> > > > >         | |
> > > > >         |_|
> > > > >
> > > > > Platform Name       : SiFive HiFive Unleashed A00
> > > > > Platform Features   : timer,mfdeleg
> > > > > Platform HART Count : 4
> > > > > Boot HART ID        : 3
> > > > > Boot HART ISA       : rv64imafdcsu
> > > >
> > > > This is a 64-bit hardware.
> > >
> > > You are right. It's not 32-bit, that was my mistake. I'm used to my
> > > first test being 32-bit, but in this case it's not.
> > >
> > > It looks like this commit instead breaks the sifive_u for 64-bit with
> > > the 5.3 kernel.
> > >
> > > >
> > > > > BOOT HART Features  : pmp,scounteren,mcounteren
> > > > > BOOT HART PMP Count : 16
> > > > > Firmware Base       : 0x80000000
> > > > > Firmware Size       : 116 KB
> > > > > Runtime SBI Version : 0.2
> > > > >
> > > > > MIDELEG : 0x0000000000000222
> > > > > MEDELEG : 0x000000000000b109
> > > > > PMP0    : 0x0000000080000000-0x000000008001ffff (A)
> > > > > PMP1    : 0x0000000000000000-0xffffffffffffffff (A,R,W,X)
> > > > > [    0.000000] OF: fdt: Ignoring memory range 0x80000000 - 0x80200000
> > > > > [    0.000000] Linux version 5.3.0 (oe-user@oe-host) (gcc version
> > > >
> > > > It seems that you are using quite an old kernel. Can you please try
> > > > the latest version?
> > >
> > > It is an old kernel, but old kernels should still keep working (or we
> > > should at least know why they don't)
> > >
> > > >
> > > > > 9.2.0 (GCC)) #1 SMP Thu Sep 19 18:34:52 UTC 2019
> > > > > [    0.000000] earlycon: sbi0 at I/O port 0x0 (options '')
> > > > > [    0.000000] printk: bootconsole [sbi0] enabled
> > > > > [    0.000000] initrd not found or empty - disabling initrd
> > > > > [    0.000000] Zone ranges:
> > > > > [    0.000000]   DMA32    [mem 0x0000000080200000-0x00000000bfffffff]
> > > > > [    0.000000]   Normal   empty
> > > > > [    0.000000] Movable zone start for each node
> > > > > [    0.000000] Early memory node ranges
> > > > > [    0.000000]   node   0: [mem 0x0000000080200000-0x00000000bfffffff]
> > > > > [    0.000000] Initmem setup node 0 [mem 0x0000000080200000-0x00000000bfffffff]
> > > > > [    0.000000] OF: fdt: Invalid device tree blob header
> > > > > [    0.000000] software IO TLB: mapped [mem 0xbb1fe000-0xbf1fe000] (64MB)
> > > > >
> > > > > Without this patch I can boot all the way to looking for a rootFS.
> > > > >
> > > > > Please don't send new versions of patches without addresses regressions.
> > > >
> > > > The patches were sent after addressing all regressions you reported
> > > > (well the 32-bit Linux booting issue is actually not a QEMU
> > > > regression, but one that exists in the Linux kernel side for a long
> > > > time).
> > >
> > > Yep, that is my mistake. Sorry about the confusion.
> > >
> > > >
> > > > I just tested 64-bit Linux boot on both virt and sifive_u, and they
> > > > both can boot all the way to looking for a root fs.
> > >
> > > Can you test with older kernels?
> > >
> >
> > OK I will investigate.
>
> I've located the commit in the newer Linux kernel that fixed the boot
> failure of the 5.3 kernel. See below:
>
> commit 922b0375fc93fb1a20c5617e37c389c26bbccb70
> Author: Albert Ou <aou@eecs.berkeley.edu>
> Date:   Fri Sep 27 16:14:18 2019 -0700
>
>     riscv: Fix memblock reservation for device tree blob
>
>     This fixes an error with how the FDT blob is reserved in memblock.
>     An incorrect physical address calculation exposed the FDT header to
>     unintended corruption, which typically manifested with of_fdt_raw_init()
>     faulting during late boot after fdt_totalsize() returned a wrong value.
>     Systems with smaller physical memory sizes more frequently trigger this
>     issue, as the kernel is more likely to allocate from the DMA32 zone
>     where bbl places the DTB after the kernel image.
>
>     Commit 671f9a3e2e24 ("RISC-V: Setup initial page tables in two stages")
>     changed the mapping of the DTB to reside in the fixmap area.
>     Consequently, early_init_fdt_reserve_self() cannot be used anymore in
>     setup_bootmem() since it relies on __pa() to derive a physical address,
>     which does not work with dtb_early_va that is no longer a valid kernel
>     logical address.
>
>     The reserved[0x1] region shows the effect of the pointer underflow
>     resulting from the __pa(initial_boot_params) offset subtraction:
>
>     [    0.000000] MEMBLOCK configuration:
>     [    0.000000]  memory size = 0x000000001fe00000 reserved size =
> 0x0000000000a2e514
>     [    0.000000]  memory.cnt  = 0x1
>     [    0.000000]  memory[0x0]
> [0x0000000080200000-0x000000009fffffff], 0x000000001fe00000 bytes
> flags: 0x0
>     [    0.000000]  reserved.cnt  = 0x2
>     [    0.000000]  reserved[0x0]
> [0x0000000080200000-0x0000000080c2dfeb], 0x0000000000a2dfec bytes
> flags: 0x0
>     [    0.000000]  reserved[0x1]
> [0xfffffff080100000-0xfffffff080100527], 0x0000000000000528 bytes
> flags: 0x0
>
>     With the fix applied:
>
>     [    0.000000] MEMBLOCK configuration:
>     [    0.000000]  memory size = 0x000000001fe00000 reserved size =
> 0x0000000000a2e514
>     [    0.000000]  memory.cnt  = 0x1
>     [    0.000000]  memory[0x0]
> [0x0000000080200000-0x000000009fffffff], 0x000000001fe00000 bytes
> flags: 0x0
>     [    0.000000]  reserved.cnt  = 0x2
>     [    0.000000]  reserved[0x0]
> [0x0000000080200000-0x0000000080c2dfeb], 0x0000000000a2dfec bytes
> flags: 0x0
>     [    0.000000]  reserved[0x1]
> [0x0000000080e00000-0x0000000080e00527], 0x0000000000000528 bytes
> flags: 0x0
>
>     Fixes: 671f9a3e2e24 ("RISC-V: Setup initial page tables in two stages")
>     Signed-off-by: Albert Ou <aou@eecs.berkeley.edu>
>     Tested-by: Bin Meng <bmeng.cn@gmail.com>
>     Reviewed-by: Anup Patel <anup@brainfault.org>
>     Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
>
> As the commit message says, this commit is a fix to commit
> 671f9a3e2e24 ("RISC-V: Setup initial page tables in two stages") which
> is in the 5.3 kernel:
>
> commit 671f9a3e2e24cdeb2d2856abee7422f093e23e29
> Author: Anup Patel <Anup.Patel@wdc.com>
> Date:   Fri Jun 28 13:36:21 2019 -0700
>
>     RISC-V: Setup initial page tables in two stages
>
>     Currently, the setup_vm() does initial page table setup in one-shot
>     very early before enabling MMU. Due to this, the setup_vm() has to map
>     all possible kernel virtual addresses since it does not know size and
>     location of RAM. This means we have kernel mappings for non-existent
>     RAM and any buggy driver (or kernel) code doing out-of-bound access
>     to RAM will not fault and cause underterministic behaviour.
>
>     Further, the setup_vm() creates PMD mappings (i.e. 2M mappings) for
>     RV64 systems. This means for PAGE_OFFSET=0xffffffe000000000 (i.e.
>     MAXPHYSMEM_128GB=y), the setup_vm() will require 129 pages (i.e.
>     516 KB) of memory for initial page tables which is never freed. The
>     memory required for initial page tables will further increase if
>     we chose a lower value of PAGE_OFFSET (e.g. 0xffffff0000000000)
>
>     This patch implements two-staged initial page table setup, as follows:
>     1. Early (i.e. setup_vm()): This stage maps kernel image and DTB in
>     a early page table (i.e. early_pg_dir). The early_pg_dir will be used
>     only by boot HART so it can be freed as-part of init memory free-up.
>     2. Final (i.e. setup_vm_final()): This stage maps all possible RAM
>     banks in the final page table (i.e. swapper_pg_dir). The boot HART
>     will start using swapper_pg_dir at the end of setup_vm_final(). All
>     non-boot HARTs directly use the swapper_pg_dir created by boot HART.
>
>     We have following advantages with this new approach:
>     1. Kernel mappings for non-existent RAM don't exists anymore.
>     2. Memory consumed by initial page tables is now indpendent of the
>     chosen PAGE_OFFSET.
>     3. Memory consumed by initial page tables on RV64 system is 2 pages
>     (i.e. 8 KB) which has significantly reduced and these pages will be
>     freed as-part of the init memory free-up.
>
>     The patch also provides a foundation for implementing strict kernel
>     mappings where we protect kernel text and rodata using PTE permissions.
>
>     Suggested-by: Mike Rapoport <rppt@linux.ibm.com>
>     Signed-off-by: Anup Patel <anup.patel@wdc.com>
>     [paul.walmsley@sifive.com: updated to apply; fixed a checkpatch warning]
>     Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
>
> I cherry-picked 922b0375fc93fb1a20c5617e37c389c26bbccb70 on top of a
> 5.3 kernel, and with that the kernel boots again with this patch set.

Thanks for looking into this. It's unfortunate that OpenSBI won't work
with older kernels, but people can always use their own OpenSBI build
in that case. 5.4 is an LTS and it works so that seems fine to me.

Alistair

>
> >
> > > If we can't support older kernels with the default bios option we at
> > > least need to know why and list that in the release notes.
> > >
>
> I will include the above findings in my commit message of this series,
> mentioning that to test an older kernel like 5.3 version Linux,
> 922b0375fc93fb1a20c5617e37c389c26bbccb70 should be cherry-picked.
>
> Regards,
> Bin