From: Igor Mammedov <imammedo@redhat.com>

replaces ad-hoc build_isa_devices_aml() with generic AcpiDevAmlIf
way to build bridge AML including all devices that are attached to
its ISA bus.

Later when PCI is converted to AcpiDevAmlIf, build_piix4_isa_bridge()
will also be dropped since PCI parts itself will take care of
building device prologue/epilogue AML for each enumerated PCI
device.

Expected AML change is contextual, where ISA devices are moved
from separately declared _SB.PCI0.ISA scope , directly under
Device(ISA) node.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
Message-Id: <20220608135340.3304695-20-imammedo@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/i386/acpi-build.c | 16 +++++++++++-----
 hw/isa/piix3.c       | 17 +++++++++++++++++
 2 files changed, 28 insertions(+), 5 deletions(-)

diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index d943354999..f903f30b7e 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -1280,15 +1280,22 @@ static void build_piix4_isa_bridge(Aml *table)
 {
     Aml *dev;
     Aml *scope;
+    Object *obj;
+    bool ambiguous;
+
+    /*
+     * temporarily fish out isa bridge, build_piix4_isa_bridge() will be dropped
+     * once PCI is converted to AcpiDevAmlIf and would be ble to generate
+     * AML for bridge itself
+     */
+    obj = object_resolve_path_type("", TYPE_PIIX3_PCI_DEVICE, &ambiguous);
+    assert(obj && !ambiguous);
 
     scope =  aml_scope("_SB.PCI0");
     dev = aml_device("ISA");
     aml_append(dev, aml_name_decl("_ADR", aml_int(0x00010000)));
 
-    /* PIIX PCI to ISA irq remapping */
-    aml_append(dev, aml_operation_region("P40C", AML_PCI_CONFIG,
-                                         aml_int(0x60), 0x04));
-
+    call_dev_aml_func(DEVICE(obj), dev);
     aml_append(scope, dev);
     aml_append(table, scope);
 }
@@ -1476,7 +1483,6 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
             build_hpet_aml(dsdt);
         }
         build_piix4_isa_bridge(dsdt);
-        build_isa_devices_aml(dsdt);
         if (pm->pcihp_bridge_en || pm->pcihp_root_en) {
             build_x86_acpi_pci_hotplug(dsdt, pm->pcihp_io_base);
         }
diff --git a/hw/isa/piix3.c b/hw/isa/piix3.c
index dab901c9ad..bfccd666d4 100644
--- a/hw/isa/piix3.c
+++ b/hw/isa/piix3.c
@@ -32,6 +32,7 @@
 #include "sysemu/reset.h"
 #include "sysemu/runstate.h"
 #include "migration/vmstate.h"
+#include "hw/acpi/acpi_aml_interface.h"
 
 #define XEN_PIIX_NUM_PIRQS      128ULL
 
@@ -286,10 +287,24 @@ static void piix3_realize(PCIDevice *dev, Error **errp)
     qemu_register_reset(piix3_reset, d);
 }
 
+static void build_pci_isa_aml(AcpiDevAmlIf *adev, Aml *scope)
+{
+    BusChild *kid;
+    BusState *bus = qdev_get_child_bus(DEVICE(adev), "isa.0");
+
+    /* PIIX PCI to ISA irq remapping */
+    aml_append(scope, aml_operation_region("P40C", AML_PCI_CONFIG,
+                                         aml_int(0x60), 0x04));
+    QTAILQ_FOREACH(kid, &bus->children, sibling) {
+        call_dev_aml_func(DEVICE(kid->child), scope);
+    }
+}
+
 static void pci_piix3_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
     PCIDeviceClass *k = PCI_DEVICE_CLASS(klass);
+    AcpiDevAmlIfClass *adevc = ACPI_DEV_AML_IF_CLASS(klass);
 
     dc->desc        = "ISA bridge";
     dc->vmsd        = &vmstate_piix3;
@@ -304,6 +319,7 @@ static void pci_piix3_class_init(ObjectClass *klass, void *data)
      * pc_piix.c's pc_init1()
      */
     dc->user_creatable = false;
+    adevc->build_dev_aml = build_pci_isa_aml;
 }
 
 static const TypeInfo piix3_pci_type_info = {
@@ -314,6 +330,7 @@ static const TypeInfo piix3_pci_type_info = {
     .class_init = pci_piix3_class_init,
     .interfaces = (InterfaceInfo[]) {
         { INTERFACE_CONVENTIONAL_PCI_DEVICE },
+        { TYPE_ACPI_DEV_AML_IF },
         { },
     },
 };
-- 
MST

Am 10.06.22 um 09:57 schrieb Michael S. Tsirkin:
> From: Igor Mammedov <imammedo@redhat.com>
> 
> replaces ad-hoc build_isa_devices_aml() with generic AcpiDevAmlIf
> way to build bridge AML including all devices that are attached to
> its ISA bus.
> 
> Later when PCI is converted to AcpiDevAmlIf, build_piix4_isa_bridge()
> will also be dropped since PCI parts itself will take care of
> building device prologue/epilogue AML for each enumerated PCI
> device.
> 
> Expected AML change is contextual, where ISA devices are moved
> from separately declared _SB.PCI0.ISA scope , directly under
> Device(ISA) node.
> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> Message-Id: <20220608135340.3304695-20-imammedo@redhat.com>
> Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

Hi,
while trying to reproduce another issue, I ended up with a Windows 10
guest that would boot with QEMU 7.0, but get stuck after the Windows
logo/spinning circles with QEMU 7.1 (also with 8.0.0-rc1). Machine type
is pc-i440fx-6.2[0]. Bisecting led to this commit.

It only happens the first time the VM is booted, killing the process and
re-trying always worked afterwards. So it's not a big deal and might
just be some ACPI-related Windows quirk. But I thought I should ask here
to be sure.

For bisecting, I restored the disk state after each attempt. While
getting stuck sometimes took 3-4 attempts, I tested about 10 times until
I declared a commit good, and re-tested the commit before this one 15
times, so I'm pretty sure this is the one where the issue started appearing.

So, anything that could potentially be wrong with the commit or is this
most likely just some Windows quirk/bug we can't do much about?

If you need more information, please let me know!

Best Regards,
Fiona

[0] command line:
> ./qemu-system-x86_64 \
>   -accel 'kvm' \
>   -name 'stuckafterrollbackonboot,debug-threads=on' \
>   -no-shutdown \
>   -chardev 'socket,id=qmp,path=/var/run/qemu-server/161.qmp,server=on,wait=off' \
>   -mon 'chardev=qmp,mode=control' \
>   -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5' \
>   -mon 'chardev=qmp-event,mode=control' \
>   -pidfile /var/run/qemu-server/161.pid \
>   -smbios 'type=1,uuid=f2b77ed0-73c1-4372-9490-b2c1b59431af' \
>   -smp '4,sockets=1,cores=4,maxcpus=4' \
>   -nodefaults \
>   -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' \
>   -vnc 'unix:/var/run/qemu-server/161.vnc,password=on' \
>   -no-hpet \
>   -cpu 'kvm64,enforce,hv_ipi,hv_relaxed,hv_reset,hv_runtime,hv_spinlocks=0x1fff,hv_stimer,hv_synic,hv_time,hv_vapic,hv_vpindex,+kvm_pv_eoi,+kvm_pv_unhalt,+lahf_lm,+sep' \
>   -m 6144 \
>   -device 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' \
>   -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' \
>   -device 'pci-bridge,id=pci.3,chassis_nr=3,bus=pci.0,addr=0x5' \
>   -device 'vmgenid,guid=faa21a64-5921-45fe-9ff3-1f132b9ed029' \
>   -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' \
>   -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' \
>   -device 'VGA,id=vga,bus=pci.0,addr=0x2,edid=off' \
>   -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3,free-page-reporting=on' \
>   -iscsi 'initiator-name=iqn.1993-08.org.debian:01:7d9a912f961' \
>   -device 'ahci,id=ahci0,multifunction=on,bus=pci.0,addr=0x7' \
>   -drive 'file=/dev/pve/vm-161-disk-0,if=none,id=drive-sata0,format=raw,cache=none,aio=io_uring,detect-zeroes=on' \
>   -device 'ide-hd,bus=ahci0.0,drive=drive-sata0,id=sata0,bootindex=100' \
>   -netdev 'type=tap,id=net0,ifname=tap161i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown' \
>   -device 'e1000,mac=42:BF:8B:AE:68:05,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=102' \
>   -rtc 'driftfix=slew,base=localtime' \
>   -machine 'type=pc-i440fx-6.2' \
>   -global 'kvm-pit.lost_tick_policy=discard'

On Tue, 28 Mar 2023 14:58:21 +0200
Fiona Ebner <f.ebner@proxmox.com> wrote:

> Am 10.06.22 um 09:57 schrieb Michael S. Tsirkin:
> > From: Igor Mammedov <imammedo@redhat.com>
> > 
> > replaces ad-hoc build_isa_devices_aml() with generic AcpiDevAmlIf
> > way to build bridge AML including all devices that are attached to
> > its ISA bus.
> > 
> > Later when PCI is converted to AcpiDevAmlIf, build_piix4_isa_bridge()
> > will also be dropped since PCI parts itself will take care of
> > building device prologue/epilogue AML for each enumerated PCI
> > device.
> > 
> > Expected AML change is contextual, where ISA devices are moved
> > from separately declared _SB.PCI0.ISA scope , directly under
> > Device(ISA) node.
> > 
> > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> > Message-Id: <20220608135340.3304695-20-imammedo@redhat.com>
> > Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>  
> 
> Hi,
> while trying to reproduce another issue, I ended up with a Windows 10
> guest that would boot with QEMU 7.0, but get stuck after the Windows
> logo/spinning circles with QEMU 7.1 (also with 8.0.0-rc1). Machine type
> is pc-i440fx-6.2[0]. Bisecting led to this commit.
> 
> It only happens the first time the VM is booted, killing the process and
> re-trying always worked afterwards. So it's not a big deal and might
> just be some ACPI-related Windows quirk. But I thought I should ask here
> to be sure.
> 
> For bisecting, I restored the disk state after each attempt. While
> getting stuck sometimes took 3-4 attempts, I tested about 10 times until
> I declared a commit good, and re-tested the commit before this one 15
> times, so I'm pretty sure this is the one where the issue started appearing.
> 
> So, anything that could potentially be wrong with the commit or is this
> most likely just some Windows quirk/bug we can't do much about?
> 
> If you need more information, please let me know!

Please describe in more detail your setup/steps where it reproduces
(incl. Windows version/build, used QEMU CLI) so I could try to reproduce it locally.

(in past there were issues with German version that some where
experience but not reproducible on my side, that resolved with
upgrading to newer QEMU (if I recall correctly issue was opened
on QEMU's gitlab tracker))

> 
> Best Regards,
> Fiona
> 
> [0] command line:
> > ./qemu-system-x86_64 \
> >   -accel 'kvm' \
> >   -name 'stuckafterrollbackonboot,debug-threads=on' \
> >   -no-shutdown \
> >   -chardev 'socket,id=qmp,path=/var/run/qemu-server/161.qmp,server=on,wait=off' \
> >   -mon 'chardev=qmp,mode=control' \
> >   -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5' \
> >   -mon 'chardev=qmp-event,mode=control' \
> >   -pidfile /var/run/qemu-server/161.pid \
> >   -smbios 'type=1,uuid=f2b77ed0-73c1-4372-9490-b2c1b59431af' \
> >   -smp '4,sockets=1,cores=4,maxcpus=4' \
> >   -nodefaults \
> >   -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' \
> >   -vnc 'unix:/var/run/qemu-server/161.vnc,password=on' \
> >   -no-hpet \
> >   -cpu 'kvm64,enforce,hv_ipi,hv_relaxed,hv_reset,hv_runtime,hv_spinlocks=0x1fff,hv_stimer,hv_synic,hv_time,hv_vapic,hv_vpindex,+kvm_pv_eoi,+kvm_pv_unhalt,+lahf_lm,+sep' \
> >   -m 6144 \
> >   -device 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' \
> >   -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' \
> >   -device 'pci-bridge,id=pci.3,chassis_nr=3,bus=pci.0,addr=0x5' \
> >   -device 'vmgenid,guid=faa21a64-5921-45fe-9ff3-1f132b9ed029' \
> >   -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' \
> >   -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' \
> >   -device 'VGA,id=vga,bus=pci.0,addr=0x2,edid=off' \
> >   -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3,free-page-reporting=on' \
> >   -iscsi 'initiator-name=iqn.1993-08.org.debian:01:7d9a912f961' \
> >   -device 'ahci,id=ahci0,multifunction=on,bus=pci.0,addr=0x7' \
> >   -drive 'file=/dev/pve/vm-161-disk-0,if=none,id=drive-sata0,format=raw,cache=none,aio=io_uring,detect-zeroes=on' \
> >   -device 'ide-hd,bus=ahci0.0,drive=drive-sata0,id=sata0,bootindex=100' \
> >   -netdev 'type=tap,id=net0,ifname=tap161i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown' \
> >   -device 'e1000,mac=42:BF:8B:AE:68:05,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=102' \
> >   -rtc 'driftfix=slew,base=localtime' \
> >   -machine 'type=pc-i440fx-6.2' \
> >   -global 'kvm-pit.lost_tick_policy=discard'  
> 
>

Am 30.03.23 um 10:22 schrieb Igor Mammedov:
> On Tue, 28 Mar 2023 14:58:21 +0200
> Fiona Ebner <f.ebner@proxmox.com> wrote:
> 
>> Am 10.06.22 um 09:57 schrieb Michael S. Tsirkin:
>>> From: Igor Mammedov <imammedo@redhat.com>
>>>
>>> replaces ad-hoc build_isa_devices_aml() with generic AcpiDevAmlIf
>>> way to build bridge AML including all devices that are attached to
>>> its ISA bus.
>>>
>>> Later when PCI is converted to AcpiDevAmlIf, build_piix4_isa_bridge()
>>> will also be dropped since PCI parts itself will take care of
>>> building device prologue/epilogue AML for each enumerated PCI
>>> device.
>>>
>>> Expected AML change is contextual, where ISA devices are moved
>>> from separately declared _SB.PCI0.ISA scope , directly under
>>> Device(ISA) node.
>>>
>>> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
>>> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
>>> Message-Id: <20220608135340.3304695-20-imammedo@redhat.com>
>>> Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
>>> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>  
>>
>> Hi,
>> while trying to reproduce another issue, I ended up with a Windows 10
>> guest that would boot with QEMU 7.0, but get stuck after the Windows
>> logo/spinning circles with QEMU 7.1 (also with 8.0.0-rc1). Machine type
>> is pc-i440fx-6.2[0]. Bisecting led to this commit.
>>
>> It only happens the first time the VM is booted, killing the process and
>> re-trying always worked afterwards. So it's not a big deal and might
>> just be some ACPI-related Windows quirk. But I thought I should ask here
>> to be sure.
>>
>> For bisecting, I restored the disk state after each attempt. While
>> getting stuck sometimes took 3-4 attempts, I tested about 10 times until
>> I declared a commit good, and re-tested the commit before this one 15
>> times, so I'm pretty sure this is the one where the issue started appearing.
>>
>> So, anything that could potentially be wrong with the commit or is this
>> most likely just some Windows quirk/bug we can't do much about?
>>
>> If you need more information, please let me know!
> 
> Please describe in more detail your setup/steps where it reproduces
> (incl. Windows version/build, used QEMU CLI) so I could try to reproduce it locally.
> 
> (in past there were issues with German version that some where
> experience but not reproducible on my side, that resolved with
> upgrading to newer QEMU (if I recall correctly issue was opened
> on QEMU's gitlab tracker))
> 

Windows 10 Education
Version 1809
Build 17763.1

It's not the German ISO, I used default settings (except location
Austria and German keymap) and I don't think I did anything other than
shutdown after the install was over.

The command line is below. I did use our patched QEMU builds when I got
into the situation, but I don't think they touch anything ACPI-related
and bisecting was done without our patches on top.

I tried to reproduce the situation again from scratch today, but wasn't
able to. I do still have the problematic disk (snapshot) where the issue
occurs as an LVM-Thin volume. If you'd like to have access to that,
please send me a direct mail and we can discuss the details there.

Best Regards,
Fiona

>>
>> Best Regards,
>> Fiona
>>
>> [0] command line:
>>> ./qemu-system-x86_64 \
>>>   -accel 'kvm' \
>>>   -name 'stuckafterrollbackonboot,debug-threads=on' \
>>>   -no-shutdown \
>>>   -chardev 'socket,id=qmp,path=/var/run/qemu-server/161.qmp,server=on,wait=off' \
>>>   -mon 'chardev=qmp,mode=control' \
>>>   -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5' \
>>>   -mon 'chardev=qmp-event,mode=control' \
>>>   -pidfile /var/run/qemu-server/161.pid \
>>>   -smbios 'type=1,uuid=f2b77ed0-73c1-4372-9490-b2c1b59431af' \
>>>   -smp '4,sockets=1,cores=4,maxcpus=4' \
>>>   -nodefaults \
>>>   -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' \
>>>   -vnc 'unix:/var/run/qemu-server/161.vnc,password=on' \
>>>   -no-hpet \
>>>   -cpu 'kvm64,enforce,hv_ipi,hv_relaxed,hv_reset,hv_runtime,hv_spinlocks=0x1fff,hv_stimer,hv_synic,hv_time,hv_vapic,hv_vpindex,+kvm_pv_eoi,+kvm_pv_unhalt,+lahf_lm,+sep' \
>>>   -m 6144 \
>>>   -device 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' \
>>>   -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' \
>>>   -device 'pci-bridge,id=pci.3,chassis_nr=3,bus=pci.0,addr=0x5' \
>>>   -device 'vmgenid,guid=faa21a64-5921-45fe-9ff3-1f132b9ed029' \
>>>   -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' \
>>>   -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' \
>>>   -device 'VGA,id=vga,bus=pci.0,addr=0x2,edid=off' \
>>>   -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3,free-page-reporting=on' \
>>>   -iscsi 'initiator-name=iqn.1993-08.org.debian:01:7d9a912f961' \
>>>   -device 'ahci,id=ahci0,multifunction=on,bus=pci.0,addr=0x7' \
>>>   -drive 'file=/dev/pve/vm-161-disk-0,if=none,id=drive-sata0,format=raw,cache=none,aio=io_uring,detect-zeroes=on' \
>>>   -device 'ide-hd,bus=ahci0.0,drive=drive-sata0,id=sata0,bootindex=100' \
>>>   -netdev 'type=tap,id=net0,ifname=tap161i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown' \
>>>   -device 'e1000,mac=42:BF:8B:AE:68:05,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=102' \
>>>   -rtc 'driftfix=slew,base=localtime' \
>>>   -machine 'type=pc-i440fx-6.2' \
>>>   -global 'kvm-pit.lost_tick_policy=discard'  
>>

On Thu, 30 Mar 2023 13:58:22 +0200
Fiona Ebner <f.ebner@proxmox.com> wrote:

> Am 30.03.23 um 10:22 schrieb Igor Mammedov:
> > On Tue, 28 Mar 2023 14:58:21 +0200
> > Fiona Ebner <f.ebner@proxmox.com> wrote:
> >   
> >> Am 10.06.22 um 09:57 schrieb Michael S. Tsirkin:  
> >>> From: Igor Mammedov <imammedo@redhat.com>
> >>>
> >>> replaces ad-hoc build_isa_devices_aml() with generic AcpiDevAmlIf
> >>> way to build bridge AML including all devices that are attached to
> >>> its ISA bus.
> >>>
> >>> Later when PCI is converted to AcpiDevAmlIf, build_piix4_isa_bridge()
> >>> will also be dropped since PCI parts itself will take care of
> >>> building device prologue/epilogue AML for each enumerated PCI
> >>> device.
> >>>
> >>> Expected AML change is contextual, where ISA devices are moved
> >>> from separately declared _SB.PCI0.ISA scope , directly under
> >>> Device(ISA) node.
> >>>
> >>> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> >>> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> >>> Message-Id: <20220608135340.3304695-20-imammedo@redhat.com>
> >>> Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> >>> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>    
> >>
> >> Hi,
> >> while trying to reproduce another issue, I ended up with a Windows 10
> >> guest that would boot with QEMU 7.0, but get stuck after the Windows
> >> logo/spinning circles with QEMU 7.1 (also with 8.0.0-rc1). Machine type
> >> is pc-i440fx-6.2[0]. Bisecting led to this commit.
> >>
> >> It only happens the first time the VM is booted, killing the process and
> >> re-trying always worked afterwards. So it's not a big deal and might
> >> just be some ACPI-related Windows quirk. But I thought I should ask here
> >> to be sure.
> >>
> >> For bisecting, I restored the disk state after each attempt. While
> >> getting stuck sometimes took 3-4 attempts, I tested about 10 times until
> >> I declared a commit good, and re-tested the commit before this one 15
> >> times, so I'm pretty sure this is the one where the issue started appearing.
> >>
> >> So, anything that could potentially be wrong with the commit or is this
> >> most likely just some Windows quirk/bug we can't do much about?
> >>
> >> If you need more information, please let me know!  
> > 
> > Please describe in more detail your setup/steps where it reproduces
> > (incl. Windows version/build, used QEMU CLI) so I could try to reproduce it locally.
> > 
> > (in past there were issues with German version that some where
> > experience but not reproducible on my side, that resolved with
> > upgrading to newer QEMU (if I recall correctly issue was opened
> > on QEMU's gitlab tracker))
> >   
> 
> Windows 10 Education
> Version 1809
> Build 17763.1
> 
> It's not the German ISO, I used default settings (except location
> Austria and German keymap) and I don't think I did anything other than
> shutdown after the install was over.
> 
> The command line is below. I did use our patched QEMU builds when I got
> into the situation, but I don't think they touch anything ACPI-related
> and bisecting was done without our patches on top.
> 
> I tried to reproduce the situation again from scratch today, but wasn't
> able to. I do still have the problematic disk (snapshot) where the issue
> occurs as an LVM-Thin volume. If you'd like to have access to that,
> please send me a direct mail and we can discuss the details there.

I couldn't reproduce the issue on my host either.
If you still have access to 'broken' disk image, you can try to enable
kernel debug mode in guest and try to attach with debugger to it to see
where it is stuck.

quick instructions how to do it:
 https://gitlab.com/qemu-project/qemu/-/issues/774#note_1270248862
or read more extensive MS docs on topic.

> 
> Best Regards,
> Fiona
> 
> >>
> >> Best Regards,
> >> Fiona
> >>
> >> [0] command line:  
> >>> ./qemu-system-x86_64 \
> >>>   -accel 'kvm' \
> >>>   -name 'stuckafterrollbackonboot,debug-threads=on' \
> >>>   -no-shutdown \
> >>>   -chardev 'socket,id=qmp,path=/var/run/qemu-server/161.qmp,server=on,wait=off' \
> >>>   -mon 'chardev=qmp,mode=control' \
> >>>   -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5' \
> >>>   -mon 'chardev=qmp-event,mode=control' \
> >>>   -pidfile /var/run/qemu-server/161.pid \
> >>>   -smbios 'type=1,uuid=f2b77ed0-73c1-4372-9490-b2c1b59431af' \
> >>>   -smp '4,sockets=1,cores=4,maxcpus=4' \
> >>>   -nodefaults \
> >>>   -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' \
> >>>   -vnc 'unix:/var/run/qemu-server/161.vnc,password=on' \
> >>>   -no-hpet \
> >>>   -cpu 'kvm64,enforce,hv_ipi,hv_relaxed,hv_reset,hv_runtime,hv_spinlocks=0x1fff,hv_stimer,hv_synic,hv_time,hv_vapic,hv_vpindex,+kvm_pv_eoi,+kvm_pv_unhalt,+lahf_lm,+sep' \
> >>>   -m 6144 \
> >>>   -device 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' \
> >>>   -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' \
> >>>   -device 'pci-bridge,id=pci.3,chassis_nr=3,bus=pci.0,addr=0x5' \
> >>>   -device 'vmgenid,guid=faa21a64-5921-45fe-9ff3-1f132b9ed029' \
> >>>   -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' \
> >>>   -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' \
> >>>   -device 'VGA,id=vga,bus=pci.0,addr=0x2,edid=off' \
> >>>   -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3,free-page-reporting=on' \
> >>>   -iscsi 'initiator-name=iqn.1993-08.org.debian:01:7d9a912f961' \
> >>>   -device 'ahci,id=ahci0,multifunction=on,bus=pci.0,addr=0x7' \
> >>>   -drive 'file=/dev/pve/vm-161-disk-0,if=none,id=drive-sata0,format=raw,cache=none,aio=io_uring,detect-zeroes=on' \
> >>>   -device 'ide-hd,bus=ahci0.0,drive=drive-sata0,id=sata0,bootindex=100' \
> >>>   -netdev 'type=tap,id=net0,ifname=tap161i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown' \
> >>>   -device 'e1000,mac=42:BF:8B:AE:68:05,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=102' \
> >>>   -rtc 'driftfix=slew,base=localtime' \
> >>>   -machine 'type=pc-i440fx-6.2' \
> >>>   -global 'kvm-pit.lost_tick_policy=discard'    
> >>  
> 
>

Am 12.04.23 um 14:18 schrieb Igor Mammedov:
> On Thu, 30 Mar 2023 13:58:22 +0200
> Fiona Ebner <f.ebner@proxmox.com> wrote:
> 
>> Am 30.03.23 um 10:22 schrieb Igor Mammedov:
>>> On Tue, 28 Mar 2023 14:58:21 +0200
>>> Fiona Ebner <f.ebner@proxmox.com> wrote:
>>>   
>>>>
>>>> Hi,
>>>> while trying to reproduce another issue, I ended up with a Windows 10
>>>> guest that would boot with QEMU 7.0, but get stuck after the Windows
>>>> logo/spinning circles with QEMU 7.1 (also with 8.0.0-rc1). Machine type
>>>> is pc-i440fx-6.2[0]. Bisecting led to this commit.
>>>>
>>>> It only happens the first time the VM is booted, killing the process and
>>>> re-trying always worked afterwards. So it's not a big deal and might
>>>> just be some ACPI-related Windows quirk. But I thought I should ask here
>>>> to be sure.
>>>>
>>>> For bisecting, I restored the disk state after each attempt. While
>>>> getting stuck sometimes took 3-4 attempts, I tested about 10 times until
>>>> I declared a commit good, and re-tested the commit before this one 15
>>>> times, so I'm pretty sure this is the one where the issue started appearing.
>>>>
>>>> So, anything that could potentially be wrong with the commit or is this
>>>> most likely just some Windows quirk/bug we can't do much about?
>>>>
>>>> If you need more information, please let me know!  
>>>
>>> Please describe in more detail your setup/steps where it reproduces
>>> (incl. Windows version/build, used QEMU CLI) so I could try to reproduce it locally.
>>>
>>> (in past there were issues with German version that some where
>>> experience but not reproducible on my side, that resolved with
>>> upgrading to newer QEMU (if I recall correctly issue was opened
>>> on QEMU's gitlab tracker))
>>>   
>>
>> Windows 10 Education
>> Version 1809
>> Build 17763.1
>>
>> It's not the German ISO, I used default settings (except location
>> Austria and German keymap) and I don't think I did anything other than
>> shutdown after the install was over.
>>
>> The command line is below. I did use our patched QEMU builds when I got
>> into the situation, but I don't think they touch anything ACPI-related
>> and bisecting was done without our patches on top.
>>
>> I tried to reproduce the situation again from scratch today, but wasn't
>> able to. I do still have the problematic disk (snapshot) where the issue
>> occurs as an LVM-Thin volume. If you'd like to have access to that,
>> please send me a direct mail and we can discuss the details there.
> 
> I couldn't reproduce the issue on my host either.
> If you still have access to 'broken' disk image, you can try to enable
> kernel debug mode in guest and try to attach with debugger to it to see
> where it is stuck.
> 
> quick instructions how to do it:
>  https://gitlab.com/qemu-project/qemu/-/issues/774#note_1270248862
> or read more extensive MS docs on topic.
> 

Hmm, I guess I won't be able to enable kernel debug mode without losing
the problematic state of the image. The VM only gets stuck during the
first boot attempt.

Still, I wanted to give it a shot in the hope I can trigger it again
when shutting down with QEMU 6.2.0 and booting with QEMU 7.1.0. I made a
copy of the VM intending to use it as the debug host, but didn't get the
COM port to show up in the guest with
-serial unix:/tmp/com1,server,nowait
I checked in the Device Manager with "Show hidden devices" enabled.

Anyway, when starting the original problematic VM again, it now also got
stuck (visually, in the same place) with QEMU 6.2.0! But only until I
rebooted my host, which made it working with QEMU 6.2.0 again. So I'd
say this commit has nothing to do with the issue after all, just made it
more likely to trigger for me. And also seems less likely to be a QEMU
issue now :)

Best Regards,
Fiona

Sorry for the noise, but just curious, how did you shutdown Windows?
Did you use 'shutdown /s' or just press power button?
Could it be that Windows was actually hibernated.
So, when you try to boot it on the new (old) QEMU version with changed
PCI topology, this could make it upset.
I observed similar behaviour in case of Windows for ARM, but there was
true GSOD afterwards.
When windows is starting again its hibernated state dropped and all goes fine.

Best Regards,
Mike


On Thu, Apr 13, 2023 at 1:34 PM Fiona Ebner <f.ebner@proxmox.com> wrote:
>
> Am 12.04.23 um 14:18 schrieb Igor Mammedov:
> > On Thu, 30 Mar 2023 13:58:22 +0200
> > Fiona Ebner <f.ebner@proxmox.com> wrote:
> >
> >> Am 30.03.23 um 10:22 schrieb Igor Mammedov:
> >>> On Tue, 28 Mar 2023 14:58:21 +0200
> >>> Fiona Ebner <f.ebner@proxmox.com> wrote:
> >>>
> >>>>
> >>>> Hi,
> >>>> while trying to reproduce another issue, I ended up with a Windows 10
> >>>> guest that would boot with QEMU 7.0, but get stuck after the Windows
> >>>> logo/spinning circles with QEMU 7.1 (also with 8.0.0-rc1). Machine type
> >>>> is pc-i440fx-6.2[0]. Bisecting led to this commit.
> >>>>
> >>>> It only happens the first time the VM is booted, killing the process and
> >>>> re-trying always worked afterwards. So it's not a big deal and might
> >>>> just be some ACPI-related Windows quirk. But I thought I should ask here
> >>>> to be sure.
> >>>>
> >>>> For bisecting, I restored the disk state after each attempt. While
> >>>> getting stuck sometimes took 3-4 attempts, I tested about 10 times until
> >>>> I declared a commit good, and re-tested the commit before this one 15
> >>>> times, so I'm pretty sure this is the one where the issue started appearing.
> >>>>
> >>>> So, anything that could potentially be wrong with the commit or is this
> >>>> most likely just some Windows quirk/bug we can't do much about?
> >>>>
> >>>> If you need more information, please let me know!
> >>>
> >>> Please describe in more detail your setup/steps where it reproduces
> >>> (incl. Windows version/build, used QEMU CLI) so I could try to reproduce it locally.
> >>>
> >>> (in past there were issues with German version that some where
> >>> experience but not reproducible on my side, that resolved with
> >>> upgrading to newer QEMU (if I recall correctly issue was opened
> >>> on QEMU's gitlab tracker))
> >>>
> >>
> >> Windows 10 Education
> >> Version 1809
> >> Build 17763.1
> >>
> >> It's not the German ISO, I used default settings (except location
> >> Austria and German keymap) and I don't think I did anything other than
> >> shutdown after the install was over.
> >>
> >> The command line is below. I did use our patched QEMU builds when I got
> >> into the situation, but I don't think they touch anything ACPI-related
> >> and bisecting was done without our patches on top.
> >>
> >> I tried to reproduce the situation again from scratch today, but wasn't
> >> able to. I do still have the problematic disk (snapshot) where the issue
> >> occurs as an LVM-Thin volume. If you'd like to have access to that,
> >> please send me a direct mail and we can discuss the details there.
> >
> > I couldn't reproduce the issue on my host either.
> > If you still have access to 'broken' disk image, you can try to enable
> > kernel debug mode in guest and try to attach with debugger to it to see
> > where it is stuck.
> >
> > quick instructions how to do it:
> >  https://gitlab.com/qemu-project/qemu/-/issues/774#note_1270248862
> > or read more extensive MS docs on topic.
> >
>
> Hmm, I guess I won't be able to enable kernel debug mode without losing
> the problematic state of the image. The VM only gets stuck during the
> first boot attempt.
>
> Still, I wanted to give it a shot in the hope I can trigger it again
> when shutting down with QEMU 6.2.0 and booting with QEMU 7.1.0. I made a
> copy of the VM intending to use it as the debug host, but didn't get the
> COM port to show up in the guest with
> -serial unix:/tmp/com1,server,nowait
> I checked in the Device Manager with "Show hidden devices" enabled.
>
> Anyway, when starting the original problematic VM again, it now also got
> stuck (visually, in the same place) with QEMU 6.2.0! But only until I
> rebooted my host, which made it working with QEMU 6.2.0 again. So I'd
> say this commit has nothing to do with the issue after all, just made it
> more likely to trigger for me. And also seems less likely to be a QEMU
> issue now :)
>
> Best Regards,
> Fiona
>
>

Am 13.04.23 um 13:46 schrieb Mike Maslenkin:
> Sorry for the noise, but just curious, how did you shutdown Windows?
> Did you use 'shutdown /s' or just press power button?
> Could it be that Windows was actually hibernated.
> So, when you try to boot it on the new (old) QEMU version with changed
> PCI topology, this could make it upset.
> I observed similar behaviour in case of Windows for ARM, but there was
> true GSOD afterwards.
> When windows is starting again its hibernated state dropped and all goes fine.
> 
> Best Regards,
> Mike

I think I either pressed the shutdown button in our UI, which sends
system_powerdown via QMP or via "Shut down" in the Windows start menu.
Hibernation is surely something I need to consider (next time), so thank
you for the hint, but if it were that, I'd be surprised at why it got
stuck even with QEMU 6.2.0 today.

If I try "shutdown /h" explicitly, I get "The request is not
supported.(50)".

Best Regards,
Fiona

Hibernate is disabled by default, that is why "shutdown /h" fails.
It can be enabled by 'powercfg.exe /h /size 100; powercfg.exe /h on'

anyway, my hypotheses can be checked easily: just run the VM with
changed RAM size (±1). This should have to drop hibernate state.

BTW I couldn't reproduce problem as well.

On Thu, Apr 13, 2023 at 3:10 PM Fiona Ebner <f.ebner@proxmox.com> wrote:
>
> Am 13.04.23 um 13:46 schrieb Mike Maslenkin:
> > Sorry for the noise, but just curious, how did you shutdown Windows?
> > Did you use 'shutdown /s' or just press power button?
> > Could it be that Windows was actually hibernated.
> > So, when you try to boot it on the new (old) QEMU version with changed
> > PCI topology, this could make it upset.
> > I observed similar behaviour in case of Windows for ARM, but there was
> > true GSOD afterwards.
> > When windows is starting again its hibernated state dropped and all goes fine.
> >
> > Best Regards,
> > Mike
>
> I think I either pressed the shutdown button in our UI, which sends
> system_powerdown via QMP or via "Shut down" in the Windows start menu.
> Hibernation is surely something I need to consider (next time), so thank
> you for the hint, but if it were that, I'd be surprised at why it got
> stuck even with QEMU 6.2.0 today.
>
> If I try "shutdown /h" explicitly, I get "The request is not
> supported.(50)".
>
> Best Regards,
> Fiona
>

Am 14.04.23 um 02:07 schrieb Mike Maslenkin:
> Hibernate is disabled by default, that is why "shutdown /h" fails.
> It can be enabled by 'powercfg.exe /h /size 100; powercfg.exe /h on'
> 
> anyway, my hypotheses can be checked easily: just run the VM with
> changed RAM size (±1). This should have to drop hibernate state.

You got it! The VM boots when I change the RAM size. I now enabled
hibernation and tried to reproduce the issue with "shutdown /h", but
couldn't. And not sure how the VM ended up hibernated when it was
disabled, maybe somehow as part of the installation/upgrades?

Best Regards,
Fiona

> 
> BTW I couldn't reproduce problem as well.
> 
> On Thu, Apr 13, 2023 at 3:10 PM Fiona Ebner <f.ebner@proxmox.com> wrote:
>>
>> Am 13.04.23 um 13:46 schrieb Mike Maslenkin:
>>> Sorry for the noise, but just curious, how did you shutdown Windows?
>>> Did you use 'shutdown /s' or just press power button?
>>> Could it be that Windows was actually hibernated.
>>> So, when you try to boot it on the new (old) QEMU version with changed
>>> PCI topology, this could make it upset.
>>> I observed similar behaviour in case of Windows for ARM, but there was
>>> true GSOD afterwards.
>>> When windows is starting again its hibernated state dropped and all goes fine.
>>>
>>> Best Regards,
>>> Mike
>>
>> I think I either pressed the shutdown button in our UI, which sends
>> system_powerdown via QMP or via "Shut down" in the Windows start menu.
>> Hibernation is surely something I need to consider (next time), so thank
>> you for the hint, but if it were that, I'd be surprised at why it got
>> stuck even with QEMU 6.2.0 today.
>>
>> If I try "shutdown /h" explicitly, I get "The request is not
>> supported.(50)".
>>
>> Best Regards,
>> Fiona
>>
> 
>