Hi, The first patch fixes GIC and brings the secondary CPU. Unfortunately, this brings up to light an annoying issue with CPUIDLE driver. Changes since v1 ================ 1. Use one cast to obj in patch 1/5 (suggested by Peter). 2. Add Peter's review tags. 3. Re-order patches 3 and 4 to silence the checkpatch warning earlier. Overview of the problem ======================= On Exynos4210, by default Linux kernel uses cpuidle driver which tries to enter low power mode, called AFTR (Arm Off, Top Running). On real hardware this brings some power savings. This AFTR mode requires second CPU to be off, so the driver (coupled cpuidle driver) when system is idle: 1. Turns off second CPU, 2. Enters AFTR on CPU0. However the QEMU system is then totally unresponsive (e.g. on serial console) and RCU stalls appear from time to time. I spent some time on it and did not find the real cause behind the lag. Maybe it is because the second CPU does not really power down itself and system just burns the cycles under spin locks? Looking at recent stable kernels: - 3.10, 3.16 - works fine with two CPUs (no CPUIDLE driver), - 4.1 and newer - stalls and are unresponsive due to CPUIDLE being enabled. The cpuidle driver is not relying on DTS. It is just enabled in exynos_defconfig and works. Workarounds =========== 1. Boot with only 1 cpu 2. Build a kernel with disabled CONFIG_CPU_IDLE Best regards, Krzysztof Krzysztof Kozlowski (5): hw/intc/exynos4210_gic: Fix GIC memory mappings for secondary CPU hw/intc/exynos4210_gic: Use more meaningful name for local variable hw/timer/exynos4210_mct: Fix checkpatch style errors hw/timer/exynos4210_mct: Cleanup indentation and empty new lines hw/timer/exynos4210_mct: Remove unused defines hw/intc/exynos4210_gic.c | 33 +++++++++++++++++++------------ hw/timer/exynos4210_mct.c | 50 ++++++++++++++++++++--------------------------- 2 files changed, 41 insertions(+), 42 deletions(-) -- 2.9.3
Krzysztof Kozlowski <krzk@kernel.org> writes: > Hi, > <snip> > > Overview of the problem > ======================= > On Exynos4210, by default Linux kernel uses cpuidle driver which tries > to enter low power mode, called AFTR (Arm Off, Top Running). On real > hardware this brings some power savings. This AFTR mode requires second > CPU to be off, so the driver (coupled cpuidle driver) when system is idle: > 1. Turns off second CPU, > 2. Enters AFTR on CPU0. > > However the QEMU system is then totally unresponsive (e.g. on serial console) > and RCU stalls appear from time to time. I spent some time on it and did not > find the real cause behind the lag. Maybe it is because the second CPU > does not really power down itself and system just burns the cycles under spin > locks? Was this tested post the MTTCG merge? Maybe there is an interaction between power saving/halting the vCPU? I mention this because I already made some minor tweaks for handing the WFI instruction in MTTCG. See commit c22edfebff. You can test this by forcing the original behaviour by adding: -accel tcg,thread=single to your command line. > > Looking at recent stable kernels: > - 3.10, 3.16 - works fine with two CPUs (no CPUIDLE driver), > - 4.1 and newer - stalls and are unresponsive due to CPUIDLE being enabled. > > The cpuidle driver is not relying on DTS. It is just enabled in exynos_defconfig > and works. > > Workarounds > =========== > 1. Boot with only 1 cpu > 2. Build a kernel with disabled CONFIG_CPU_IDLE > > > > Best regards, > Krzysztof > > Krzysztof Kozlowski (5): > hw/intc/exynos4210_gic: Fix GIC memory mappings for secondary CPU > hw/intc/exynos4210_gic: Use more meaningful name for local variable > hw/timer/exynos4210_mct: Fix checkpatch style errors > hw/timer/exynos4210_mct: Cleanup indentation and empty new lines > hw/timer/exynos4210_mct: Remove unused defines > > hw/intc/exynos4210_gic.c | 33 +++++++++++++++++++------------ > hw/timer/exynos4210_mct.c | 50 ++++++++++++++++++++--------------------------- > 2 files changed, 41 insertions(+), 42 deletions(-) -- Alex Bennée
On Tue, Mar 14, 2017 at 04:55:34PM +0000, Alex Bennée wrote:
>
> Krzysztof Kozlowski <krzk@kernel.org> writes:
>
> > Hi,
> >
> <snip>
> >
> > Overview of the problem
> > =======================
> > On Exynos4210, by default Linux kernel uses cpuidle driver which tries
> > to enter low power mode, called AFTR (Arm Off, Top Running). On real
> > hardware this brings some power savings. This AFTR mode requires second
> > CPU to be off, so the driver (coupled cpuidle driver) when system is idle:
> > 1. Turns off second CPU,
> > 2. Enters AFTR on CPU0.
> >
> > However the QEMU system is then totally unresponsive (e.g. on serial console)
> > and RCU stalls appear from time to time. I spent some time on it and did not
> > find the real cause behind the lag. Maybe it is because the second CPU
> > does not really power down itself and system just burns the cycles under spin
> > locks?
>
> Was this tested post the MTTCG merge? Maybe there is an interaction
> between power saving/halting the vCPU?
>
> I mention this because I already made some minor tweaks for handing the
> WFI instruction in MTTCG. See commit c22edfebff. You can test this by
> forcing the original behaviour by adding:
>
> -accel tcg,thread=single
>
> to your command line.
>
I am testing it on v2.8.0-2005-g17783ac828ad. Without any -accel, the
system boots but responsiveness of serial console is bad. Bad but at
least it is responding still a little bit.
With the -accel you mentioned, system does not even reach user-space
(initramfs).
I tried rebasing on newer (e.g. v2.8.0-2182-g5e2fb7c598c6) but
compilation fails all the time on:
/home/dev/qemu/qemu/vl.c: In function ‘main’:
/home/dev/qemu/qemu/vl.c:3131:18: error: ‘QEMU_OPTION_blockdev’ undeclared (first use in this function)
case QEMU_OPTION_blockdev:
Best regards,
Krzysztof
That means you're on a pre-mttcg tree so that's not the reason. On 14 Mar 2017 5:35 pm, "Krzysztof Kozlowski" <krzk@kernel.org> wrote: > On Tue, Mar 14, 2017 at 04:55:34PM +0000, Alex Bennée wrote: > > > > Krzysztof Kozlowski <krzk@kernel.org> writes: > > > > > Hi, > > > > > <snip> > > > > > > Overview of the problem > > > ======================= > > > On Exynos4210, by default Linux kernel uses cpuidle driver which tries > > > to enter low power mode, called AFTR (Arm Off, Top Running). On real > > > hardware this brings some power savings. This AFTR mode requires > second > > > CPU to be off, so the driver (coupled cpuidle driver) when system is > idle: > > > 1. Turns off second CPU, > > > 2. Enters AFTR on CPU0. > > > > > > However the QEMU system is then totally unresponsive (e.g. on serial > console) > > > and RCU stalls appear from time to time. I spent some time on it and > did not > > > find the real cause behind the lag. Maybe it is because the second CPU > > > does not really power down itself and system just burns the cycles > under spin > > > locks? > > > > Was this tested post the MTTCG merge? Maybe there is an interaction > > between power saving/halting the vCPU? > > > > I mention this because I already made some minor tweaks for handing the > > WFI instruction in MTTCG. See commit c22edfebff. You can test this by > > forcing the original behaviour by adding: > > > > -accel tcg,thread=single > > > > to your command line. > > > > I am testing it on v2.8.0-2005-g17783ac828ad. Without any -accel, the > system boots but responsiveness of serial console is bad. Bad but at > least it is responding still a little bit. > > With the -accel you mentioned, system does not even reach user-space > (initramfs). > > I tried rebasing on newer (e.g. v2.8.0-2182-g5e2fb7c598c6) but > compilation fails all the time on: > /home/dev/qemu/qemu/vl.c: In function ‘main’: > /home/dev/qemu/qemu/vl.c:3131:18: error: ‘QEMU_OPTION_blockdev’ > undeclared (first use in this function) > case QEMU_OPTION_blockdev: > > > Best regards, > Krzysztof > >
On Tue, Mar 14, 2017 at 06:24:33PM +0000, Alex Bennée wrote: > That means you're on a pre-mttcg tree so that's not the reason. I managed to build it on current master (v2.8.0-2182-g5e2fb7c598c6). It behaves the same (with "-accel tcg,thread=single": cannot reach switching to init). Best regards, Krzysztof
Krzysztof Kozlowski <krzk@kernel.org> writes: > On Tue, Mar 14, 2017 at 06:24:33PM +0000, Alex Bennée wrote: >> That means you're on a pre-mttcg tree so that's not the reason. > > I managed to build it on current master (v2.8.0-2182-g5e2fb7c598c6). It > behaves the same (with "-accel tcg,thread=single": cannot reach > switching to init). Interesting. Can you post your exact qemu command line and I'll have a go at reproducing. In the meantime you can always --enable-debug-tcg your build and see if any of the asserts fire. > > Best regards, > Krzysztof -- Alex Bennée
On Wed, Mar 15, 2017 at 08:05:56AM +0000, Alex Bennée wrote: > > Krzysztof Kozlowski <krzk@kernel.org> writes: > > > On Tue, Mar 14, 2017 at 06:24:33PM +0000, Alex Bennée wrote: > >> That means you're on a pre-mttcg tree so that's not the reason. > > > > I managed to build it on current master (v2.8.0-2182-g5e2fb7c598c6). It > > behaves the same (with "-accel tcg,thread=single": cannot reach > > switching to init). > > Interesting. Can you post your exact qemu command line and I'll have a > go at reproducing. I build usually with: ../qemu/configure --cc="ccache cc" --cxx="ccache c++" --enable-debug --enable-fdt \ --enable-kvm --enable-libusb --enable-libssh2 --enable-lzo \ --enable-bzip2 --enable-curses --enable-gtk --enable-cap-ng Running with command: arm-softmmu/qemu-system-arm -m 1024 -M smdkc210 -smp 2 -append \ 'console=ttySAC0,115200n8 console=ttyS0 earlyprintk' -d guest_errors \ -serial stdio -D ..//log-smdkc210.log -kernel ../cur-linux/zImage -dtb ../cur-linux/dts/exynos4210-smdkv310.dtb -initrd ../armv7-odroidu3-exynos-v4.10-initramfs.cpio.gz Plus the -accel line, depending on the needs. The initramfs is a simple one from Arch Arm. > > In the meantime you can always --enable-debug-tcg your build and see if > any of the asserts fire. enable-debug-tcg did not change much - no asserts. The last dmesg is: [ 72.616013] Registering SWP/SWPB emulation handler [ 74.484873] s3c64xx-spi 13940000.spi: spi bus clock parent not specified, using clock at index 0 as parent [ 74.485994] s3c64xx-spi 13940000.spi: number of chip select lines not specified, assuming 1 chip select line [ 74.493249] s3c64xx-spi 13940000.spi: Failed to get RX DMA channel [ 74.503885] hctosys: unable to open rtc device (rtc0) [ 74.523004] ALSA device list: [ 74.523711] No soundcards found. [ 74.542278] samsung-uart 13800000.serial: DMA request failed, DMA will not be used Best regards, Krzysztof
© 2016 - 2025 Red Hat, Inc.