From nobody Mon May 20 13:01:29 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1560134937; cv=none; d=zoho.com; s=zohoarc; b=na7pZt8vTKNwJoVqmQiYFiLl18YAIKAWZRxR7cQGPuDQoPZUuXIOoblX9jDkx6twhUP7pvV+XlD3eQNvfeTB9vF9fpvp79k9ztQjmGHyv4sAnPgcrAV11P6iw1v+XECHoFjz5zkjH3DD66nOhwNGhLXRYG9IHFhrPaXMuy8diOk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1560134937; h=Content-Transfer-Encoding:Cc:Date:From:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:To:ARC-Authentication-Results; bh=zBdGRlWryzQ+sbGmyV9PdLWkai7VP8+7eCSgHpNc27c=; b=LiJwlf4PDzIMAOr2l9xJF8vlvnx1ch3b+kB1D0yRT+Ezdr8nYDcnqfgDbiC2+EpKtuuIan5NqOXZwBWZLa/L+z7CIvT9uVHiP4lfZH13ug8F9TSZzFdpkRj4np57Cw5nusO2/bUYz8qdARnqkUXz4dFK9bpH5eXJTwhsHci2kGQ= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 156013493767737.99582916490715; Sun, 9 Jun 2019 19:48:57 -0700 (PDT) Received: from localhost ([::1]:39964 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1haAMf-0002kD-E8 for importer@patchew.org; Sun, 09 Jun 2019 22:48:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56646) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1ha9uW-0005fI-3E for qemu-devel@nongnu.org; Sun, 09 Jun 2019 22:19:49 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ha9uU-0000mx-Ub for qemu-devel@nongnu.org; Sun, 09 Jun 2019 22:19:48 -0400 Received: from mga11.intel.com ([192.55.52.93]:11390) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ha9uU-0000lx-Mj for qemu-devel@nongnu.org; Sun, 09 Jun 2019 22:19:46 -0400 Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 09 Jun 2019 19:19:42 -0700 Received: from coxu-arch-shz.sh.intel.com ([10.239.160.21]) by orsmga008.jf.intel.com with ESMTP; 09 Jun 2019 19:19:41 -0700 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-ExtLoop1: 1 From: Colin Xu To: qemu-devel@nongnu.org Date: Mon, 10 Jun 2019 10:19:39 +0800 Message-Id: <20190610021939.13669-1-colin.xu@intel.com> X-Mailer: git-send-email 2.21.0 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 192.55.52.93 Subject: [Qemu-devel] [PATCH] hax: Honor CPUState::halted X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: colin.xu@intel.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" QEMU tracks whether a vcpu is halted using CPUState::halted. E.g., after initialization or reset, halted is 0 for the BSP (vcpu 0) and 1 for the APs (vcpu 1, 2, ...). A halted vcpu should not be handed to the hypervisor to run (e.g. hax_vcpu_run()). Under HAXM, Android Emulator sometimes boots into a "vcpu shutdown request" error while executing in SeaBIOS, with the HAXM driver logging a guest triple fault in vcpu 1, 2, ... at RIP 0x3. That is ultimately because the HAX accelerator asks HAXM to run those APs when they are still in the halted state. Normally, the vcpu thread for an AP will start by looping in qemu_wait_io_event(), until the BSP kicks it via a pair of IPIs (INIT followed by SIPI). But because the HAX accelerator does not honor cpu->halted, it allows the AP vcpu thread to proceed to hax_vcpu_run() as soon as it receives any kick, even if the kick does not come from the BSP. It turns out that emulator has a worker thread which periodically kicks every vcpu thread (possibly to collect CPU usage data), and if one of these kicks comes before those by the BSP, the AP will start execution from the wrong RIP, resulting in the aforementioned SMP boot failure. The solution is inspired by the KVM accelerator (credit to Chuanxiao Dong for the pointer): 1. Get rid of questionable logic that unconditionally resets cpu->halted before hax_vcpu_run(). Instead, only reset it at the right moments (there are only a few "unhalt" events). 2. Add a check for cpu->halted before hax_vcpu_run(). Note that although the non-Unrestricted Guest (!ug_platform) code path also forcibly resets cpu->halted, it is left untouched, because only the UG code path supports SMP guests. The patch is first merged to android emulator with Change-Id: I9c5752cc737fd305d7eace1768ea12a07309d716 Cc: Yu Ning Cc: Chuanxiao Dong Signed-off-by: Colin Xu --- cpus.c | 1 - target/i386/hax-all.c | 36 ++++++++++++++++++++++++++++++++++-- 2 files changed, 34 insertions(+), 3 deletions(-) diff --git a/cpus.c b/cpus.c index ffc57119ca5e..c1a56cd9ab01 100644 --- a/cpus.c +++ b/cpus.c @@ -1591,7 +1591,6 @@ static void *qemu_hax_cpu_thread_fn(void *arg) =20 cpu->thread_id =3D qemu_get_thread_id(); cpu->created =3D true; - cpu->halted =3D 0; current_cpu =3D cpu; =20 hax_init_vcpu(cpu); diff --git a/target/i386/hax-all.c b/target/i386/hax-all.c index 44b89c1d74ae..58a27b475ec8 100644 --- a/target/i386/hax-all.c +++ b/target/i386/hax-all.c @@ -471,13 +471,35 @@ static int hax_vcpu_hax_exec(CPUArchState *env) return 0; } =20 - cpu->halted =3D 0; - if (cpu->interrupt_request & CPU_INTERRUPT_POLL) { cpu->interrupt_request &=3D ~CPU_INTERRUPT_POLL; apic_poll_irq(x86_cpu->apic_state); } =20 + /* After a vcpu is halted (either because it is an AP and has just been + * reset, or because it has executed the HLT instruction), it will not= be + * run (hax_vcpu_run()) until it is unhalted. The next few if blocks c= heck + * for events that may change the halted state of this vcpu: + * a) Maskable interrupt, when RFLAGS.IF is 1; + * Note: env->eflags may not reflect the current RFLAGS state, bec= ause + * it is not updated after each hax_vcpu_run(). We cannot af= ford + * to fail to recognize any unhalt-by-maskable-interrupt eve= nt + * (in which case the vcpu will halt forever), and yet we ca= nnot + * afford the overhead of hax_vcpu_sync_state(). The current + * solution is to err on the side of caution and have the HLT + * handler (see case HAX_EXIT_HLT below) unconditionally set= the + * IF_MASK bit in env->eflags, which, in effect, disables the + * RFLAGS.IF check. + * b) NMI; + * c) INIT signal; + * d) SIPI signal. + */ + if (((cpu->interrupt_request & CPU_INTERRUPT_HARD) && + (env->eflags & IF_MASK)) || + (cpu->interrupt_request & CPU_INTERRUPT_NMI)) { + cpu->halted =3D 0; + } + if (cpu->interrupt_request & CPU_INTERRUPT_INIT) { DPRINTF("\nhax_vcpu_hax_exec: handling INIT for %d\n", cpu->cpu_index); @@ -493,6 +515,16 @@ static int hax_vcpu_hax_exec(CPUArchState *env) hax_vcpu_sync_state(env, 1); } =20 + if (cpu->halted) { + /* If this vcpu is halted, we must not ask HAXM to run it. Instead= , we + * break out of hax_smp_cpu_exec() as if this vcpu had executed HL= T. + * That way, this vcpu thread will be trapped in qemu_wait_io_even= t(), + * until the vcpu is unhalted. + */ + cpu->exception_index =3D EXCP_HLT; + return 0; + } + do { int hax_ret; =20 --=20 2.21.0