From nobody Sun Sep 29 02:26:20 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1614251034; cv=none; d=zohomail.com; s=zohoarc; b=h5nMF4hAA/wNDYdq4Xlo35obhAMpZnQBIj9p3C/Rzw6Mpf3Xy+KVZBhUegyF3vR+e/XLgT64VV/eJk9NDoYMK2g8VqbFx7EM98pe0hdN6+LDLZnz42Kr72Q9vnexb8GLxW0su2iDBisjVEoYt49lP+a5SZlEh1RYAr83ZNB1aWc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1614251034; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=ncjMp4E4zev3b8VcxrD0qNmouCr6rzxQIlBE1cxMgOc=; b=Lo+6DgrJjPpGxb55lmxa7kkn0X0XTcCnqYuHYrF3/4hOxsv6FNVhmL4skkfiEw88UDb9j2lfeXaV4xWOPqdqqBzdF0UliRI2R9+e/Cd2/aE3xZZbH71E6LcqyEWboGvDtXpOvf4Oh8UGiaQbVOHZkNJB0WSyuiLfhzJnAJOLQm4= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1614251034422994.1291523745008; Thu, 25 Feb 2021 03:03:54 -0800 (PST) Received: from localhost ([::1]:58158 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lFEQy-000368-Qd for importer@patchew.org; Thu, 25 Feb 2021 06:03:52 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:53112) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lFEDg-0002ZT-5v for qemu-devel@nongnu.org; Thu, 25 Feb 2021 05:50:08 -0500 Received: from mx2.suse.de ([195.135.220.15]:40858) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lFEDc-0003Nx-4s for qemu-devel@nongnu.org; Thu, 25 Feb 2021 05:50:07 -0500 Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id C4235AE05; Thu, 25 Feb 2021 10:49:50 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de From: Claudio Fontana To: Paolo Bonzini , Richard Henderson , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Eduardo Habkost , Peter Maydell , =?UTF-8?q?Alex=20Benn=C3=A9e?= Subject: [PATCH v23 09/17] i386: split tcg excp_helper into sysemu and user parts Date: Thu, 25 Feb 2021 11:49:33 +0100 Message-Id: <20210225104941.15688-10-cfontana@suse.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210225104941.15688-1-cfontana@suse.de> References: <20210225104941.15688-1-cfontana@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=195.135.220.15; envelope-from=cfontana@suse.de; helo=mx2.suse.de X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Thomas Huth , Roman Bolshakov , Claudio Fontana , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Signed-off-by: Claudio Fontana Reviewed-by: Richard Henderson --- target/i386/tcg/excp_helper.c | 572 -------------------------- target/i386/tcg/sysemu/excp_helper.c | 581 +++++++++++++++++++++++++++ target/i386/tcg/user/excp_helper.c | 39 ++ target/i386/tcg/sysemu/meson.build | 1 + target/i386/tcg/user/meson.build | 1 + 5 files changed, 622 insertions(+), 572 deletions(-) create mode 100644 target/i386/tcg/sysemu/excp_helper.c create mode 100644 target/i386/tcg/user/excp_helper.c diff --git a/target/i386/tcg/excp_helper.c b/target/i386/tcg/excp_helper.c index b7d6259e4a..0183f3932e 100644 --- a/target/i386/tcg/excp_helper.c +++ b/target/i386/tcg/excp_helper.c @@ -137,575 +137,3 @@ void raise_exception_ra(CPUX86State *env, int excepti= on_index, uintptr_t retaddr { raise_interrupt2(env, exception_index, 0, 0, 0, retaddr); } - -#if !defined(CONFIG_USER_ONLY) -static hwaddr get_hphys(CPUState *cs, hwaddr gphys, MMUAccessType access_t= ype, - int *prot) -{ - CPUX86State *env =3D &X86_CPU(cs)->env; - uint64_t rsvd_mask =3D PG_HI_RSVD_MASK; - uint64_t ptep, pte; - uint64_t exit_info_1 =3D 0; - target_ulong pde_addr, pte_addr; - uint32_t page_offset; - int page_size; - - if (likely(!(env->hflags2 & HF2_NPT_MASK))) { - return gphys; - } - - if (!(env->nested_pg_mode & SVM_NPT_NXE)) { - rsvd_mask |=3D PG_NX_MASK; - } - - if (env->nested_pg_mode & SVM_NPT_PAE) { - uint64_t pde, pdpe; - target_ulong pdpe_addr; - -#ifdef TARGET_X86_64 - if (env->nested_pg_mode & SVM_NPT_LMA) { - uint64_t pml5e; - uint64_t pml4e_addr, pml4e; - - pml5e =3D env->nested_cr3; - ptep =3D PG_NX_MASK | PG_USER_MASK | PG_RW_MASK; - - pml4e_addr =3D (pml5e & PG_ADDRESS_MASK) + - (((gphys >> 39) & 0x1ff) << 3); - pml4e =3D x86_ldq_phys(cs, pml4e_addr); - if (!(pml4e & PG_PRESENT_MASK)) { - goto do_fault; - } - if (pml4e & (rsvd_mask | PG_PSE_MASK)) { - goto do_fault_rsvd; - } - if (!(pml4e & PG_ACCESSED_MASK)) { - pml4e |=3D PG_ACCESSED_MASK; - x86_stl_phys_notdirty(cs, pml4e_addr, pml4e); - } - ptep &=3D pml4e ^ PG_NX_MASK; - pdpe_addr =3D (pml4e & PG_ADDRESS_MASK) + - (((gphys >> 30) & 0x1ff) << 3); - pdpe =3D x86_ldq_phys(cs, pdpe_addr); - if (!(pdpe & PG_PRESENT_MASK)) { - goto do_fault; - } - if (pdpe & rsvd_mask) { - goto do_fault_rsvd; - } - ptep &=3D pdpe ^ PG_NX_MASK; - if (!(pdpe & PG_ACCESSED_MASK)) { - pdpe |=3D PG_ACCESSED_MASK; - x86_stl_phys_notdirty(cs, pdpe_addr, pdpe); - } - if (pdpe & PG_PSE_MASK) { - /* 1 GB page */ - page_size =3D 1024 * 1024 * 1024; - pte_addr =3D pdpe_addr; - pte =3D pdpe; - goto do_check_protect; - } - } else -#endif - { - pdpe_addr =3D (env->nested_cr3 & ~0x1f) + ((gphys >> 27) & 0x1= 8); - pdpe =3D x86_ldq_phys(cs, pdpe_addr); - if (!(pdpe & PG_PRESENT_MASK)) { - goto do_fault; - } - rsvd_mask |=3D PG_HI_USER_MASK; - if (pdpe & (rsvd_mask | PG_NX_MASK)) { - goto do_fault_rsvd; - } - ptep =3D PG_NX_MASK | PG_USER_MASK | PG_RW_MASK; - } - - pde_addr =3D (pdpe & PG_ADDRESS_MASK) + (((gphys >> 21) & 0x1ff) <= < 3); - pde =3D x86_ldq_phys(cs, pde_addr); - if (!(pde & PG_PRESENT_MASK)) { - goto do_fault; - } - if (pde & rsvd_mask) { - goto do_fault_rsvd; - } - ptep &=3D pde ^ PG_NX_MASK; - if (pde & PG_PSE_MASK) { - /* 2 MB page */ - page_size =3D 2048 * 1024; - pte_addr =3D pde_addr; - pte =3D pde; - goto do_check_protect; - } - /* 4 KB page */ - if (!(pde & PG_ACCESSED_MASK)) { - pde |=3D PG_ACCESSED_MASK; - x86_stl_phys_notdirty(cs, pde_addr, pde); - } - pte_addr =3D (pde & PG_ADDRESS_MASK) + (((gphys >> 12) & 0x1ff) <<= 3); - pte =3D x86_ldq_phys(cs, pte_addr); - if (!(pte & PG_PRESENT_MASK)) { - goto do_fault; - } - if (pte & rsvd_mask) { - goto do_fault_rsvd; - } - /* combine pde and pte nx, user and rw protections */ - ptep &=3D pte ^ PG_NX_MASK; - page_size =3D 4096; - } else { - uint32_t pde; - - /* page directory entry */ - pde_addr =3D (env->nested_cr3 & ~0xfff) + ((gphys >> 20) & 0xffc); - pde =3D x86_ldl_phys(cs, pde_addr); - if (!(pde & PG_PRESENT_MASK)) { - goto do_fault; - } - ptep =3D pde | PG_NX_MASK; - - /* if host cr4 PSE bit is set, then we use a 4MB page */ - if ((pde & PG_PSE_MASK) && (env->nested_pg_mode & SVM_NPT_PSE)) { - page_size =3D 4096 * 1024; - pte_addr =3D pde_addr; - - /* Bits 20-13 provide bits 39-32 of the address, bit 21 is res= erved. - * Leave bits 20-13 in place for setting accessed/dirty bits b= elow. - */ - pte =3D pde | ((pde & 0x1fe000LL) << (32 - 13)); - rsvd_mask =3D 0x200000; - goto do_check_protect_pse36; - } - - if (!(pde & PG_ACCESSED_MASK)) { - pde |=3D PG_ACCESSED_MASK; - x86_stl_phys_notdirty(cs, pde_addr, pde); - } - - /* page directory entry */ - pte_addr =3D (pde & ~0xfff) + ((gphys >> 10) & 0xffc); - pte =3D x86_ldl_phys(cs, pte_addr); - if (!(pte & PG_PRESENT_MASK)) { - goto do_fault; - } - /* combine pde and pte user and rw protections */ - ptep &=3D pte | PG_NX_MASK; - page_size =3D 4096; - rsvd_mask =3D 0; - } - - do_check_protect: - rsvd_mask |=3D (page_size - 1) & PG_ADDRESS_MASK & ~PG_PSE_PAT_MASK; - do_check_protect_pse36: - if (pte & rsvd_mask) { - goto do_fault_rsvd; - } - ptep ^=3D PG_NX_MASK; - - if (!(ptep & PG_USER_MASK)) { - goto do_fault_protect; - } - if (ptep & PG_NX_MASK) { - if (access_type =3D=3D MMU_INST_FETCH) { - goto do_fault_protect; - } - *prot &=3D ~PAGE_EXEC; - } - if (!(ptep & PG_RW_MASK)) { - if (access_type =3D=3D MMU_DATA_STORE) { - goto do_fault_protect; - } - *prot &=3D ~PAGE_WRITE; - } - - pte &=3D PG_ADDRESS_MASK & ~(page_size - 1); - page_offset =3D gphys & (page_size - 1); - return pte + page_offset; - - do_fault_rsvd: - exit_info_1 |=3D SVM_NPTEXIT_RSVD; - do_fault_protect: - exit_info_1 |=3D SVM_NPTEXIT_P; - do_fault: - x86_stq_phys(cs, env->vm_vmcb + offsetof(struct vmcb, control.exit_inf= o_2), - gphys); - exit_info_1 |=3D SVM_NPTEXIT_US; - if (access_type =3D=3D MMU_DATA_STORE) { - exit_info_1 |=3D SVM_NPTEXIT_RW; - } else if (access_type =3D=3D MMU_INST_FETCH) { - exit_info_1 |=3D SVM_NPTEXIT_ID; - } - if (prot) { - exit_info_1 |=3D SVM_NPTEXIT_GPA; - } else { /* page table access */ - exit_info_1 |=3D SVM_NPTEXIT_GPT; - } - cpu_vmexit(env, SVM_EXIT_NPF, exit_info_1, env->retaddr); -} - -/* return value: - * -1 =3D cannot handle fault - * 0 =3D nothing more to do - * 1 =3D generate PF fault - */ -static int handle_mmu_fault(CPUState *cs, vaddr addr, int size, - int is_write1, int mmu_idx) -{ - X86CPU *cpu =3D X86_CPU(cs); - CPUX86State *env =3D &cpu->env; - uint64_t ptep, pte; - int32_t a20_mask; - target_ulong pde_addr, pte_addr; - int error_code =3D 0; - int is_dirty, prot, page_size, is_write, is_user; - hwaddr paddr; - uint64_t rsvd_mask =3D PG_HI_RSVD_MASK; - uint32_t page_offset; - target_ulong vaddr; - uint32_t pkr; - - is_user =3D mmu_idx =3D=3D MMU_USER_IDX; -#if defined(DEBUG_MMU) - printf("MMU fault: addr=3D%" VADDR_PRIx " w=3D%d u=3D%d eip=3D" TARGET= _FMT_lx "\n", - addr, is_write1, is_user, env->eip); -#endif - is_write =3D is_write1 & 1; - - a20_mask =3D x86_get_a20_mask(env); - if (!(env->cr[0] & CR0_PG_MASK)) { - pte =3D addr; -#ifdef TARGET_X86_64 - if (!(env->hflags & HF_LMA_MASK)) { - /* Without long mode we can only address 32bits in real mode */ - pte =3D (uint32_t)pte; - } -#endif - prot =3D PAGE_READ | PAGE_WRITE | PAGE_EXEC; - page_size =3D 4096; - goto do_mapping; - } - - if (!(env->efer & MSR_EFER_NXE)) { - rsvd_mask |=3D PG_NX_MASK; - } - - if (env->cr[4] & CR4_PAE_MASK) { - uint64_t pde, pdpe; - target_ulong pdpe_addr; - -#ifdef TARGET_X86_64 - if (env->hflags & HF_LMA_MASK) { - bool la57 =3D env->cr[4] & CR4_LA57_MASK; - uint64_t pml5e_addr, pml5e; - uint64_t pml4e_addr, pml4e; - int32_t sext; - - /* test virtual address sign extension */ - sext =3D la57 ? (int64_t)addr >> 56 : (int64_t)addr >> 47; - if (sext !=3D 0 && sext !=3D -1) { - env->error_code =3D 0; - cs->exception_index =3D EXCP0D_GPF; - return 1; - } - - if (la57) { - pml5e_addr =3D ((env->cr[3] & ~0xfff) + - (((addr >> 48) & 0x1ff) << 3)) & a20_mask; - pml5e_addr =3D get_hphys(cs, pml5e_addr, MMU_DATA_STORE, N= ULL); - pml5e =3D x86_ldq_phys(cs, pml5e_addr); - if (!(pml5e & PG_PRESENT_MASK)) { - goto do_fault; - } - if (pml5e & (rsvd_mask | PG_PSE_MASK)) { - goto do_fault_rsvd; - } - if (!(pml5e & PG_ACCESSED_MASK)) { - pml5e |=3D PG_ACCESSED_MASK; - x86_stl_phys_notdirty(cs, pml5e_addr, pml5e); - } - ptep =3D pml5e ^ PG_NX_MASK; - } else { - pml5e =3D env->cr[3]; - ptep =3D PG_NX_MASK | PG_USER_MASK | PG_RW_MASK; - } - - pml4e_addr =3D ((pml5e & PG_ADDRESS_MASK) + - (((addr >> 39) & 0x1ff) << 3)) & a20_mask; - pml4e_addr =3D get_hphys(cs, pml4e_addr, MMU_DATA_STORE, false= ); - pml4e =3D x86_ldq_phys(cs, pml4e_addr); - if (!(pml4e & PG_PRESENT_MASK)) { - goto do_fault; - } - if (pml4e & (rsvd_mask | PG_PSE_MASK)) { - goto do_fault_rsvd; - } - if (!(pml4e & PG_ACCESSED_MASK)) { - pml4e |=3D PG_ACCESSED_MASK; - x86_stl_phys_notdirty(cs, pml4e_addr, pml4e); - } - ptep &=3D pml4e ^ PG_NX_MASK; - pdpe_addr =3D ((pml4e & PG_ADDRESS_MASK) + (((addr >> 30) & 0x= 1ff) << 3)) & - a20_mask; - pdpe_addr =3D get_hphys(cs, pdpe_addr, MMU_DATA_STORE, NULL); - pdpe =3D x86_ldq_phys(cs, pdpe_addr); - if (!(pdpe & PG_PRESENT_MASK)) { - goto do_fault; - } - if (pdpe & rsvd_mask) { - goto do_fault_rsvd; - } - ptep &=3D pdpe ^ PG_NX_MASK; - if (!(pdpe & PG_ACCESSED_MASK)) { - pdpe |=3D PG_ACCESSED_MASK; - x86_stl_phys_notdirty(cs, pdpe_addr, pdpe); - } - if (pdpe & PG_PSE_MASK) { - /* 1 GB page */ - page_size =3D 1024 * 1024 * 1024; - pte_addr =3D pdpe_addr; - pte =3D pdpe; - goto do_check_protect; - } - } else -#endif - { - /* XXX: load them when cr3 is loaded ? */ - pdpe_addr =3D ((env->cr[3] & ~0x1f) + ((addr >> 27) & 0x18)) & - a20_mask; - pdpe_addr =3D get_hphys(cs, pdpe_addr, MMU_DATA_STORE, false); - pdpe =3D x86_ldq_phys(cs, pdpe_addr); - if (!(pdpe & PG_PRESENT_MASK)) { - goto do_fault; - } - rsvd_mask |=3D PG_HI_USER_MASK; - if (pdpe & (rsvd_mask | PG_NX_MASK)) { - goto do_fault_rsvd; - } - ptep =3D PG_NX_MASK | PG_USER_MASK | PG_RW_MASK; - } - - pde_addr =3D ((pdpe & PG_ADDRESS_MASK) + (((addr >> 21) & 0x1ff) <= < 3)) & - a20_mask; - pde_addr =3D get_hphys(cs, pde_addr, MMU_DATA_STORE, NULL); - pde =3D x86_ldq_phys(cs, pde_addr); - if (!(pde & PG_PRESENT_MASK)) { - goto do_fault; - } - if (pde & rsvd_mask) { - goto do_fault_rsvd; - } - ptep &=3D pde ^ PG_NX_MASK; - if (pde & PG_PSE_MASK) { - /* 2 MB page */ - page_size =3D 2048 * 1024; - pte_addr =3D pde_addr; - pte =3D pde; - goto do_check_protect; - } - /* 4 KB page */ - if (!(pde & PG_ACCESSED_MASK)) { - pde |=3D PG_ACCESSED_MASK; - x86_stl_phys_notdirty(cs, pde_addr, pde); - } - pte_addr =3D ((pde & PG_ADDRESS_MASK) + (((addr >> 12) & 0x1ff) <<= 3)) & - a20_mask; - pte_addr =3D get_hphys(cs, pte_addr, MMU_DATA_STORE, NULL); - pte =3D x86_ldq_phys(cs, pte_addr); - if (!(pte & PG_PRESENT_MASK)) { - goto do_fault; - } - if (pte & rsvd_mask) { - goto do_fault_rsvd; - } - /* combine pde and pte nx, user and rw protections */ - ptep &=3D pte ^ PG_NX_MASK; - page_size =3D 4096; - } else { - uint32_t pde; - - /* page directory entry */ - pde_addr =3D ((env->cr[3] & ~0xfff) + ((addr >> 20) & 0xffc)) & - a20_mask; - pde_addr =3D get_hphys(cs, pde_addr, MMU_DATA_STORE, NULL); - pde =3D x86_ldl_phys(cs, pde_addr); - if (!(pde & PG_PRESENT_MASK)) { - goto do_fault; - } - ptep =3D pde | PG_NX_MASK; - - /* if PSE bit is set, then we use a 4MB page */ - if ((pde & PG_PSE_MASK) && (env->cr[4] & CR4_PSE_MASK)) { - page_size =3D 4096 * 1024; - pte_addr =3D pde_addr; - - /* Bits 20-13 provide bits 39-32 of the address, bit 21 is res= erved. - * Leave bits 20-13 in place for setting accessed/dirty bits b= elow. - */ - pte =3D pde | ((pde & 0x1fe000LL) << (32 - 13)); - rsvd_mask =3D 0x200000; - goto do_check_protect_pse36; - } - - if (!(pde & PG_ACCESSED_MASK)) { - pde |=3D PG_ACCESSED_MASK; - x86_stl_phys_notdirty(cs, pde_addr, pde); - } - - /* page directory entry */ - pte_addr =3D ((pde & ~0xfff) + ((addr >> 10) & 0xffc)) & - a20_mask; - pte_addr =3D get_hphys(cs, pte_addr, MMU_DATA_STORE, NULL); - pte =3D x86_ldl_phys(cs, pte_addr); - if (!(pte & PG_PRESENT_MASK)) { - goto do_fault; - } - /* combine pde and pte user and rw protections */ - ptep &=3D pte | PG_NX_MASK; - page_size =3D 4096; - rsvd_mask =3D 0; - } - -do_check_protect: - rsvd_mask |=3D (page_size - 1) & PG_ADDRESS_MASK & ~PG_PSE_PAT_MASK; -do_check_protect_pse36: - if (pte & rsvd_mask) { - goto do_fault_rsvd; - } - ptep ^=3D PG_NX_MASK; - - /* can the page can be put in the TLB? prot will tell us */ - if (is_user && !(ptep & PG_USER_MASK)) { - goto do_fault_protect; - } - - prot =3D 0; - if (mmu_idx !=3D MMU_KSMAP_IDX || !(ptep & PG_USER_MASK)) { - prot |=3D PAGE_READ; - if ((ptep & PG_RW_MASK) || (!is_user && !(env->cr[0] & CR0_WP_MASK= ))) { - prot |=3D PAGE_WRITE; - } - } - if (!(ptep & PG_NX_MASK) && - (mmu_idx =3D=3D MMU_USER_IDX || - !((env->cr[4] & CR4_SMEP_MASK) && (ptep & PG_USER_MASK)))) { - prot |=3D PAGE_EXEC; - } - - if (!(env->hflags & HF_LMA_MASK)) { - pkr =3D 0; - } else if (ptep & PG_USER_MASK) { - pkr =3D env->cr[4] & CR4_PKE_MASK ? env->pkru : 0; - } else { - pkr =3D env->cr[4] & CR4_PKS_MASK ? env->pkrs : 0; - } - if (pkr) { - uint32_t pk =3D (pte & PG_PKRU_MASK) >> PG_PKRU_BIT; - uint32_t pkr_ad =3D (pkr >> pk * 2) & 1; - uint32_t pkr_wd =3D (pkr >> pk * 2) & 2; - uint32_t pkr_prot =3D PAGE_READ | PAGE_WRITE | PAGE_EXEC; - - if (pkr_ad) { - pkr_prot &=3D ~(PAGE_READ | PAGE_WRITE); - } else if (pkr_wd && (is_user || env->cr[0] & CR0_WP_MASK)) { - pkr_prot &=3D ~PAGE_WRITE; - } - - prot &=3D pkr_prot; - if ((pkr_prot & (1 << is_write1)) =3D=3D 0) { - assert(is_write1 !=3D 2); - error_code |=3D PG_ERROR_PK_MASK; - goto do_fault_protect; - } - } - - if ((prot & (1 << is_write1)) =3D=3D 0) { - goto do_fault_protect; - } - - /* yes, it can! */ - is_dirty =3D is_write && !(pte & PG_DIRTY_MASK); - if (!(pte & PG_ACCESSED_MASK) || is_dirty) { - pte |=3D PG_ACCESSED_MASK; - if (is_dirty) { - pte |=3D PG_DIRTY_MASK; - } - x86_stl_phys_notdirty(cs, pte_addr, pte); - } - - if (!(pte & PG_DIRTY_MASK)) { - /* only set write access if already dirty... otherwise wait - for dirty access */ - assert(!is_write); - prot &=3D ~PAGE_WRITE; - } - - do_mapping: - pte =3D pte & a20_mask; - - /* align to page_size */ - pte &=3D PG_ADDRESS_MASK & ~(page_size - 1); - page_offset =3D addr & (page_size - 1); - paddr =3D get_hphys(cs, pte + page_offset, is_write1, &prot); - - /* Even if 4MB pages, we map only one 4KB page in the cache to - avoid filling it too fast */ - vaddr =3D addr & TARGET_PAGE_MASK; - paddr &=3D TARGET_PAGE_MASK; - - assert(prot & (1 << is_write1)); - tlb_set_page_with_attrs(cs, vaddr, paddr, cpu_get_mem_attrs(env), - prot, mmu_idx, page_size); - return 0; - do_fault_rsvd: - error_code |=3D PG_ERROR_RSVD_MASK; - do_fault_protect: - error_code |=3D PG_ERROR_P_MASK; - do_fault: - error_code |=3D (is_write << PG_ERROR_W_BIT); - if (is_user) - error_code |=3D PG_ERROR_U_MASK; - if (is_write1 =3D=3D 2 && - (((env->efer & MSR_EFER_NXE) && - (env->cr[4] & CR4_PAE_MASK)) || - (env->cr[4] & CR4_SMEP_MASK))) - error_code |=3D PG_ERROR_I_D_MASK; - if (env->intercept_exceptions & (1 << EXCP0E_PAGE)) { - /* cr2 is not modified in case of exceptions */ - x86_stq_phys(cs, - env->vm_vmcb + offsetof(struct vmcb, control.exit_info_2), - addr); - } else { - env->cr[2] =3D addr; - } - env->error_code =3D error_code; - cs->exception_index =3D EXCP0E_PAGE; - return 1; -} -#endif - -bool x86_cpu_tlb_fill(CPUState *cs, vaddr addr, int size, - MMUAccessType access_type, int mmu_idx, - bool probe, uintptr_t retaddr) -{ - X86CPU *cpu =3D X86_CPU(cs); - CPUX86State *env =3D &cpu->env; - -#ifdef CONFIG_USER_ONLY - /* user mode only emulation */ - env->cr[2] =3D addr; - env->error_code =3D (access_type =3D=3D MMU_DATA_STORE) << PG_ERROR_W_= BIT; - env->error_code |=3D PG_ERROR_U_MASK; - cs->exception_index =3D EXCP0E_PAGE; - env->exception_is_int =3D 0; - env->exception_next_eip =3D -1; - cpu_loop_exit_restore(cs, retaddr); -#else - env->retaddr =3D retaddr; - if (handle_mmu_fault(cs, addr, size, access_type, mmu_idx)) { - /* FIXME: On error in get_hphys we have already jumped out. */ - g_assert(!probe); - raise_exception_err_ra(env, cs->exception_index, - env->error_code, retaddr); - } - return true; -#endif -} diff --git a/target/i386/tcg/sysemu/excp_helper.c b/target/i386/tcg/sysemu/= excp_helper.c new file mode 100644 index 0000000000..37e6eead05 --- /dev/null +++ b/target/i386/tcg/sysemu/excp_helper.c @@ -0,0 +1,581 @@ +/* + * x86 exception helpers - sysemu code + * + * Copyright (c) 2003 Fabrice Bellard + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, see . + */ + +#include "qemu/osdep.h" +#include "cpu.h" +#include "tcg/helper-tcg.h" + +static hwaddr get_hphys(CPUState *cs, hwaddr gphys, MMUAccessType access_t= ype, + int *prot) +{ + CPUX86State *env =3D &X86_CPU(cs)->env; + uint64_t rsvd_mask =3D PG_HI_RSVD_MASK; + uint64_t ptep, pte; + uint64_t exit_info_1 =3D 0; + target_ulong pde_addr, pte_addr; + uint32_t page_offset; + int page_size; + + if (likely(!(env->hflags2 & HF2_NPT_MASK))) { + return gphys; + } + + if (!(env->nested_pg_mode & SVM_NPT_NXE)) { + rsvd_mask |=3D PG_NX_MASK; + } + + if (env->nested_pg_mode & SVM_NPT_PAE) { + uint64_t pde, pdpe; + target_ulong pdpe_addr; + +#ifdef TARGET_X86_64 + if (env->nested_pg_mode & SVM_NPT_LMA) { + uint64_t pml5e; + uint64_t pml4e_addr, pml4e; + + pml5e =3D env->nested_cr3; + ptep =3D PG_NX_MASK | PG_USER_MASK | PG_RW_MASK; + + pml4e_addr =3D (pml5e & PG_ADDRESS_MASK) + + (((gphys >> 39) & 0x1ff) << 3); + pml4e =3D x86_ldq_phys(cs, pml4e_addr); + if (!(pml4e & PG_PRESENT_MASK)) { + goto do_fault; + } + if (pml4e & (rsvd_mask | PG_PSE_MASK)) { + goto do_fault_rsvd; + } + if (!(pml4e & PG_ACCESSED_MASK)) { + pml4e |=3D PG_ACCESSED_MASK; + x86_stl_phys_notdirty(cs, pml4e_addr, pml4e); + } + ptep &=3D pml4e ^ PG_NX_MASK; + pdpe_addr =3D (pml4e & PG_ADDRESS_MASK) + + (((gphys >> 30) & 0x1ff) << 3); + pdpe =3D x86_ldq_phys(cs, pdpe_addr); + if (!(pdpe & PG_PRESENT_MASK)) { + goto do_fault; + } + if (pdpe & rsvd_mask) { + goto do_fault_rsvd; + } + ptep &=3D pdpe ^ PG_NX_MASK; + if (!(pdpe & PG_ACCESSED_MASK)) { + pdpe |=3D PG_ACCESSED_MASK; + x86_stl_phys_notdirty(cs, pdpe_addr, pdpe); + } + if (pdpe & PG_PSE_MASK) { + /* 1 GB page */ + page_size =3D 1024 * 1024 * 1024; + pte_addr =3D pdpe_addr; + pte =3D pdpe; + goto do_check_protect; + } + } else +#endif + { + pdpe_addr =3D (env->nested_cr3 & ~0x1f) + ((gphys >> 27) & 0x1= 8); + pdpe =3D x86_ldq_phys(cs, pdpe_addr); + if (!(pdpe & PG_PRESENT_MASK)) { + goto do_fault; + } + rsvd_mask |=3D PG_HI_USER_MASK; + if (pdpe & (rsvd_mask | PG_NX_MASK)) { + goto do_fault_rsvd; + } + ptep =3D PG_NX_MASK | PG_USER_MASK | PG_RW_MASK; + } + + pde_addr =3D (pdpe & PG_ADDRESS_MASK) + (((gphys >> 21) & 0x1ff) <= < 3); + pde =3D x86_ldq_phys(cs, pde_addr); + if (!(pde & PG_PRESENT_MASK)) { + goto do_fault; + } + if (pde & rsvd_mask) { + goto do_fault_rsvd; + } + ptep &=3D pde ^ PG_NX_MASK; + if (pde & PG_PSE_MASK) { + /* 2 MB page */ + page_size =3D 2048 * 1024; + pte_addr =3D pde_addr; + pte =3D pde; + goto do_check_protect; + } + /* 4 KB page */ + if (!(pde & PG_ACCESSED_MASK)) { + pde |=3D PG_ACCESSED_MASK; + x86_stl_phys_notdirty(cs, pde_addr, pde); + } + pte_addr =3D (pde & PG_ADDRESS_MASK) + (((gphys >> 12) & 0x1ff) <<= 3); + pte =3D x86_ldq_phys(cs, pte_addr); + if (!(pte & PG_PRESENT_MASK)) { + goto do_fault; + } + if (pte & rsvd_mask) { + goto do_fault_rsvd; + } + /* combine pde and pte nx, user and rw protections */ + ptep &=3D pte ^ PG_NX_MASK; + page_size =3D 4096; + } else { + uint32_t pde; + + /* page directory entry */ + pde_addr =3D (env->nested_cr3 & ~0xfff) + ((gphys >> 20) & 0xffc); + pde =3D x86_ldl_phys(cs, pde_addr); + if (!(pde & PG_PRESENT_MASK)) { + goto do_fault; + } + ptep =3D pde | PG_NX_MASK; + + /* if host cr4 PSE bit is set, then we use a 4MB page */ + if ((pde & PG_PSE_MASK) && (env->nested_pg_mode & SVM_NPT_PSE)) { + page_size =3D 4096 * 1024; + pte_addr =3D pde_addr; + + /* Bits 20-13 provide bits 39-32 of the address, bit 21 is res= erved. + * Leave bits 20-13 in place for setting accessed/dirty bits b= elow. + */ + pte =3D pde | ((pde & 0x1fe000LL) << (32 - 13)); + rsvd_mask =3D 0x200000; + goto do_check_protect_pse36; + } + + if (!(pde & PG_ACCESSED_MASK)) { + pde |=3D PG_ACCESSED_MASK; + x86_stl_phys_notdirty(cs, pde_addr, pde); + } + + /* page directory entry */ + pte_addr =3D (pde & ~0xfff) + ((gphys >> 10) & 0xffc); + pte =3D x86_ldl_phys(cs, pte_addr); + if (!(pte & PG_PRESENT_MASK)) { + goto do_fault; + } + /* combine pde and pte user and rw protections */ + ptep &=3D pte | PG_NX_MASK; + page_size =3D 4096; + rsvd_mask =3D 0; + } + + do_check_protect: + rsvd_mask |=3D (page_size - 1) & PG_ADDRESS_MASK & ~PG_PSE_PAT_MASK; + do_check_protect_pse36: + if (pte & rsvd_mask) { + goto do_fault_rsvd; + } + ptep ^=3D PG_NX_MASK; + + if (!(ptep & PG_USER_MASK)) { + goto do_fault_protect; + } + if (ptep & PG_NX_MASK) { + if (access_type =3D=3D MMU_INST_FETCH) { + goto do_fault_protect; + } + *prot &=3D ~PAGE_EXEC; + } + if (!(ptep & PG_RW_MASK)) { + if (access_type =3D=3D MMU_DATA_STORE) { + goto do_fault_protect; + } + *prot &=3D ~PAGE_WRITE; + } + + pte &=3D PG_ADDRESS_MASK & ~(page_size - 1); + page_offset =3D gphys & (page_size - 1); + return pte + page_offset; + + do_fault_rsvd: + exit_info_1 |=3D SVM_NPTEXIT_RSVD; + do_fault_protect: + exit_info_1 |=3D SVM_NPTEXIT_P; + do_fault: + x86_stq_phys(cs, env->vm_vmcb + offsetof(struct vmcb, control.exit_inf= o_2), + gphys); + exit_info_1 |=3D SVM_NPTEXIT_US; + if (access_type =3D=3D MMU_DATA_STORE) { + exit_info_1 |=3D SVM_NPTEXIT_RW; + } else if (access_type =3D=3D MMU_INST_FETCH) { + exit_info_1 |=3D SVM_NPTEXIT_ID; + } + if (prot) { + exit_info_1 |=3D SVM_NPTEXIT_GPA; + } else { /* page table access */ + exit_info_1 |=3D SVM_NPTEXIT_GPT; + } + cpu_vmexit(env, SVM_EXIT_NPF, exit_info_1, env->retaddr); +} + +/* return value: + * -1 =3D cannot handle fault + * 0 =3D nothing more to do + * 1 =3D generate PF fault + */ +static int handle_mmu_fault(CPUState *cs, vaddr addr, int size, + int is_write1, int mmu_idx) +{ + X86CPU *cpu =3D X86_CPU(cs); + CPUX86State *env =3D &cpu->env; + uint64_t ptep, pte; + int32_t a20_mask; + target_ulong pde_addr, pte_addr; + int error_code =3D 0; + int is_dirty, prot, page_size, is_write, is_user; + hwaddr paddr; + uint64_t rsvd_mask =3D PG_HI_RSVD_MASK; + uint32_t page_offset; + target_ulong vaddr; + uint32_t pkr; + + is_user =3D mmu_idx =3D=3D MMU_USER_IDX; +#if defined(DEBUG_MMU) + printf("MMU fault: addr=3D%" VADDR_PRIx " w=3D%d u=3D%d eip=3D" TARGET= _FMT_lx "\n", + addr, is_write1, is_user, env->eip); +#endif + is_write =3D is_write1 & 1; + + a20_mask =3D x86_get_a20_mask(env); + if (!(env->cr[0] & CR0_PG_MASK)) { + pte =3D addr; +#ifdef TARGET_X86_64 + if (!(env->hflags & HF_LMA_MASK)) { + /* Without long mode we can only address 32bits in real mode */ + pte =3D (uint32_t)pte; + } +#endif + prot =3D PAGE_READ | PAGE_WRITE | PAGE_EXEC; + page_size =3D 4096; + goto do_mapping; + } + + if (!(env->efer & MSR_EFER_NXE)) { + rsvd_mask |=3D PG_NX_MASK; + } + + if (env->cr[4] & CR4_PAE_MASK) { + uint64_t pde, pdpe; + target_ulong pdpe_addr; + +#ifdef TARGET_X86_64 + if (env->hflags & HF_LMA_MASK) { + bool la57 =3D env->cr[4] & CR4_LA57_MASK; + uint64_t pml5e_addr, pml5e; + uint64_t pml4e_addr, pml4e; + int32_t sext; + + /* test virtual address sign extension */ + sext =3D la57 ? (int64_t)addr >> 56 : (int64_t)addr >> 47; + if (sext !=3D 0 && sext !=3D -1) { + env->error_code =3D 0; + cs->exception_index =3D EXCP0D_GPF; + return 1; + } + + if (la57) { + pml5e_addr =3D ((env->cr[3] & ~0xfff) + + (((addr >> 48) & 0x1ff) << 3)) & a20_mask; + pml5e_addr =3D get_hphys(cs, pml5e_addr, MMU_DATA_STORE, N= ULL); + pml5e =3D x86_ldq_phys(cs, pml5e_addr); + if (!(pml5e & PG_PRESENT_MASK)) { + goto do_fault; + } + if (pml5e & (rsvd_mask | PG_PSE_MASK)) { + goto do_fault_rsvd; + } + if (!(pml5e & PG_ACCESSED_MASK)) { + pml5e |=3D PG_ACCESSED_MASK; + x86_stl_phys_notdirty(cs, pml5e_addr, pml5e); + } + ptep =3D pml5e ^ PG_NX_MASK; + } else { + pml5e =3D env->cr[3]; + ptep =3D PG_NX_MASK | PG_USER_MASK | PG_RW_MASK; + } + + pml4e_addr =3D ((pml5e & PG_ADDRESS_MASK) + + (((addr >> 39) & 0x1ff) << 3)) & a20_mask; + pml4e_addr =3D get_hphys(cs, pml4e_addr, MMU_DATA_STORE, false= ); + pml4e =3D x86_ldq_phys(cs, pml4e_addr); + if (!(pml4e & PG_PRESENT_MASK)) { + goto do_fault; + } + if (pml4e & (rsvd_mask | PG_PSE_MASK)) { + goto do_fault_rsvd; + } + if (!(pml4e & PG_ACCESSED_MASK)) { + pml4e |=3D PG_ACCESSED_MASK; + x86_stl_phys_notdirty(cs, pml4e_addr, pml4e); + } + ptep &=3D pml4e ^ PG_NX_MASK; + pdpe_addr =3D ((pml4e & PG_ADDRESS_MASK) + (((addr >> 30) & 0x= 1ff) << 3)) & + a20_mask; + pdpe_addr =3D get_hphys(cs, pdpe_addr, MMU_DATA_STORE, NULL); + pdpe =3D x86_ldq_phys(cs, pdpe_addr); + if (!(pdpe & PG_PRESENT_MASK)) { + goto do_fault; + } + if (pdpe & rsvd_mask) { + goto do_fault_rsvd; + } + ptep &=3D pdpe ^ PG_NX_MASK; + if (!(pdpe & PG_ACCESSED_MASK)) { + pdpe |=3D PG_ACCESSED_MASK; + x86_stl_phys_notdirty(cs, pdpe_addr, pdpe); + } + if (pdpe & PG_PSE_MASK) { + /* 1 GB page */ + page_size =3D 1024 * 1024 * 1024; + pte_addr =3D pdpe_addr; + pte =3D pdpe; + goto do_check_protect; + } + } else +#endif + { + /* XXX: load them when cr3 is loaded ? */ + pdpe_addr =3D ((env->cr[3] & ~0x1f) + ((addr >> 27) & 0x18)) & + a20_mask; + pdpe_addr =3D get_hphys(cs, pdpe_addr, MMU_DATA_STORE, false); + pdpe =3D x86_ldq_phys(cs, pdpe_addr); + if (!(pdpe & PG_PRESENT_MASK)) { + goto do_fault; + } + rsvd_mask |=3D PG_HI_USER_MASK; + if (pdpe & (rsvd_mask | PG_NX_MASK)) { + goto do_fault_rsvd; + } + ptep =3D PG_NX_MASK | PG_USER_MASK | PG_RW_MASK; + } + + pde_addr =3D ((pdpe & PG_ADDRESS_MASK) + (((addr >> 21) & 0x1ff) <= < 3)) & + a20_mask; + pde_addr =3D get_hphys(cs, pde_addr, MMU_DATA_STORE, NULL); + pde =3D x86_ldq_phys(cs, pde_addr); + if (!(pde & PG_PRESENT_MASK)) { + goto do_fault; + } + if (pde & rsvd_mask) { + goto do_fault_rsvd; + } + ptep &=3D pde ^ PG_NX_MASK; + if (pde & PG_PSE_MASK) { + /* 2 MB page */ + page_size =3D 2048 * 1024; + pte_addr =3D pde_addr; + pte =3D pde; + goto do_check_protect; + } + /* 4 KB page */ + if (!(pde & PG_ACCESSED_MASK)) { + pde |=3D PG_ACCESSED_MASK; + x86_stl_phys_notdirty(cs, pde_addr, pde); + } + pte_addr =3D ((pde & PG_ADDRESS_MASK) + (((addr >> 12) & 0x1ff) <<= 3)) & + a20_mask; + pte_addr =3D get_hphys(cs, pte_addr, MMU_DATA_STORE, NULL); + pte =3D x86_ldq_phys(cs, pte_addr); + if (!(pte & PG_PRESENT_MASK)) { + goto do_fault; + } + if (pte & rsvd_mask) { + goto do_fault_rsvd; + } + /* combine pde and pte nx, user and rw protections */ + ptep &=3D pte ^ PG_NX_MASK; + page_size =3D 4096; + } else { + uint32_t pde; + + /* page directory entry */ + pde_addr =3D ((env->cr[3] & ~0xfff) + ((addr >> 20) & 0xffc)) & + a20_mask; + pde_addr =3D get_hphys(cs, pde_addr, MMU_DATA_STORE, NULL); + pde =3D x86_ldl_phys(cs, pde_addr); + if (!(pde & PG_PRESENT_MASK)) { + goto do_fault; + } + ptep =3D pde | PG_NX_MASK; + + /* if PSE bit is set, then we use a 4MB page */ + if ((pde & PG_PSE_MASK) && (env->cr[4] & CR4_PSE_MASK)) { + page_size =3D 4096 * 1024; + pte_addr =3D pde_addr; + + /* Bits 20-13 provide bits 39-32 of the address, bit 21 is res= erved. + * Leave bits 20-13 in place for setting accessed/dirty bits b= elow. + */ + pte =3D pde | ((pde & 0x1fe000LL) << (32 - 13)); + rsvd_mask =3D 0x200000; + goto do_check_protect_pse36; + } + + if (!(pde & PG_ACCESSED_MASK)) { + pde |=3D PG_ACCESSED_MASK; + x86_stl_phys_notdirty(cs, pde_addr, pde); + } + + /* page directory entry */ + pte_addr =3D ((pde & ~0xfff) + ((addr >> 10) & 0xffc)) & + a20_mask; + pte_addr =3D get_hphys(cs, pte_addr, MMU_DATA_STORE, NULL); + pte =3D x86_ldl_phys(cs, pte_addr); + if (!(pte & PG_PRESENT_MASK)) { + goto do_fault; + } + /* combine pde and pte user and rw protections */ + ptep &=3D pte | PG_NX_MASK; + page_size =3D 4096; + rsvd_mask =3D 0; + } + +do_check_protect: + rsvd_mask |=3D (page_size - 1) & PG_ADDRESS_MASK & ~PG_PSE_PAT_MASK; +do_check_protect_pse36: + if (pte & rsvd_mask) { + goto do_fault_rsvd; + } + ptep ^=3D PG_NX_MASK; + + /* can the page can be put in the TLB? prot will tell us */ + if (is_user && !(ptep & PG_USER_MASK)) { + goto do_fault_protect; + } + + prot =3D 0; + if (mmu_idx !=3D MMU_KSMAP_IDX || !(ptep & PG_USER_MASK)) { + prot |=3D PAGE_READ; + if ((ptep & PG_RW_MASK) || (!is_user && !(env->cr[0] & CR0_WP_MASK= ))) { + prot |=3D PAGE_WRITE; + } + } + if (!(ptep & PG_NX_MASK) && + (mmu_idx =3D=3D MMU_USER_IDX || + !((env->cr[4] & CR4_SMEP_MASK) && (ptep & PG_USER_MASK)))) { + prot |=3D PAGE_EXEC; + } + + if (!(env->hflags & HF_LMA_MASK)) { + pkr =3D 0; + } else if (ptep & PG_USER_MASK) { + pkr =3D env->cr[4] & CR4_PKE_MASK ? env->pkru : 0; + } else { + pkr =3D env->cr[4] & CR4_PKS_MASK ? env->pkrs : 0; + } + if (pkr) { + uint32_t pk =3D (pte & PG_PKRU_MASK) >> PG_PKRU_BIT; + uint32_t pkr_ad =3D (pkr >> pk * 2) & 1; + uint32_t pkr_wd =3D (pkr >> pk * 2) & 2; + uint32_t pkr_prot =3D PAGE_READ | PAGE_WRITE | PAGE_EXEC; + + if (pkr_ad) { + pkr_prot &=3D ~(PAGE_READ | PAGE_WRITE); + } else if (pkr_wd && (is_user || env->cr[0] & CR0_WP_MASK)) { + pkr_prot &=3D ~PAGE_WRITE; + } + + prot &=3D pkr_prot; + if ((pkr_prot & (1 << is_write1)) =3D=3D 0) { + assert(is_write1 !=3D 2); + error_code |=3D PG_ERROR_PK_MASK; + goto do_fault_protect; + } + } + + if ((prot & (1 << is_write1)) =3D=3D 0) { + goto do_fault_protect; + } + + /* yes, it can! */ + is_dirty =3D is_write && !(pte & PG_DIRTY_MASK); + if (!(pte & PG_ACCESSED_MASK) || is_dirty) { + pte |=3D PG_ACCESSED_MASK; + if (is_dirty) { + pte |=3D PG_DIRTY_MASK; + } + x86_stl_phys_notdirty(cs, pte_addr, pte); + } + + if (!(pte & PG_DIRTY_MASK)) { + /* only set write access if already dirty... otherwise wait + for dirty access */ + assert(!is_write); + prot &=3D ~PAGE_WRITE; + } + + do_mapping: + pte =3D pte & a20_mask; + + /* align to page_size */ + pte &=3D PG_ADDRESS_MASK & ~(page_size - 1); + page_offset =3D addr & (page_size - 1); + paddr =3D get_hphys(cs, pte + page_offset, is_write1, &prot); + + /* Even if 4MB pages, we map only one 4KB page in the cache to + avoid filling it too fast */ + vaddr =3D addr & TARGET_PAGE_MASK; + paddr &=3D TARGET_PAGE_MASK; + + assert(prot & (1 << is_write1)); + tlb_set_page_with_attrs(cs, vaddr, paddr, cpu_get_mem_attrs(env), + prot, mmu_idx, page_size); + return 0; + do_fault_rsvd: + error_code |=3D PG_ERROR_RSVD_MASK; + do_fault_protect: + error_code |=3D PG_ERROR_P_MASK; + do_fault: + error_code |=3D (is_write << PG_ERROR_W_BIT); + if (is_user) + error_code |=3D PG_ERROR_U_MASK; + if (is_write1 =3D=3D 2 && + (((env->efer & MSR_EFER_NXE) && + (env->cr[4] & CR4_PAE_MASK)) || + (env->cr[4] & CR4_SMEP_MASK))) + error_code |=3D PG_ERROR_I_D_MASK; + if (env->intercept_exceptions & (1 << EXCP0E_PAGE)) { + /* cr2 is not modified in case of exceptions */ + x86_stq_phys(cs, + env->vm_vmcb + offsetof(struct vmcb, control.exit_info_2), + addr); + } else { + env->cr[2] =3D addr; + } + env->error_code =3D error_code; + cs->exception_index =3D EXCP0E_PAGE; + return 1; +} + +bool x86_cpu_tlb_fill(CPUState *cs, vaddr addr, int size, + MMUAccessType access_type, int mmu_idx, + bool probe, uintptr_t retaddr) +{ + X86CPU *cpu =3D X86_CPU(cs); + CPUX86State *env =3D &cpu->env; + + env->retaddr =3D retaddr; + if (handle_mmu_fault(cs, addr, size, access_type, mmu_idx)) { + /* FIXME: On error in get_hphys we have already jumped out. */ + g_assert(!probe); + raise_exception_err_ra(env, cs->exception_index, + env->error_code, retaddr); + } + return true; +} diff --git a/target/i386/tcg/user/excp_helper.c b/target/i386/tcg/user/excp= _helper.c new file mode 100644 index 0000000000..a89b5228fd --- /dev/null +++ b/target/i386/tcg/user/excp_helper.c @@ -0,0 +1,39 @@ +/* + * x86 exception helpers - user-mode specific code + * + * Copyright (c) 2003 Fabrice Bellard + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, see . + */ + +#include "qemu/osdep.h" +#include "cpu.h" +#include "exec/exec-all.h" +#include "tcg/helper-tcg.h" + +bool x86_cpu_tlb_fill(CPUState *cs, vaddr addr, int size, + MMUAccessType access_type, int mmu_idx, + bool probe, uintptr_t retaddr) +{ + X86CPU *cpu =3D X86_CPU(cs); + CPUX86State *env =3D &cpu->env; + + env->cr[2] =3D addr; + env->error_code =3D (access_type =3D=3D MMU_DATA_STORE) << PG_ERROR_W_= BIT; + env->error_code |=3D PG_ERROR_U_MASK; + cs->exception_index =3D EXCP0E_PAGE; + env->exception_is_int =3D 0; + env->exception_next_eip =3D -1; + cpu_loop_exit_restore(cs, retaddr); +} diff --git a/target/i386/tcg/sysemu/meson.build b/target/i386/tcg/sysemu/me= son.build index 35ba16dc3d..6d0a0a0fee 100644 --- a/target/i386/tcg/sysemu/meson.build +++ b/target/i386/tcg/sysemu/meson.build @@ -1,4 +1,5 @@ i386_softmmu_ss.add(when: ['CONFIG_TCG', 'CONFIG_SOFTMMU'], if_true: files( 'tcg-cpu.c', 'smm_helper.c', + 'excp_helper.c', )) diff --git a/target/i386/tcg/user/meson.build b/target/i386/tcg/user/meson.= build index 7aecc53155..e0ef0f02e2 100644 --- a/target/i386/tcg/user/meson.build +++ b/target/i386/tcg/user/meson.build @@ -1,2 +1,3 @@ i386_user_ss.add(when: ['CONFIG_TCG', 'CONFIG_USER_ONLY'], if_true: files( + 'excp_helper.c', )) --=20 2.26.2