From nobody Tue Feb 10 02:42:49 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1571839191; cv=none; d=zoho.com; s=zohoarc; b=LO8cEeoqW6sKS8LPjVlwRYIQ7ZsN7IjscJaHnPziye2d+OV1EV5IUq0E1PtoW/EJlRrp+UIjcgPP6reDGNjwf/wD9xDHrustXjFcOIaVh4Xn+hWGzGch3GEpVHxd3uDUBU+SwBHKVXNbhmmySveiDClie2PjaudUhIeXBXIXDdc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1571839191; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=lO0CNzLyI4HwYD/QC+4iPC9LgVilOFdEWcfLQa8UoTM=; b=D/bCUAC4K0Jcx37gNvgoWxQvzoT8EJr4OeoY6djr1SxYm0xpbi4TSAdO26KE+Njix3Tx7hWuCvbkhwLeiOmccYaSmtrhULQAqcdvSQQu82ZRvkYPfVXP7DqameiGkplrF1PUptOxweSEpA26/4fCVncvz+pUO1cpSEEUh4OQ25s= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=fail; spf=none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1571839191154577.8037250462701; Wed, 23 Oct 2019 06:59:51 -0700 (PDT) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iNH9y-0004tX-5f; Wed, 23 Oct 2019 13:58:46 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iNH9w-0004ss-OD for xen-devel@lists.xenproject.org; Wed, 23 Oct 2019 13:58:44 +0000 Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 2d076042-f59d-11e9-947f-12813bfff9fa; Wed, 23 Oct 2019 13:58:22 +0000 (UTC) X-Inumbo-ID: 2d076042-f59d-11e9-947f-12813bfff9fa DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1571839102; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=yxQ5/6HF4S6wOWP4cNN9AgpK/s9kaLRvodHnGyLG7ww=; b=WG7ldNNgccKJFbQL0Y0+y1iQwGOHE/tZG3qm8lMjTS4mPvYi9PVPQFYj vMpDJH6I2kg9wcekrRMzOOo3CMmDKrWwEfi4p1IPwJz0IrDbT3JSvdG9z LgLdoWeqOUiaYgyw12OAOqXURXrSTt6zlzsWKnknkqa+L2xbiX5ETFB9I Y=; Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=andrew.cooper3@citrix.com; spf=Pass smtp.mailfrom=Andrew.Cooper3@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Received-SPF: None (esa5.hc3370-68.iphmx.com: no sender authenticity information available from domain of andrew.cooper3@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="andrew.cooper3@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa5.hc3370-68.iphmx.com: domain of Andrew.Cooper3@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="Andrew.Cooper3@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa5.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: iUc6/qyRMrU/WRyqUPfDBT/HPnw0sDcE7Gs1kFLR1r9QxEoOzCsUPqba067dIZhqhv5MmicEzw Jx2rE2uFFT/eahXhaMe9mRJuXTRzP/r3+MZLAcK8TIEGsDCQ83z73FT2WBlK//w6gbJ6AtHX4K 19KVvze8P8gN3LD9cS7emsVGY00tVma/bCzWiEXJ9x1jQPko31iM2wRG2MJTzFfHwQ4bP/7mWC D+WXFoddYeD3KI3aawlgzXYZkwHDi2N7iCCxKdDmYYJvHB4GTrLM+sie0t6n1SwaX5qwx3WQtS ng0= X-SBRS: 2.7 X-MesageID: 7627993 X-Ironport-Server: esa5.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.68,221,1569297600"; d="scan'208";a="7627993" From: Andrew Cooper To: Xen-devel Date: Wed, 23 Oct 2019 14:58:07 +0100 Message-ID: <20191023135812.21348-3-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20191023135812.21348-1-andrew.cooper3@citrix.com> References: <20191023135812.21348-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH v3 2/7] xen/nospec: Use always_inline to fix code gen for evaluate_nospec X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , Stefano Stabellini , Julien Grall , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Jan Beulich , Ian Jackson Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) evaluate_nospec() is incredibly fragile, and this is one giant bodge. To correctly protect jumps, the generated code needs to be of the form: cmp/test jcc 1f lfence ... 1: lfence ... Critically, the lfence must be at the head of both basic blocks, later in t= he instruction stream than the conditional jump in need of protection. When a static inline is involved, the optimiser decides to be clever and rearranges the code as: pred: lfence ret call pred cmp $0, %eax jcc 1f ... 1: ... which breaks the speculative safety. Any use of evaluate_nospec() needs all static inline predicates which use it to be declared always_inline to prevent the optimiser having the flexibility to generate unsafe code. Signed-off-by: Andrew Cooper Acked-by: Jan Beulich --- CC: George Dunlap CC: Ian Jackson CC: Jan Beulich CC: Konrad Rzeszutek Wilk CC: Stefano Stabellini CC: Wei Liu CC: Julien Grall CC: Juergen Gross This is the transitive set of predicates which I can spot which need protecting. There are probably ones I've missed. Personally, I'm -1 for t= his approach, but the only other option for 4.13 is to revert it all to unbreak livepatching. v3: * New --- xen/arch/x86/domain.c | 2 +- xen/arch/x86/pv/mm.h | 12 ++++++------ xen/include/asm-x86/event.h | 2 +- xen/include/asm-x86/guest_pt.h | 28 ++++++++++++++++------------ xen/include/asm-x86/hvm/nestedhvm.h | 2 +- xen/include/asm-x86/paging.h | 2 +- xen/include/xen/sched.h | 20 ++++++++++---------- 7 files changed, 36 insertions(+), 32 deletions(-) diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c index c8d7f491ea..1b88cc2d68 100644 --- a/xen/arch/x86/domain.c +++ b/xen/arch/x86/domain.c @@ -1699,7 +1699,7 @@ static void _update_runstate_area(struct vcpu *v) * regular per-CPU GDT frame to appear with selectors at the appropriate * offset. */ -static inline bool need_full_gdt(const struct domain *d) +static always_inline bool need_full_gdt(const struct domain *d) { return is_pv_domain(d) && !is_idle_domain(d); } diff --git a/xen/arch/x86/pv/mm.h b/xen/arch/x86/pv/mm.h index 2d427b418d..a1bd473b29 100644 --- a/xen/arch/x86/pv/mm.h +++ b/xen/arch/x86/pv/mm.h @@ -88,8 +88,8 @@ static inline bool update_intpte(intpte_t *p, intpte_t ol= d, intpte_t new, _t ## e_get_intpte(_o), _t ## e_get_intpte(_n), \ (_m), (_v), (_ad)) =20 -static inline l1_pgentry_t adjust_guest_l1e(l1_pgentry_t l1e, - const struct domain *d) +static always_inline l1_pgentry_t adjust_guest_l1e(l1_pgentry_t l1e, + const struct domain *d) { if ( likely(l1e_get_flags(l1e) & _PAGE_PRESENT) && likely(!is_pv_32bit_domain(d)) ) @@ -120,8 +120,8 @@ static inline l2_pgentry_t adjust_guest_l2e(l2_pgentry_= t l2e, return l2e; } =20 -static inline l3_pgentry_t adjust_guest_l3e(l3_pgentry_t l3e, - const struct domain *d) +static always_inline l3_pgentry_t adjust_guest_l3e(l3_pgentry_t l3e, + const struct domain *d) { if ( likely(l3e_get_flags(l3e) & _PAGE_PRESENT) ) l3e_add_flags(l3e, (likely(!is_pv_32bit_domain(d)) @@ -140,8 +140,8 @@ static inline l3_pgentry_t unadjust_guest_l3e(l3_pgentr= y_t l3e, return l3e; } =20 -static inline l4_pgentry_t adjust_guest_l4e(l4_pgentry_t l4e, - const struct domain *d) +static always_inline l4_pgentry_t adjust_guest_l4e(l4_pgentry_t l4e, + const struct domain *d) { /* * When shadowing an L4 behind the guests back (e.g. for per-pcpu diff --git a/xen/include/asm-x86/event.h b/xen/include/asm-x86/event.h index 2f6ea54bcb..98a85233cb 100644 --- a/xen/include/asm-x86/event.h +++ b/xen/include/asm-x86/event.h @@ -20,7 +20,7 @@ static inline int vcpu_event_delivery_is_enabled(struct v= cpu *v) } =20 int hvm_local_events_need_delivery(struct vcpu *v); -static inline int local_events_need_delivery(void) +static always_inline bool local_events_need_delivery(void) { struct vcpu *v =3D current; =20 diff --git a/xen/include/asm-x86/guest_pt.h b/xen/include/asm-x86/guest_pt.h index 8684b83fd6..6ab2041e48 100644 --- a/xen/include/asm-x86/guest_pt.h +++ b/xen/include/asm-x86/guest_pt.h @@ -202,7 +202,7 @@ static inline guest_l4e_t guest_l4e_from_gfn(gfn_t gfn,= u32 flags) =20 /* Which pagetable features are supported on this vcpu? */ =20 -static inline bool guest_can_use_l2_superpages(const struct vcpu *v) +static always_inline bool guest_can_use_l2_superpages(const struct vcpu *v) { /* * PV guests use Xen's paging settings. Being 4-level, 2M @@ -218,7 +218,7 @@ static inline bool guest_can_use_l2_superpages(const st= ruct vcpu *v) (v->arch.hvm.guest_cr[4] & X86_CR4_PSE)); } =20 -static inline bool guest_can_use_l3_superpages(const struct domain *d) +static always_inline bool guest_can_use_l3_superpages(const struct domain = *d) { /* * There are no control register settings for the hardware pagewalk on= the @@ -252,7 +252,7 @@ static inline bool guest_can_use_pse36(const struct dom= ain *d) return paging_mode_hap(d) && cpu_has_pse36; } =20 -static inline bool guest_nx_enabled(const struct vcpu *v) +static always_inline bool guest_nx_enabled(const struct vcpu *v) { if ( GUEST_PAGING_LEVELS =3D=3D 2 ) /* NX has no effect witout CR4.PAE= . */ return false; @@ -261,23 +261,23 @@ static inline bool guest_nx_enabled(const struct vcpu= *v) return is_pv_vcpu(v) ? cpu_has_nx : hvm_nx_enabled(v); } =20 -static inline bool guest_wp_enabled(const struct vcpu *v) +static always_inline bool guest_wp_enabled(const struct vcpu *v) { /* PV guests can't control CR0.WP, and it is unconditionally set by Xe= n. */ return is_pv_vcpu(v) || hvm_wp_enabled(v); } =20 -static inline bool guest_smep_enabled(const struct vcpu *v) +static always_inline bool guest_smep_enabled(const struct vcpu *v) { return !is_pv_vcpu(v) && hvm_smep_enabled(v); } =20 -static inline bool guest_smap_enabled(const struct vcpu *v) +static always_inline bool guest_smap_enabled(const struct vcpu *v) { return !is_pv_vcpu(v) && hvm_smap_enabled(v); } =20 -static inline bool guest_pku_enabled(const struct vcpu *v) +static always_inline bool guest_pku_enabled(const struct vcpu *v) { return !is_pv_vcpu(v) && hvm_pku_enabled(v); } @@ -285,19 +285,21 @@ static inline bool guest_pku_enabled(const struct vcp= u *v) /* Helpers for identifying whether guest entries have reserved bits set. */ =20 /* Bits reserved because of maxphysaddr, and (lack of) EFER.NX */ -static inline uint64_t guest_rsvd_bits(const struct vcpu *v) +static always_inline uint64_t guest_rsvd_bits(const struct vcpu *v) { return ((PADDR_MASK & ~((1ul << v->domain->arch.cpuid->extd.maxphysaddr) - 1)) | (guest_nx_enabled(v) ? 0 : put_pte_flags(_PAGE_NX_BIT))); } =20 -static inline bool guest_l1e_rsvd_bits(const struct vcpu *v, guest_l1e_t l= 1e) +static always_inline bool guest_l1e_rsvd_bits(const struct vcpu *v, + guest_l1e_t l1e) { return l1e.l1 & (guest_rsvd_bits(v) | GUEST_L1_PAGETABLE_RSVD); } =20 -static inline bool guest_l2e_rsvd_bits(const struct vcpu *v, guest_l2e_t l= 2e) +static always_inline bool guest_l2e_rsvd_bits(const struct vcpu *v, + guest_l2e_t l2e) { uint64_t rsvd_bits =3D guest_rsvd_bits(v); =20 @@ -311,7 +313,8 @@ static inline bool guest_l2e_rsvd_bits(const struct vcp= u *v, guest_l2e_t l2e) } =20 #if GUEST_PAGING_LEVELS >=3D 3 -static inline bool guest_l3e_rsvd_bits(const struct vcpu *v, guest_l3e_t l= 3e) +static always_inline bool guest_l3e_rsvd_bits(const struct vcpu *v, + guest_l3e_t l3e) { return ((l3e.l3 & (guest_rsvd_bits(v) | GUEST_L3_PAGETABLE_RSVD | (guest_can_use_l3_superpages(v->domain) ? 0 : _PAGE= _PSE))) || @@ -320,7 +323,8 @@ static inline bool guest_l3e_rsvd_bits(const struct vcp= u *v, guest_l3e_t l3e) } =20 #if GUEST_PAGING_LEVELS >=3D 4 -static inline bool guest_l4e_rsvd_bits(const struct vcpu *v, guest_l4e_t l= 4e) +static always_inline bool guest_l4e_rsvd_bits(const struct vcpu *v, + guest_l4e_t l4e) { return l4e.l4 & (guest_rsvd_bits(v) | GUEST_L4_PAGETABLE_RSVD | ((v->domain->arch.cpuid->x86_vendor =3D=3D X86_VENDOR= _AMD) diff --git a/xen/include/asm-x86/hvm/nestedhvm.h b/xen/include/asm-x86/hvm/= nestedhvm.h index e09fa9d47d..256fed733a 100644 --- a/xen/include/asm-x86/hvm/nestedhvm.h +++ b/xen/include/asm-x86/hvm/nestedhvm.h @@ -33,7 +33,7 @@ enum nestedhvm_vmexits { }; =20 /* Nested HVM on/off per domain */ -static inline bool nestedhvm_enabled(const struct domain *d) +static always_inline bool nestedhvm_enabled(const struct domain *d) { return is_hvm_domain(d) && d->arch.hvm.params && d->arch.hvm.params[HVM_PARAM_NESTEDHVM]; diff --git a/xen/include/asm-x86/paging.h b/xen/include/asm-x86/paging.h index 8c2027c791..7544f73121 100644 --- a/xen/include/asm-x86/paging.h +++ b/xen/include/asm-x86/paging.h @@ -383,7 +383,7 @@ static inline bool gfn_valid(const struct domain *d, gf= n_t gfn) } =20 /* Maxphysaddr supportable by the paging infrastructure. */ -static inline unsigned int paging_max_paddr_bits(const struct domain *d) +static always_inline unsigned int paging_max_paddr_bits(const struct domai= n *d) { unsigned int bits =3D paging_mode_hap(d) ? hap_paddr_bits : paddr_bits; =20 diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h index 629a4c52e0..9f7bc69293 100644 --- a/xen/include/xen/sched.h +++ b/xen/include/xen/sched.h @@ -968,50 +968,50 @@ void watchdog_domain_destroy(struct domain *d); =20 #define VM_ASSIST(d, t) (test_bit(VMASST_TYPE_ ## t, &(d)->vm_assist)) =20 -static inline bool is_pv_domain(const struct domain *d) +static always_inline bool is_pv_domain(const struct domain *d) { return IS_ENABLED(CONFIG_PV) && evaluate_nospec(!(d->options & XEN_DOMCTL_CDF_hvm)); } =20 -static inline bool is_pv_vcpu(const struct vcpu *v) +static always_inline bool is_pv_vcpu(const struct vcpu *v) { return is_pv_domain(v->domain); } =20 #ifdef CONFIG_COMPAT -static inline bool is_pv_32bit_domain(const struct domain *d) +static always_inline bool is_pv_32bit_domain(const struct domain *d) { return is_pv_domain(d) && d->arch.is_32bit_pv; } =20 -static inline bool is_pv_32bit_vcpu(const struct vcpu *v) +static always_inline bool is_pv_32bit_vcpu(const struct vcpu *v) { return is_pv_32bit_domain(v->domain); } =20 -static inline bool is_pv_64bit_domain(const struct domain *d) +static always_inline bool is_pv_64bit_domain(const struct domain *d) { return is_pv_domain(d) && !d->arch.is_32bit_pv; } =20 -static inline bool is_pv_64bit_vcpu(const struct vcpu *v) +static always_inline bool is_pv_64bit_vcpu(const struct vcpu *v) { return is_pv_64bit_domain(v->domain); } #endif -static inline bool is_hvm_domain(const struct domain *d) +static always_inline bool is_hvm_domain(const struct domain *d) { return IS_ENABLED(CONFIG_HVM) && evaluate_nospec(d->options & XEN_DOMCTL_CDF_hvm); } =20 -static inline bool is_hvm_vcpu(const struct vcpu *v) +static always_inline bool is_hvm_vcpu(const struct vcpu *v) { return is_hvm_domain(v->domain); } =20 -static inline bool hap_enabled(const struct domain *d) +static always_inline bool hap_enabled(const struct domain *d) { /* sanitise_domain_config() rejects HAP && !HVM */ return IS_ENABLED(CONFIG_HVM) && @@ -1034,7 +1034,7 @@ static inline bool is_xenstore_domain(const struct do= main *d) return d->options & XEN_DOMCTL_CDF_xs_domain; } =20 -static inline bool is_iommu_enabled(const struct domain *d) +static always_inline bool is_iommu_enabled(const struct domain *d) { return evaluate_nospec(d->options & XEN_DOMCTL_CDF_iommu); } --=20 2.11.0 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel