From nobody Mon Feb 9 08:12:09 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass header.i=teddy.astie@vates.tech; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=vates.tech ARC-Seal: i=1; a=rsa-sha256; t=1713441471; cv=none; d=zohomail.com; s=zohoarc; b=DLa2s3uChlVtCN6ypLAa1eLrS8rbuJlkSbBBs7UiJqUDMhfoQAUiAjcLu4knukxIFmTOX3+GjlSYL69cBH3hOi5JbyK5LRMZh/Xcjk3XmQf5QO+16zDDIdxWoo8SnrfObqoL+VR3DxmmajvnRbuXAfMCA6A8m4ymcVJv0EZDHAI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1713441471; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=IMCKkvZ/LwChBkz9aTUisa2lVAiO0Qw2N3JNoUZBFGA=; b=dG3DfxjvRMHYvNl4c+Q4dc2RWrNf+C1ZY0urU+aTR+Ufebksmr8NecASPCvEUiKrzp3o6RqHM2Rf6xKR86lJPTehzMBvsiD1jTMyUMsS0eBC04OtbPO437fbx9mOvA38acM4Rp66F+y1oeyT4Y2eyXNSrFFhADqK89BFr3F8wlI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=teddy.astie@vates.tech; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 171344147159121.916093940642327; Thu, 18 Apr 2024 04:57:51 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.708258.1106974 (Exim 4.92) (envelope-from ) id 1rxQON-0003jO-9g; Thu, 18 Apr 2024 11:57:27 +0000 Received: by outflank-mailman (output) from mailman id 708258.1106974; Thu, 18 Apr 2024 11:57:27 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rxQON-0003iJ-2l; Thu, 18 Apr 2024 11:57:27 +0000 Received: by outflank-mailman (input) for mailman id 708258; Thu, 18 Apr 2024 11:57:26 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rxQOM-0003Zj-5T for xen-devel@lists.xenproject.org; Thu, 18 Apr 2024 11:57:26 +0000 Received: from mail145-32.atl61.mandrillapp.com (mail145-32.atl61.mandrillapp.com [198.2.145.32]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id d05c4e8a-fd7a-11ee-b909-491648fe20b8; Thu, 18 Apr 2024 13:57:22 +0200 (CEST) Received: from pmta06.mandrill.prod.atl01.rsglab.com (localhost [127.0.0.1]) by mail145-32.atl61.mandrillapp.com (Mailchimp) with ESMTP id 4VKx8x049VzP0JyST for ; Thu, 18 Apr 2024 11:57:21 +0000 (GMT) Received: from [37.26.189.201] by mandrillapp.com id 849d46ce67cc49bda7395dc97d134692; Thu, 18 Apr 2024 11:57:20 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: d05c4e8a-fd7a-11ee-b909-491648fe20b8 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandrillapp.com; s=mte1; t=1713441441; x=1713701941; bh=IMCKkvZ/LwChBkz9aTUisa2lVAiO0Qw2N3JNoUZBFGA=; h=From:Subject:To:Cc:Message-Id:In-Reply-To:References:Feedback-ID: Date:MIME-Version:Content-Type:Content-Transfer-Encoding:CC:Date: Subject:From; b=reaQQn1zhnYEB4jFabYbItV3ovKhWSkulYfcbkBtin29ecuz8KK5AGquIC79vyZCw Dc18QYk+Y9bOCpK8PrrgGwW/0zeHylkwrnZCGvsy16cn5dfEMPb8/qtAecnh0pUTwD EgAkbE+Kjbv0RF3sUzInAzGokxL7Log3o1G1m2fA2UAXqtWhEOAcibujxLFmquzemu FTGraF5lmNefTkAj4B0JyuxDrQhC6Bh/aPZ41oNkA6av9zVv3vLsxX7cbN3yxz1x3w gAGzkp3GLc9+Aa09MFGb6vTSltTqrcfQ04dgZiTMII/x0WKRZWCe1sCigdLhAgMDd4 Z/znqeLY79SEw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vates.tech; s=mte1; t=1713441441; x=1713701941; i=teddy.astie@vates.tech; bh=IMCKkvZ/LwChBkz9aTUisa2lVAiO0Qw2N3JNoUZBFGA=; h=From:Subject:To:Cc:Message-Id:In-Reply-To:References:Feedback-ID: Date:MIME-Version:Content-Type:Content-Transfer-Encoding:CC:Date: Subject:From; b=y+BPT+HC/AqS2fxMuvgmeNsc+vxoJXbly5EOEgef52KPyUwvZ9RKZghxAdTEylt9l vqZiKqK4ex4kuVMfuzdXJT/Tz1IDuI7aVSuFBKlR9QRWBaJV7gucUPENlknwAxHl5M irMelW9CBU8x8c3FW/mT84y/AK1L9zp9mt134g6dAmp9ajCtaMkdY1FiUEgtFT84k2 aV7k9V7dxQZVrKuBEormL6Gn/iHnZzaV/siwyQNAgVz9DxVCyYMcIQpHhWAkk2F+RJ bPg+6VNHvb1qQ3fUXiwLYcuDhz6Sbwwlb29oZOF4WEQw4CS4yhXUIvNvoj9n8V1M9x jEIhFvUCZFk5w== From: Teddy Astie Subject: =?utf-8?Q?[XEN=20PATCH=20v5=201/3]=20x86/iommu:=20Disable=20IOMMU=20if=20cx16=20isn't=20supported?= X-Mailer: git-send-email 2.44.0 X-Bm-Disclaimer: Yes X-Bm-Milter-Handled: 4ffbd6c1-ee69-4e1b-aabd-f977039bd3e2 X-Bm-Transport-Timestamp: 1713441439912 To: xen-devel@lists.xenproject.org Cc: Teddy Astie , Jan Beulich , Andrew Cooper , =?utf-8?Q?Roger=20Pau=20Monn=C3=A9?= Message-Id: <4656eab84f7b4b807fc61f54b9ba5c0fc4fae64d.1713433029.git.teddy.astie@vates.tech> In-Reply-To: References: X-Native-Encoded: 1 X-Report-Abuse: =?UTF-8?Q?Please=20forward=20a=20copy=20of=20this=20message,=20including=20all=20headers,=20to=20abuse@mandrill.com.=20You=20can=20also=20report=20abuse=20here:=20https://mandrillapp.com/contact/abuse=3Fid=3D30504962.849d46ce67cc49bda7395dc97d134692?= X-Mandrill-User: md_30504962 Feedback-ID: 30504962:30504962.20240418:md Date: Thu, 18 Apr 2024 11:57:20 +0000 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @mandrillapp.com) (identity teddy.astie@vates.tech) X-ZM-MESSAGEID: 1713441472651100005 Content-Type: text/plain; charset="utf-8" All hardware with VT-d/AMD-Vi has CMPXCHG16B support. Check this at initialisation time, and remove the effectively-dead logic for the non-cx16 case. Suggested-by: Andrew Cooper Signed-off-by: Teddy Astie --- xen/drivers/passthrough/amd/iommu_map.c | 42 ++++-------- xen/drivers/passthrough/amd/pci_amd_iommu.c | 6 ++ xen/drivers/passthrough/vtd/iommu.c | 73 +++++++-------------- 3 files changed, 45 insertions(+), 76 deletions(-) diff --git a/xen/drivers/passthrough/amd/iommu_map.c b/xen/drivers/passthro= ugh/amd/iommu_map.c index e0f4fe736a..f67975e700 100644 --- a/xen/drivers/passthrough/amd/iommu_map.c +++ b/xen/drivers/passthrough/amd/iommu_map.c @@ -167,15 +167,14 @@ int amd_iommu_set_root_page_table(struct amd_iommu_dt= e *dte, { bool valid =3D flags & SET_ROOT_VALID; =20 - if ( dte->v && dte->tv && - (cpu_has_cx16 || (flags & SET_ROOT_WITH_UNITY_MAP)) ) + if ( dte->v && dte->tv ) { union { struct amd_iommu_dte dte; uint64_t raw64[4]; __uint128_t raw128[2]; } ldte =3D { .dte =3D *dte }; - __uint128_t old =3D ldte.raw128[0]; + __uint128_t res, old =3D ldte.raw128[0]; int ret =3D 0; =20 ldte.dte.domain_id =3D domain_id; @@ -185,33 +184,20 @@ int amd_iommu_set_root_page_table(struct amd_iommu_dt= e *dte, ldte.dte.paging_mode =3D paging_mode; ldte.dte.v =3D valid; =20 - if ( cpu_has_cx16 ) - { - __uint128_t res =3D cmpxchg16b(dte, &old, &ldte.raw128[0]); + res =3D cmpxchg16b(dte, &old, &ldte.raw128[0]); =20 - /* - * Hardware does not update the DTE behind our backs, so the - * return value should match "old". - */ - if ( res !=3D old ) - { - printk(XENLOG_ERR - "Dom%d: unexpected DTE %016lx_%016lx (expected %016= lx_%016lx)\n", - domain_id, - (uint64_t)(res >> 64), (uint64_t)res, - (uint64_t)(old >> 64), (uint64_t)old); - ret =3D -EILSEQ; - } - } - else /* Best effort, updating domain_id last. */ + /* + * Hardware does not update the DTE behind our backs, so the + * return value should match "old". + */ + if ( res !=3D old ) { - uint64_t *ptr =3D (void *)dte; - - write_atomic(ptr + 0, ldte.raw64[0]); - /* No barrier should be needed between these two. */ - write_atomic(ptr + 1, ldte.raw64[1]); - - ret =3D 1; + printk(XENLOG_ERR + "Dom%d: unexpected DTE %016lx_%016lx (expected %016lx_%= 016lx)\n", + domain_id, + (uint64_t)(res >> 64), (uint64_t)res, + (uint64_t)(old >> 64), (uint64_t)old); + ret =3D -EILSEQ; } =20 return ret; diff --git a/xen/drivers/passthrough/amd/pci_amd_iommu.c b/xen/drivers/pass= through/amd/pci_amd_iommu.c index f6efd88e36..3a6a23f706 100644 --- a/xen/drivers/passthrough/amd/pci_amd_iommu.c +++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c @@ -305,6 +305,12 @@ static int __init cf_check iov_detect(void) if ( !iommu_enable && !iommu_intremap ) return 0; =20 + if ( unlikely(!cpu_has_cx16) ) + { + printk("AMD-Vi: CPU doesn't support CMPXCHG16B, disabling\n"); + return -ENODEV; + } + if ( (init_done ? amd_iommu_init_late() : amd_iommu_init(false)) !=3D 0 ) { diff --git a/xen/drivers/passthrough/vtd/iommu.c b/xen/drivers/passthrough/= vtd/iommu.c index c7110af7c9..9c787ba9eb 100644 --- a/xen/drivers/passthrough/vtd/iommu.c +++ b/xen/drivers/passthrough/vtd/iommu.c @@ -1482,7 +1482,7 @@ int domain_context_mapping_one( { struct domain_iommu *hd =3D dom_iommu(domain); struct context_entry *context, *context_entries, lctxt; - __uint128_t old; + __uint128_t res, old; uint64_t maddr; uint16_t seg =3D iommu->drhd->segment, prev_did =3D 0; struct domain *prev_dom =3D NULL; @@ -1580,55 +1580,23 @@ int domain_context_mapping_one( ASSERT(!context_fault_disable(lctxt)); } =20 - if ( cpu_has_cx16 ) - { - __uint128_t res =3D cmpxchg16b(context, &old, &lctxt.full); - - /* - * Hardware does not update the context entry behind our backs, - * so the return value should match "old". - */ - if ( res !=3D old ) - { - if ( pdev ) - check_cleanup_domid_map(domain, pdev, iommu); - printk(XENLOG_ERR - "%pp: unexpected context entry %016lx_%016lx (expected = %016lx_%016lx)\n", - &PCI_SBDF(seg, bus, devfn), - (uint64_t)(res >> 64), (uint64_t)res, - (uint64_t)(old >> 64), (uint64_t)old); - rc =3D -EILSEQ; - goto unlock; - } - } - else if ( !prev_dom || !(mode & MAP_WITH_RMRR) ) - { - context_clear_present(*context); - iommu_sync_cache(context, sizeof(*context)); + res =3D cmpxchg16b(context, &old, &lctxt.full); =20 - write_atomic(&context->hi, lctxt.hi); - /* No barrier should be needed between these two. */ - write_atomic(&context->lo, lctxt.lo); - } - else /* Best effort, updating DID last. */ + /* + * Hardware does not update the context entry behind our backs, + * so the return value should match "old". + */ + if ( res !=3D old ) { - /* - * By non-atomically updating the context entry's DID field last, - * during a short window in time TLB entries with the old domain = ID - * but the new page tables may be inserted. This could affect I/O - * of other devices using this same (old) domain ID. Such updati= ng - * therefore is not a problem if this was the only device associa= ted - * with the old domain ID. Diverting I/O of any of a dying domai= n's - * devices to the quarantine page tables is intended anyway. - */ - if ( !(mode & (MAP_OWNER_DYING | MAP_SINGLE_DEVICE)) ) - printk(XENLOG_WARNING VTDPREFIX - " %pp: reassignment may cause %pd data corruption\n", - &PCI_SBDF(seg, bus, devfn), prev_dom); - - write_atomic(&context->lo, lctxt.lo); - /* No barrier should be needed between these two. */ - write_atomic(&context->hi, lctxt.hi); + if ( pdev ) + check_cleanup_domid_map(domain, pdev, iommu); + printk(XENLOG_ERR + "%pp: unexpected context entry %016lx_%016lx (expected %01= 6lx_%016lx)\n", + &PCI_SBDF(seg, bus, devfn), + (uint64_t)(res >> 64), (uint64_t)res, + (uint64_t)(old >> 64), (uint64_t)old); + rc =3D -EILSEQ; + goto unlock; } =20 iommu_sync_cache(context, sizeof(struct context_entry)); @@ -2630,6 +2598,15 @@ static int __init cf_check vtd_setup(void) int ret; bool reg_inval_supported =3D true; =20 + if ( unlikely(!cpu_has_cx16) ) + { + printk(XENLOG_ERR VTDPREFIX + "IOMMU: CPU doesn't support CMPXCHG16B, disabling\n"); + + ret =3D -ENODEV; + goto error; + } + if ( list_empty(&acpi_drhd_units) ) { ret =3D -ENODEV; --=20 2.44.0 Teddy Astie | Vates XCP-ng Intern XCP-ng & Xen Orchestra - Vates solutions web: https://vates.tech