From nobody Sat May 18 00:08:55 2024 Delivered-To: importer@patchew.org Received-SPF: none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=gmail.com ARC-Seal: i=1; a=rsa-sha256; t=1569585560; cv=none; d=zoho.com; s=zohoarc; b=TpqW+GUV/WpFK0rAutrXEPZl+7SQwGSnzD6CRSKkviH4E1EiUtap0i4Kn4wlKb9ipsp5Lw61x7tsfj5saVwHxlS5EZGKXEJURqIkjXBubiZpWyVxf97i2xCTloxMSMOp62WpaVf7t43zvBExVw0MT7/81zHMhpMzkRwV8dRGSoQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569585560; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:To:ARC-Authentication-Results; bh=7eDPxR/QGWldTe2JSjxkhWO7wQDL/CgzpuQJuVQZgV4=; b=WTZT0zmk4kUdumLDe9bKfErUMTZt7GtiU573Wyh0bmUC+qSr7Y5rSg1uiMHJHtr6/7dJncuZqx8H+ba5rupE8pM2yd9ffPHSS9d7nrekbKWs3kEry0BKkS8AsMTWTp3DRzYnYzJmdeX+1LGYuhxKMUXdXkdZeJECP+mQXFjHv7o= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=fail; spf=none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 156958556054366.14306266701851; Fri, 27 Sep 2019 04:59:20 -0700 (PDT) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iDot4-0002YE-ML; Fri, 27 Sep 2019 11:58:14 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iDot4-0002Y8-9o for xen-devel@lists.xenproject.org; Fri, 27 Sep 2019 11:58:14 +0000 Received: from mail-lj1-x242.google.com (unknown [2a00:1450:4864:20::242]) by localhost (Halon) with ESMTPS id 14ae082c-e11e-11e9-bf31-bc764e2007e4; Fri, 27 Sep 2019 11:58:13 +0000 (UTC) Received: by mail-lj1-x242.google.com with SMTP id m13so2188832ljj.11 for ; Fri, 27 Sep 2019 04:58:13 -0700 (PDT) Received: from otyshchenko.kyiv.epam.com (ll-22.209.223.85.sovam.net.ua. [85.223.209.22]) by smtp.gmail.com with ESMTPSA id q13sm432014lfk.51.2019.09.27.04.58.10 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 27 Sep 2019 04:58:10 -0700 (PDT) X-Inumbo-ID: 14ae082c-e11e-11e9-bf31-bc764e2007e4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=UdLcNM0m8KyMX2czbvhpEmCu6gzp8xQvc51H9sjWnXE=; b=dmDUJOzisAA4ExQnDL+McDpFklDULWeSuC7vEt4188O2BJJPzer89gGKIewtrByAJH Qkto2Fil+ajPflDIlBe7We5L2KV0r9m515mCiopPASsD/6r1s2esvsdSRXnp+AAZeIux 6khvEJoc6beObz7Gztzu+DBHAfUagpjCJLUJ1qmnNVox5ujQ3qmMpwqY2GJfIKpTDKnt SoKwKxelyvPR6L/haNweHaVBWBFsaqzHB2UwVIDXPd6/3gPt4LynHSPrIvKAaV1qzz4c edb99XHsQnEEu2K7jbgegxyq1Qkpb1uZ1I6QPf5EUwb3ajLnCNIxG4XWWn16p6WWzKtA hu0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=UdLcNM0m8KyMX2czbvhpEmCu6gzp8xQvc51H9sjWnXE=; b=FO27k476B+wpo090Uqp+3RtDUqF9ugsu22QuPGKOvgcto0gqP0kAr16Mn5Zd/d9Mj4 LKcAiwyNaEQRK4WPxmzBuCFIaIt+u7Zl01L3nOm7QVbqxLZwaneADthSqe/gmJ9ItTjs Yu7BrHe5SA+VRYBlgCocBSAaywFX0tSyoHmoUPojVrvwbNO6aZq3GZQ+nRdWKNCZVuBE syL2ew0qKq2d+BNH0k1xnhmgOKV69tQ9Dh26ti8PYNUfG6gqMG3Zzk5//sJAGwPF5I8i 39pcSh03E7o8BJ0XByBrDN+SFPpKBgcZ91eVXlLiPkaE9jJZK7Dokw1KwZkw1RC+QMS0 LA2w== X-Gm-Message-State: APjAAAX2wDWVFSrZhvfoIoqhWfeLQb3209ouJa4Gjbpp6yDVfU8RDQ/h Vpo1Tk7VTfXyUziI2XZaX9Tg60L7MAM= X-Google-Smtp-Source: APXvYqyFbP7gvsKjcx1U6caBDg7FdNEqJqdGUxN0hmsoW+fEWgOdP49piMOeIoOJpltM9QQMTuDkZQ== X-Received: by 2002:a2e:8642:: with SMTP id i2mr2632908ljj.110.1569585491560; Fri, 27 Sep 2019 04:58:11 -0700 (PDT) From: Oleksandr Tyshchenko To: xen-devel@lists.xenproject.org Date: Fri, 27 Sep 2019 14:57:54 +0300 Message-Id: <1569585474-8955-1-git-send-email-olekstysh@gmail.com> X-Mailer: git-send-email 2.7.4 Subject: [Xen-devel] [PATCH V5] xen/arm: Restrict "p2m_ipa_bits" according to the IOMMU requirements X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Oleksandr Tyshchenko , julien.grall@arm.com, sstabellini@kernel.org, volodymyr_babchuk@epam.com MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) From: Oleksandr Tyshchenko There is a strict requirement for the IOMMU which wants to share the P2M table with the CPU. The IOMMU's Stage-2 input size must be equal to the P2M IPA size. It is not a problem when the IOMMU can support all values the CPU supports. In that case, the IOMMU driver would just use any "p2m_ipa_bits" value as is. But, there are cases when not. In order to make P2M sharing possible on the platforms which IOMMUs have a limitation in maximum Stage-2 input size introduce the following logic. First initialize the IOMMU subsystem and gather requirements regarding the maximum IPA bits supported by each IOMMU device to figure out the minimum value among them. In the P2M code, take into the account the IOMMU requirements and choose suitable "pa_range" according to the restricted "p2m_ipa_bits". Signed-off-by: Oleksandr Tyshchenko CC: Julien Grall Reviewed-by: Julien Grall --- Please note, this patch wasn't checked for the SMMU. For IPMMU it works well: w/ IOMMU: (XEN) I/O virtualisation enabled (XEN) - Dom0 mode: Relaxed (XEN) Interrupt remapping enabled (XEN) P2M: 40-bit IPA with 40-bit PA and 8-bit VMID (XEN) P2M: 3 levels with order-1 root, VTCR 0x80023558 (XEN) Adding cpu 0 to runqueue 0 w/o IOMMU: (XEN) I/O virtualisation disabled (XEN) P2M: 44-bit IPA with 44-bit PA and 8-bit VMID (XEN) P2M: 4 levels with order-0 root, VTCR 0x80043594 Changes V4 [4] -> V5: - [SMMU] Use p2m_ipa_bits instead of smmu->s2_input_size for the TTBCR calculation Changes RFC V3 [3] -> V4 [4]: - Move check for p2m_ipa_bits to be at least 40 bit under #ifdef CONFIG_ARM_32 - Reword the "panic" message Changes RFC V2 [2] -> RFC V3 [3]: - Check in setup_virt_paging() that the "restricted" P2M IPA size is at least 40-bit - Modify logic in setup_virt_paging() a bit to make it "IOMMU-agnostic" - Clarify comments in code, add some explanations - Avoid using the term "IOMMU" in P2M code where possible Changes RFC V1 [1] -> RFC V2 [2]: - Don't update p2m_ipa_bits by the IOMMU drivers directly, introduce p2m_restrict_ipa_bits() - Clarify patch subject/description - Add more comments to code - Check for equivalent "pabits" in setup_virt_paging() - Remove ASSERTs from the SMMU and IPMMU drivers [1] https://lists.xenproject.org/archives/html/xen-devel/2019-08/msg02078.h= tml [2] https://lists.xenproject.org/archives/html/xen-devel/2019-08/msg02237.h= tml [3] https://lists.xenproject.org/archives/html/xen-devel/2019-09/msg00973.h= tml [4] https://lists.xenproject.org/archives/html/xen-devel/2019-09/msg02304.h= tml --- xen/arch/arm/p2m.c | 41 ++++++++++++++++++++++++++++= ---- xen/arch/arm/setup.c | 9 +++++-- xen/drivers/passthrough/arm/ipmmu-vmsa.c | 18 ++------------ xen/drivers/passthrough/arm/smmu.c | 17 +++++++------ xen/include/asm-arm/p2m.h | 9 +++++++ 5 files changed, 63 insertions(+), 31 deletions(-) diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c index e859763..5c7504e 100644 --- a/xen/arch/arm/p2m.c +++ b/xen/arch/arm/p2m.c @@ -34,7 +34,11 @@ static unsigned int __read_mostly max_vmid =3D MAX_VMID_= 8_BIT; =20 #define P2M_ROOT_PAGES (1<mm64.pa_range < pa_range ) - pa_range =3D info->mm64.pa_range; + + /* + * Restrict "p2m_ipa_bits" if needed. As P2M table is always confi= gured + * with IPA bits =3D=3D PA bits, compare against "pabits". + */ + if ( pa_range_info[info->mm64.pa_range].pabits < p2m_ipa_bits ) + p2m_ipa_bits =3D pa_range_info[info->mm64.pa_range].pabits; =20 /* Set a flag if the current cpu does not support 16 bit VMIDs. */ if ( info->mm64.vmid_bits !=3D MM64_VMID_16_BITS_SUPPORT ) @@ -2003,6 +2026,16 @@ void __init setup_virt_paging(void) if ( !vmid_8_bit ) max_vmid =3D MAX_VMID_16_BIT; =20 + /* Choose suitable "pa_range" according to the resulted "p2m_ipa_bits"= . */ + for ( i =3D 0; i < ARRAY_SIZE(pa_range_info); i++ ) + { + if ( p2m_ipa_bits =3D=3D pa_range_info[i].pabits ) + { + pa_range =3D i; + break; + } + } + /* pa_range is 4 bits, but the defined encodings are only 3 bits */ if ( pa_range >=3D ARRAY_SIZE(pa_range_info) || !pa_range_info[pa_rang= e].pabits ) panic("Unknown encoding of ID_AA64MMFR0_EL1.PARange %x\n", pa_rang= e); diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c index fca7e68..790eab9 100644 --- a/xen/arch/arm/setup.c +++ b/xen/arch/arm/setup.c @@ -928,12 +928,17 @@ void __init start_xen(unsigned long boot_phys_offset, printk("Brought up %ld CPUs\n", (long)num_online_cpus()); /* TODO: smp_cpus_done(); */ =20 - setup_virt_paging(); - + /* + * The IOMMU subsystem must be initialized before P2M as we need + * to gather requirements regarding the maximum IPA bits supported by + * each IOMMU device. + */ rc =3D iommu_setup(); if ( !iommu_enabled && rc !=3D -ENODEV ) panic("Couldn't configure correctly all the IOMMUs.\n"); =20 + setup_virt_paging(); + do_initcalls(); =20 /* diff --git a/xen/drivers/passthrough/arm/ipmmu-vmsa.c b/xen/drivers/passthr= ough/arm/ipmmu-vmsa.c index f2fb4a2..9cfae7e 100644 --- a/xen/drivers/passthrough/arm/ipmmu-vmsa.c +++ b/xen/drivers/passthrough/arm/ipmmu-vmsa.c @@ -844,22 +844,8 @@ static int ipmmu_probe(struct dt_device_node *node) goto out; } =20 - /* - * As 4-level translation table is not supported in IPMMU, we need - * to check IPA size used for P2M table beforehand to be sure it is - * 3-level and the IPMMU will be able to use it. - * - * TODO: First initialize the IOMMU and gather the requirements and - * then initialize the P2M. In the P2M code, take into the account - * the IOMMU requirements and restrict "pa_range" if necessary. - */ - if ( IPMMU_MAX_P2M_IPA_BITS < p2m_ipa_bits ) - { - printk(XENLOG_ERR "ipmmu: P2M IPA size is not supported (P2M= =3D%u IPMMU=3D%u)!\n", - p2m_ipa_bits, IPMMU_MAX_P2M_IPA_BITS); - ret =3D -ENODEV; - goto out; - } + /* Set maximum Stage-2 input size supported by the IPMMU. */ + p2m_restrict_ipa_bits(IPMMU_MAX_P2M_IPA_BITS); =20 irq =3D platform_get_irq(node, 0); if ( irq < 0 ) diff --git a/xen/drivers/passthrough/arm/smmu.c b/xen/drivers/passthrough/a= rm/smmu.c index 8ae986a..94662a8 100644 --- a/xen/drivers/passthrough/arm/smmu.c +++ b/xen/drivers/passthrough/arm/smmu.c @@ -1110,7 +1110,11 @@ static void arm_smmu_init_context_bank(struct arm_sm= mu_domain *smmu_domain) reg =3D TTBCR_TG0_64K; =20 if (!stage1) { - reg |=3D (64 - smmu->s2_input_size) << TTBCR_T0SZ_SHIFT; + /* + * Xen: The IOMMU share the page-tables with the P2M + * which may have restrict the size further. + */ + reg |=3D (64 - p2m_ipa_bits) << TTBCR_T0SZ_SHIFT; =20 switch (smmu->s2_output_size) { case 32: @@ -2198,14 +2202,9 @@ static int arm_smmu_device_cfg_probe(struct arm_smmu= _device *smmu) size =3D arm_smmu_id_size_to_bits((id >> ID2_IAS_SHIFT) & ID2_IAS_MASK); smmu->s1_output_size =3D min_t(unsigned long, PHYS_MASK_SHIFT, size); =20 - /* Xen: Stage-2 input size has to match p2m_ipa_bits. */ - if (size < p2m_ipa_bits) { - dev_err(smmu->dev, - "P2M IPA size not supported (P2M=3D%u SMMU=3D%lu)!\n", - p2m_ipa_bits, size); - return -ENODEV; - } - smmu->s2_input_size =3D p2m_ipa_bits; + /* Xen: Set maximum Stage-2 input size supported by the SMMU. */ + p2m_restrict_ipa_bits(size); + smmu->s2_input_size =3D size; #if 0 /* Stage-2 input size limited due to pgd allocation (PTRS_PER_PGD) */ #ifdef CONFIG_64BIT diff --git a/xen/include/asm-arm/p2m.h b/xen/include/asm-arm/p2m.h index 03f2ee7..89f82df 100644 --- a/xen/include/asm-arm/p2m.h +++ b/xen/include/asm-arm/p2m.h @@ -159,6 +159,15 @@ void p2m_altp2m_check(struct vcpu *v, uint16_t idx) /* Not supported on ARM. */ } =20 +/* + * Helper to restrict "p2m_ipa_bits" according the external entity + * (e.g. IOMMU) requirements. + * + * Each corresponding driver should report the maximum IPA bits + * (Stage-2 input size) it can support. + */ +void p2m_restrict_ipa_bits(unsigned int ipa_bits); + /* Second stage paging setup, to be called on all CPUs */ void setup_virt_paging(void); =20 --=20 2.7.4 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel