From nobody Sun Feb 8 07:15:01 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 48E97EB64D8 for ; Wed, 21 Jun 2023 06:43:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230506AbjFUGno (ORCPT ); Wed, 21 Jun 2023 02:43:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39670 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230496AbjFUGnd (ORCPT ); Wed, 21 Jun 2023 02:43:33 -0400 Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com [IPv6:2607:f8b0:4864:20::1149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 691A019AD for ; Tue, 20 Jun 2023 23:43:24 -0700 (PDT) Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-561eb6c66f6so63732567b3.0 for ; Tue, 20 Jun 2023 23:43:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1687329803; x=1689921803; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=xwDD9pVjLa+TJXAJEbbYyI+224KEPbc+a+2w73kHUiI=; b=GklP6KDiFXf3o6gA21y5VFJbUznyTc1R75l4mBRj5nFR4hoMeSZ10ocJBZmnS2WrDG y1e2Cn0bqo2ZayzQKL1V6VETTYM7805/cLncPYBXTbFCKA7UmtknIdrgLlnp/Lh7+41l ODS46uvmds+v5mJnejS+409PsLB0XhiizFZfkGk4lnkMeYF/ExizizPw0JPNEUIXu9de Y8fBSonTcoEaIv/WAbBuvtRvAr/ZPBwHCDKc5Gk6b9iQdwlvfCMDP/SgbMP9mJDNNXlW QRd5ZR/ihnnjNFE1GKLMAXC1B8cUlpYrZ3N7lw9Ph/+gpwyT7oMbiGOM/J7XzkNTlTEe g4jA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687329803; x=1689921803; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=xwDD9pVjLa+TJXAJEbbYyI+224KEPbc+a+2w73kHUiI=; b=MV82tB4vDR2UpLXXqRyBLe8+DOKahT5jnuebf8denDXMki39eYMLbVxdgYWv69J+2K swOvluWdKDi5ZuFBVe2xVG0Lejlu3en89AsIizGfE5YtSptHxn/ejMX7wyf0XtcC0R11 bm9DKyrSb75W4Wkd1o+lWAs5/9Z+/3DvKGhqirZVFuzo6mNeJtHhEnW3D2Aq99RubNl2 54biCDWO8lBMdofrosuZr6mi8EEJLX26IALON0NIPc+znELhRbDv014chvGgdaVgjk3x 6TD7zYBJWn/IaCjSAP3yr1mNxlHwNtX+MLlCMXZAMQLsJg61Q+LINNIzVQzjOgm5ZIef IWow== X-Gm-Message-State: AC+VfDy28bGI9B+Asx7CbOuSp225iLKXHERLYU5t6glJT4CgC93rF4Vp npf6U7YjjjS16Zbg+jw0XZ4YU/36n1I7 X-Google-Smtp-Source: ACHHUZ5g2coWCGQBizn0+LYQ+QrLlUGiOhh/nAE0JfGA+7s++6ityIYP1l92/w+YTxSLHy+n0BFCilyqaUFu X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:384f:f7da:c61d:5a3e]) (user=mshavit job=sendgmr) by 2002:a81:b647:0:b0:56d:3c2b:2471 with SMTP id h7-20020a81b647000000b0056d3c2b2471mr6326664ywk.3.1687329803724; Tue, 20 Jun 2023 23:43:23 -0700 (PDT) Date: Wed, 21 Jun 2023 14:37:13 +0800 In-Reply-To: <20230621063825.268890-1-mshavit@google.com> Mime-Version: 1.0 References: <20230621063825.268890-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230621063825.268890-2-mshavit@google.com> Subject: [PATCH v4 01/13] iommu/arm-smmu-v3: Move ctx_desc out of s1_cfg From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" s1_cfg describes the CD table that is inserted into an SMMU's STEs. It's weird for s1_cfg to also own ctx_desc which describes a CD that is inserted into that table. It is more appropriate for arm_smmu_domain to own ctx_desc. Signed-off-by: Michael Shavit --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 2 +- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 23 +++++++-------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 28 ++++++++++--------- 3 files changed, 28 insertions(+), 25 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iomm= u/arm/arm-smmu-v3/arm-smmu-v3-sva.c index a5a63b1c947eb..968559d625c40 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -62,7 +62,7 @@ arm_smmu_share_asid(struct mm_struct *mm, u16 asid) return cd; } =20 - smmu_domain =3D container_of(cd, struct arm_smmu_domain, s1_cfg.cd); + smmu_domain =3D container_of(cd, struct arm_smmu_domain, cd); smmu =3D smmu_domain->smmu; =20 ret =3D xa_alloc(&arm_smmu_asid_xa, &new_asid, cd, diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index 3fd83fb757227..beff04b897718 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1863,7 +1863,7 @@ static void arm_smmu_tlb_inv_context(void *cookie) * careful, 007. */ if (smmu_domain->stage =3D=3D ARM_SMMU_DOMAIN_S1) { - arm_smmu_tlb_inv_asid(smmu, smmu_domain->s1_cfg.cd.asid); + arm_smmu_tlb_inv_asid(smmu, smmu_domain->cd.asid); } else { cmd.opcode =3D CMDQ_OP_TLBI_S12_VMALL; cmd.tlbi.vmid =3D smmu_domain->s2_cfg.vmid; @@ -1946,7 +1946,7 @@ static void arm_smmu_tlb_inv_range_domain(unsigned lo= ng iova, size_t size, if (smmu_domain->stage =3D=3D ARM_SMMU_DOMAIN_S1) { cmd.opcode =3D smmu_domain->smmu->features & ARM_SMMU_FEAT_E2H ? CMDQ_OP_TLBI_EL2_VA : CMDQ_OP_TLBI_NH_VA; - cmd.tlbi.asid =3D smmu_domain->s1_cfg.cd.asid; + cmd.tlbi.asid =3D smmu_domain->cd.asid; } else { cmd.opcode =3D CMDQ_OP_TLBI_S2_IPA; cmd.tlbi.vmid =3D smmu_domain->s2_cfg.vmid; @@ -2077,7 +2077,7 @@ static void arm_smmu_domain_free(struct iommu_domain = *domain) mutex_lock(&arm_smmu_asid_lock); if (cfg->cdcfg.cdtab) arm_smmu_free_cd_tables(smmu_domain); - arm_smmu_free_asid(&cfg->cd); + arm_smmu_free_asid(&smmu_domain->cd); mutex_unlock(&arm_smmu_asid_lock); } else { struct arm_smmu_s2_cfg *cfg =3D &smmu_domain->s2_cfg; @@ -2096,13 +2096,14 @@ static int arm_smmu_domain_finalise_s1(struct arm_s= mmu_domain *smmu_domain, u32 asid; struct arm_smmu_device *smmu =3D smmu_domain->smmu; struct arm_smmu_s1_cfg *cfg =3D &smmu_domain->s1_cfg; + struct arm_smmu_ctx_desc *cd =3D &smmu_domain->cd; typeof(&pgtbl_cfg->arm_lpae_s1_cfg.tcr) tcr =3D &pgtbl_cfg->arm_lpae_s1_c= fg.tcr; =20 - refcount_set(&cfg->cd.refs, 1); + refcount_set(&cd->refs, 1); =20 /* Prevent SVA from modifying the ASID until it is written to the CD */ mutex_lock(&arm_smmu_asid_lock); - ret =3D xa_alloc(&arm_smmu_asid_xa, &asid, &cfg->cd, + ret =3D xa_alloc(&arm_smmu_asid_xa, &asid, cd, XA_LIMIT(1, (1 << smmu->asid_bits) - 1), GFP_KERNEL); if (ret) goto out_unlock; @@ -2115,23 +2116,23 @@ static int arm_smmu_domain_finalise_s1(struct arm_s= mmu_domain *smmu_domain, if (ret) goto out_free_asid; =20 - cfg->cd.asid =3D (u16)asid; - cfg->cd.ttbr =3D pgtbl_cfg->arm_lpae_s1_cfg.ttbr; - cfg->cd.tcr =3D FIELD_PREP(CTXDESC_CD_0_TCR_T0SZ, tcr->tsz) | + cd->asid =3D (u16)asid; + cd->ttbr =3D pgtbl_cfg->arm_lpae_s1_cfg.ttbr; + cd->tcr =3D FIELD_PREP(CTXDESC_CD_0_TCR_T0SZ, tcr->tsz) | FIELD_PREP(CTXDESC_CD_0_TCR_TG0, tcr->tg) | FIELD_PREP(CTXDESC_CD_0_TCR_IRGN0, tcr->irgn) | FIELD_PREP(CTXDESC_CD_0_TCR_ORGN0, tcr->orgn) | FIELD_PREP(CTXDESC_CD_0_TCR_SH0, tcr->sh) | FIELD_PREP(CTXDESC_CD_0_TCR_IPS, tcr->ips) | CTXDESC_CD_0_TCR_EPD1 | CTXDESC_CD_0_AA64; - cfg->cd.mair =3D pgtbl_cfg->arm_lpae_s1_cfg.mair; + cd->mair =3D pgtbl_cfg->arm_lpae_s1_cfg.mair; =20 /* * Note that this will end up calling arm_smmu_sync_cd() before * the master has been added to the devices list for this domain. * This isn't an issue because the STE hasn't been installed yet. */ - ret =3D arm_smmu_write_ctx_desc(smmu_domain, 0, &cfg->cd); + ret =3D arm_smmu_write_ctx_desc(smmu_domain, 0, cd); if (ret) goto out_free_cd_tables; =20 @@ -2141,7 +2142,7 @@ static int arm_smmu_domain_finalise_s1(struct arm_smm= u_domain *smmu_domain, out_free_cd_tables: arm_smmu_free_cd_tables(smmu_domain); out_free_asid: - arm_smmu_free_asid(&cfg->cd); + arm_smmu_free_asid(cd); out_unlock: mutex_unlock(&arm_smmu_asid_lock); return ret; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.h index b574c58a34876..68d519f21dbd8 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -593,7 +593,6 @@ struct arm_smmu_ctx_desc_cfg { =20 struct arm_smmu_s1_cfg { struct arm_smmu_ctx_desc_cfg cdcfg; - struct arm_smmu_ctx_desc cd; u8 s1fmt; u8 s1cdmax; }; @@ -707,25 +706,28 @@ enum arm_smmu_domain_stage { }; =20 struct arm_smmu_domain { - struct arm_smmu_device *smmu; - struct mutex init_mutex; /* Protects smmu pointer */ + struct arm_smmu_device *smmu; + struct mutex init_mutex; /* Protects smmu pointer */ =20 - struct io_pgtable_ops *pgtbl_ops; - bool stall_enabled; - atomic_t nr_ats_masters; + struct io_pgtable_ops *pgtbl_ops; + bool stall_enabled; + atomic_t nr_ats_masters; =20 - enum arm_smmu_domain_stage stage; + enum arm_smmu_domain_stage stage; union { - struct arm_smmu_s1_cfg s1_cfg; - struct arm_smmu_s2_cfg s2_cfg; + struct { + struct arm_smmu_ctx_desc cd; + struct arm_smmu_s1_cfg s1_cfg; + }; + struct arm_smmu_s2_cfg s2_cfg; }; =20 - struct iommu_domain domain; + struct iommu_domain domain; =20 - struct list_head devices; - spinlock_t devices_lock; + struct list_head devices; + spinlock_t devices_lock; =20 - struct list_head mmu_notifiers; + struct list_head mmu_notifiers; }; =20 static inline struct arm_smmu_domain *to_smmu_domain(struct iommu_domain *= dom) --=20 2.41.0.162.gfafddb0af9-goog From nobody Sun Feb 8 07:15:01 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6AC93EB64D8 for ; Wed, 21 Jun 2023 06:44:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231137AbjFUGoG (ORCPT ); Wed, 21 Jun 2023 02:44:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39912 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230013AbjFUGni (ORCPT ); Wed, 21 Jun 2023 02:43:38 -0400 Received: from mail-yw1-x114a.google.com (mail-yw1-x114a.google.com [IPv6:2607:f8b0:4864:20::114a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DA09B1709 for ; Tue, 20 Jun 2023 23:43:32 -0700 (PDT) Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-56cf9a86277so64701257b3.3 for ; Tue, 20 Jun 2023 23:43:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1687329812; x=1689921812; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=9vic1ayWBGr79y9f6y1FKqdm9/OskuVUCQ3Tr10KopQ=; b=RHaZJpocBAQex9N5sK+TVgzUKOuPn86MHuZT6EA4DsHyu3bgrSiJ/6leoQ9tr/MTNx fVcPxmgqsOc7bhA2bsToHFQk+6KAT5J8fjFvvBWV+GJ4hDTWKk5sDKZrS6MdI7si7KLi U0w462zitIRC0tsUnPZ2VYTHS65xaYWpUBs3saGloaPiHLGGDwtc031gios/uY13QTlj SLI28LI7OgWZJ/KaLkGSiBakAq8YtZIW1WVFV/BHehBRDjzQk0BaP4UntwQQWNb9F+I3 WNjFWpxs2VsOOSkh253vfrsykGVgRTXkImRD20J3FM6r7C4ZtfnPpiaPfw4QU//NgebR VONQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687329812; x=1689921812; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=9vic1ayWBGr79y9f6y1FKqdm9/OskuVUCQ3Tr10KopQ=; b=iry8OVdYcROmVOCX1TzbhtUzbOFGG2sqXhMcvwSC00foUcgZsEqy8Vk8tsofkgIrY+ zP8WBX+yMLywhMASZKFI6eQYzqzTVVyLMISMYmqC38NWFdYj4akEsVJdBON4Pet811Xi wovbksTsJ36z7Df+GWbdWApsh4cCveL0lJrIEyXE7YsqHoPBks9rw+vk7FnKTibmjzBT VTD2XhthnH263VivTiEqS59D27QXdJZHR16EaUoahnBDV//f81/Qq+ePMqYZrGZhW7wD Bc0dBRPcFRSD4USZpv0/9gtxUjGdDcuipjftDwonU3r0JHIHPoVNVngpPv0kauh7HLnJ joqQ== X-Gm-Message-State: AC+VfDwYf0+wfwVPAoJjpoyU74cPkOm7D/L2hmJ6FC7tWLQ1rZnaXUVj okpzMcVbcMaVexx+UA48m6OIl1xIVaoE X-Google-Smtp-Source: ACHHUZ4TXGK0i+D7h+9Nd4z8i5gJT8bGw52lE7GnL9ZKETGpo9OOKiKvObn0KiD3N3+R9rM2oZ7yQsyYMx8u X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:384f:f7da:c61d:5a3e]) (user=mshavit job=sendgmr) by 2002:a25:c514:0:b0:ba8:b828:c8ff with SMTP id v20-20020a25c514000000b00ba8b828c8ffmr1585519ybe.10.1687329812105; Tue, 20 Jun 2023 23:43:32 -0700 (PDT) Date: Wed, 21 Jun 2023 14:37:14 +0800 In-Reply-To: <20230621063825.268890-1-mshavit@google.com> Mime-Version: 1.0 References: <20230621063825.268890-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230621063825.268890-3-mshavit@google.com> Subject: [PATCH v4 02/13] iommu/arm-smmu-v3: Add smmu_s1_cfg to smmu_master From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Except for Nested domains, arm_smmu_master will own the STEs that are inserted into the arm_smmu_device's STE table. Signed-off-by: Michael Shavit --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 28 +++++++++++++-------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 1 + 2 files changed, 18 insertions(+), 11 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index beff04b897718..023769f5ca79a 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1126,15 +1126,16 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain = *smmu_domain, int ssid, return 0; } =20 -static int arm_smmu_alloc_cd_tables(struct arm_smmu_domain *smmu_domain) +static int arm_smmu_init_s1_cfg(struct arm_smmu_master *master, + struct arm_smmu_s1_cfg *cfg) { int ret; size_t l1size; size_t max_contexts; - struct arm_smmu_device *smmu =3D smmu_domain->smmu; - struct arm_smmu_s1_cfg *cfg =3D &smmu_domain->s1_cfg; + struct arm_smmu_device *smmu =3D master->smmu; struct arm_smmu_ctx_desc_cfg *cdcfg =3D &cfg->cdcfg; =20 + cfg->s1cdmax =3D master->ssid_bits; max_contexts =3D 1 << cfg->s1cdmax; =20 if (!(smmu->features & ARM_SMMU_FEAT_2_LVL_CDTAB) || @@ -1175,12 +1176,11 @@ static int arm_smmu_alloc_cd_tables(struct arm_smmu= _domain *smmu_domain) return ret; } =20 -static void arm_smmu_free_cd_tables(struct arm_smmu_domain *smmu_domain) +static void arm_smmu_free_cd_tables(struct arm_smmu_device *smmu, + struct arm_smmu_ctx_desc_cfg *cdcfg) { int i; size_t size, l1size; - struct arm_smmu_device *smmu =3D smmu_domain->smmu; - struct arm_smmu_ctx_desc_cfg *cdcfg =3D &smmu_domain->s1_cfg.cdcfg; =20 if (cdcfg->l1_desc) { size =3D CTXDESC_L2_ENTRIES * (CTXDESC_CD_DWORDS << 3); @@ -2076,7 +2076,7 @@ static void arm_smmu_domain_free(struct iommu_domain = *domain) /* Prevent SVA from touching the CD while we're freeing it */ mutex_lock(&arm_smmu_asid_lock); if (cfg->cdcfg.cdtab) - arm_smmu_free_cd_tables(smmu_domain); + arm_smmu_free_cd_tables(smmu, &cfg->cdcfg); arm_smmu_free_asid(&smmu_domain->cd); mutex_unlock(&arm_smmu_asid_lock); } else { @@ -2108,11 +2108,9 @@ static int arm_smmu_domain_finalise_s1(struct arm_sm= mu_domain *smmu_domain, if (ret) goto out_unlock; =20 - cfg->s1cdmax =3D master->ssid_bits; - smmu_domain->stall_enabled =3D master->stall_enabled; =20 - ret =3D arm_smmu_alloc_cd_tables(smmu_domain); + ret =3D arm_smmu_init_s1_cfg(master, cfg); if (ret) goto out_free_asid; =20 @@ -2140,7 +2138,7 @@ static int arm_smmu_domain_finalise_s1(struct arm_smm= u_domain *smmu_domain, return 0; =20 out_free_cd_tables: - arm_smmu_free_cd_tables(smmu_domain); + arm_smmu_free_cd_tables(smmu, &cfg->cdcfg); out_free_asid: arm_smmu_free_asid(cd); out_unlock: @@ -2704,6 +2702,13 @@ static struct iommu_device *arm_smmu_probe_device(st= ruct device *dev) smmu->features & ARM_SMMU_FEAT_STALL_FORCE) master->stall_enabled =3D true; =20 + ret =3D arm_smmu_init_s1_cfg(master, &master->owned_s1_cfg); + if (ret) { + arm_smmu_disable_pasid(master); + arm_smmu_remove_master(master); + goto err_free_master; + } + return &smmu->iommu; =20 err_free_master: @@ -2719,6 +2724,7 @@ static void arm_smmu_release_device(struct device *de= v) if (WARN_ON(arm_smmu_master_sva_enabled(master))) iopf_queue_remove_device(master->smmu->evtq.iopf, dev); arm_smmu_detach_dev(master); + arm_smmu_free_cd_tables(master->smmu, &master->owned_s1_cfg.cdcfg); arm_smmu_disable_pasid(master); arm_smmu_remove_master(master); kfree(master); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.h index 68d519f21dbd8..053cc14c23969 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -688,6 +688,7 @@ struct arm_smmu_master { struct arm_smmu_domain *domain; struct list_head domain_head; struct arm_smmu_stream *streams; + struct arm_smmu_s1_cfg owned_s1_cfg; unsigned int num_streams; bool ats_enabled; bool stall_enabled; --=20 2.41.0.162.gfafddb0af9-goog From nobody Sun Feb 8 07:15:01 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9194EEB64D7 for ; Wed, 21 Jun 2023 06:44:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231161AbjFUGoL (ORCPT ); Wed, 21 Jun 2023 02:44:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39944 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230489AbjFUGnn (ORCPT ); Wed, 21 Jun 2023 02:43:43 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CA328198E for ; Tue, 20 Jun 2023 23:43:40 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-bd69bb4507eso6010283276.2 for ; Tue, 20 Jun 2023 23:43:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1687329820; x=1689921820; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=jzXQy+3gWCRrOq/bPJGSod35GlRHXYXuXCqPkpYte2c=; b=rvLyDf3473Ea5KgbE9MzK4m81ED5MczQdqagmULEilVLDvyqXUuA956dDqxQL8afkd I/KMEhrIM3w+cmdo4c6BxJzNUY5csQMLavWP2+i17nXGNqCj+ZP70yPu0bi+pTgs225a rAZ2HNjpybLfLtg0BQ98c3tHjAB9BijbI8q7UEO3+8148457WZur9CdIO5YNp4lm+Vdl Rq0CSZOcZtZN/6WN2azDEvWHFdlHaHUH9P77mTDTDjJVMQbgx5Uae087uO5s5uldv2nM l7OWmzWlWv7Q89Y0t3V/kW5ayh5OzqWwRmyT8IrUe+tNL09iKKoA09VyeydJpGmNpmxs Pbww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687329820; x=1689921820; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=jzXQy+3gWCRrOq/bPJGSod35GlRHXYXuXCqPkpYte2c=; b=llm04Lonz2x7I5/uzi1EAewIWMJNz0FGpSBjP6HrcDTy0n/mP2Fa1kMdIaZe5726Xi jvMslTBBesh3SzGImpvmnd3jf19ya6wQJypPL9x0XGlf9VpxIFDhvbBroxQnxw25KrTU kgoM8ko1mxtAjioj2DttGhgo2oPKVkmQc2KAeMIrqxKRzvnTK5l6f8jS1bLBLTUB2moG J006xuaGmK31VuQ1tbNm5YPKbMHVcJDDuNpwsaJGl9C1QJweEex0N6FWxomPhmLmZMAi KMGX02Jp98nYg6ZXmmqK3rszrKJYQAMKGQZHU8LIX8dmrubbSoDCP0M36tK8dPzDFDnP jlsQ== X-Gm-Message-State: AC+VfDxjKbRGTpESdtJitsCsvxYwBmeGjYNMuI8/k1+K6LXGlt/BLLX8 Idgg463w5xnZEoyQSbFN28ED+9pKkE37 X-Google-Smtp-Source: ACHHUZ6o7pYjIQ5aJehf2aHfK+lSFLWTkcaxHxTVWaWUeBs4KhACfgfWfn/usRnAdFLLIcrDe6s0cY2LaPQS X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:384f:f7da:c61d:5a3e]) (user=mshavit job=sendgmr) by 2002:a05:6902:15c4:b0:be4:d646:48d8 with SMTP id l4-20020a05690215c400b00be4d64648d8mr1659256ybu.8.1687329820101; Tue, 20 Jun 2023 23:43:40 -0700 (PDT) Date: Wed, 21 Jun 2023 14:37:15 +0800 In-Reply-To: <20230621063825.268890-1-mshavit@google.com> Mime-Version: 1.0 References: <20230621063825.268890-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230621063825.268890-4-mshavit@google.com> Subject: [PATCH v4 03/13] iommu/arm-smmu-v3: Refactor write_strtab_ent From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Explicity keep track of the s1_cfg and s2_cfg that are attached to a master in arm_smmu_master, regardless of whether they are owned by arm_smmu_master, arm_smmu_domain or userspace. Signed-off-by: Michael Shavit --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 37 +++++++++------------ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 2 ++ 2 files changed, 17 insertions(+), 22 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index 023769f5ca79a..d79c6ef5d6ed4 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1269,10 +1269,9 @@ static void arm_smmu_write_strtab_ent(struct arm_smm= u_master *master, u32 sid, */ u64 val =3D le64_to_cpu(dst[0]); bool ste_live =3D false; - struct arm_smmu_device *smmu =3D NULL; + struct arm_smmu_device *smmu =3D master->smmu; struct arm_smmu_s1_cfg *s1_cfg =3D NULL; struct arm_smmu_s2_cfg *s2_cfg =3D NULL; - struct arm_smmu_domain *smmu_domain =3D NULL; struct arm_smmu_cmdq_ent prefetch_cmd =3D { .opcode =3D CMDQ_OP_PREFETCH_CFG, .prefetch =3D { @@ -1280,24 +1279,10 @@ static void arm_smmu_write_strtab_ent(struct arm_sm= mu_master *master, u32 sid, }, }; =20 - if (master) { - smmu_domain =3D master->domain; - smmu =3D master->smmu; - } - - if (smmu_domain) { - switch (smmu_domain->stage) { - case ARM_SMMU_DOMAIN_S1: - s1_cfg =3D &smmu_domain->s1_cfg; - break; - case ARM_SMMU_DOMAIN_S2: - case ARM_SMMU_DOMAIN_NESTED: - s2_cfg =3D &smmu_domain->s2_cfg; - break; - default: - break; - } - } + if (master->s1_cfg) + s1_cfg =3D master->s1_cfg; + else if (master->s2_cfg) + s2_cfg =3D master->s2_cfg; =20 if (val & STRTAB_STE_0_V) { switch (FIELD_GET(STRTAB_STE_0_CFG, val)) { @@ -1319,8 +1304,8 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu= _master *master, u32 sid, val =3D STRTAB_STE_0_V; =20 /* Bypass/fault */ - if (!smmu_domain || !(s1_cfg || s2_cfg)) { - if (!smmu_domain && disable_bypass) + if (!(s1_cfg || s2_cfg)) { + if (disable_bypass) val |=3D FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_ABORT); else val |=3D FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_BYPASS); @@ -2401,6 +2386,8 @@ static void arm_smmu_detach_dev(struct arm_smmu_maste= r *master) =20 master->domain =3D NULL; master->ats_enabled =3D false; + master->s1_cfg =3D NULL; + master->s2_cfg =3D NULL; arm_smmu_install_ste_for_dev(master); } =20 @@ -2454,6 +2441,12 @@ static int arm_smmu_attach_dev(struct iommu_domain *= domain, struct device *dev) } =20 master->domain =3D smmu_domain; + if (smmu_domain->stage =3D=3D ARM_SMMU_DOMAIN_S1) { + master->s1_cfg =3D &smmu_domain->s1_cfg; + } else if (smmu_domain->stage =3D=3D ARM_SMMU_DOMAIN_S2 || + smmu_domain->stage =3D=3D ARM_SMMU_DOMAIN_NESTED) { + master->s2_cfg =3D &smmu_domain->s2_cfg; + } =20 /* * The SMMU does not support enabling ATS with bypass. When the STE is diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.h index 053cc14c23969..3c614fbe2b8b9 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -689,6 +689,8 @@ struct arm_smmu_master { struct list_head domain_head; struct arm_smmu_stream *streams; struct arm_smmu_s1_cfg owned_s1_cfg; + struct arm_smmu_s1_cfg *s1_cfg; + struct arm_smmu_s2_cfg *s2_cfg; unsigned int num_streams; bool ats_enabled; bool stall_enabled; --=20 2.41.0.162.gfafddb0af9-goog From nobody Sun Feb 8 07:15:01 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8574EB64D8 for ; Wed, 21 Jun 2023 06:44:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231180AbjFUGoX (ORCPT ); Wed, 21 Jun 2023 02:44:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40136 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230515AbjFUGnv (ORCPT ); Wed, 21 Jun 2023 02:43:51 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3E8DC19A5 for ; Tue, 20 Jun 2023 23:43:49 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-bd5b8a9d82cso6185852276.0 for ; Tue, 20 Jun 2023 23:43:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1687329828; x=1689921828; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=T3tJEpWHCJm3S04xQ0T/8NFLZuI/7sadQhRIHV1s9Zo=; b=tDLP783tJ0VQXKySnXfcycNFVcu1CwUxp8tUrTH0xu2NVvWELKSyiZsFc93tdsXdvB xETbRFAM6aKkKwECQOiXpyJILPwdCxEBih5pnCop8eUyqe0rhCwyzuViEs0sbi5rGrej 9rL1XXQz69682N/N0/xt2Ohaot08bo5JQ9YQ/7IvhK9fLsoPnGVAokhZW6l/U5V/54UZ dA4BV3jTZuZEfgKZnQXiYtuBp4sLNnewc3vxY6RJ0YJwjoeeizsZilVlVAnwctKlJt3T u7RzkL909yZir1kXSjPGUWrgU6j5aLvYF7i8+sm5hJkwQTUqGkvq3HSxVSeKXV8QvaHd RFmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687329828; x=1689921828; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=T3tJEpWHCJm3S04xQ0T/8NFLZuI/7sadQhRIHV1s9Zo=; b=AjRRRDy/PTTcGQ6aojT5C99P9fyW2hOdgWVwvXygeESeFjE6Kn5Wt7ehTUI9lga9D4 WnUiTXBAX3no2PK2mGYxxM6CFqQyvTXhxJRxTLBV6su3dkdiPqBuNwD9koRbAoq/Cc+D bIoDdC+op7BeSyY25pCrG7rFXeaB/MGW3Gexl57M6TWsYpYDn8H1U/sTqK1PZmm+Ag3G udOo5Cm5nc1p6XZkl5Fk+XIRojCIeSj1NNo2b/ViPe8Lps17joDXQV78lGgbBC8839it dynA0bTVrNGkC1nJ2Q6BFnO85UZZod9JU9w+ikxs37RybAJlZIgSspqDl/mDEryUl537 jPeQ== X-Gm-Message-State: AC+VfDx3n+EBGWN13GZFMcD4TtInD8dOYx20A4UMg/imHbJPkQo4Pnx9 /yAw6AK+YDkgwXYXg3r8UdX9jW8Wlv+8 X-Google-Smtp-Source: ACHHUZ7yq7ycY2qeb9AdhoMG/AMIx0w4M48vH2hQ0IrPDr4T8WnV7iAjLytDkfMdBW5HapJgIPYK6BBZnZgS X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:384f:f7da:c61d:5a3e]) (user=mshavit job=sendgmr) by 2002:a25:748e:0:b0:bac:fd63:b567 with SMTP id p136-20020a25748e000000b00bacfd63b567mr5997224ybc.4.1687329828409; Tue, 20 Jun 2023 23:43:48 -0700 (PDT) Date: Wed, 21 Jun 2023 14:37:16 +0800 In-Reply-To: <20230621063825.268890-1-mshavit@google.com> Mime-Version: 1.0 References: <20230621063825.268890-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230621063825.268890-5-mshavit@google.com> Subject: [PATCH v4 04/13] iommu/arm-smmu-v3: Refactor write_ctx_desc From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Update arm_smmu_write_ctx_desc and downstream functions to be agnostic of the CD table's ownership. Note that in practice, it will only be called in cases where the table is owned by arm_smmu_master. Whether arm_smmu_write_ctx_desc will trigger a sync is also made part of the API. Note that this change isn't a nop refactor since SVA will call arm_smmu_write_ctx_desc in a loop for every master the domain is attached to despite the fact that they all share the same CD table. Note that the next commit ceases this sharing of the s1_cfg which makes that loop necessary. Signed-off-by: Michael Shavit --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 36 ++++++++- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 80 +++++++++++-------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 6 +- 3 files changed, 81 insertions(+), 41 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iomm= u/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 968559d625c40..48fa8eb271a45 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -45,10 +45,12 @@ static struct arm_smmu_ctx_desc * arm_smmu_share_asid(struct mm_struct *mm, u16 asid) { int ret; + unsigned long flags; u32 new_asid; struct arm_smmu_ctx_desc *cd; struct arm_smmu_device *smmu; struct arm_smmu_domain *smmu_domain; + struct arm_smmu_master *master; =20 cd =3D xa_load(&arm_smmu_asid_xa, asid); if (!cd) @@ -80,7 +82,11 @@ arm_smmu_share_asid(struct mm_struct *mm, u16 asid) * be some overlap between use of both ASIDs, until we invalidate the * TLB. */ - arm_smmu_write_ctx_desc(smmu_domain, 0, cd); + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_for_each_entry(master, &smmu_domain->devices, domain_head) { + arm_smmu_write_ctx_desc(smmu, master->s1_cfg, master, 0, cd); + } + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); =20 /* Invalidate TLB entries previously associated with that context */ arm_smmu_tlb_inv_asid(smmu, asid); @@ -211,6 +217,8 @@ static void arm_smmu_mm_release(struct mmu_notifier *mn= , struct mm_struct *mm) { struct arm_smmu_mmu_notifier *smmu_mn =3D mn_to_smmu(mn); struct arm_smmu_domain *smmu_domain =3D smmu_mn->domain; + struct arm_smmu_master *master; + unsigned long flags; =20 mutex_lock(&sva_lock); if (smmu_mn->cleared) { @@ -222,7 +230,12 @@ static void arm_smmu_mm_release(struct mmu_notifier *m= n, struct mm_struct *mm) * DMA may still be running. Keep the cd valid to avoid C_BAD_CD events, * but disable translation. */ - arm_smmu_write_ctx_desc(smmu_domain, mm->pasid, &quiet_cd); + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_for_each_entry(master, &smmu_domain->devices, domain_head) { + arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, + mm->pasid, &quiet_cd); + } + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); =20 arm_smmu_tlb_inv_asid(smmu_domain->smmu, smmu_mn->cd->asid); arm_smmu_atc_inv_domain(smmu_domain, mm->pasid, 0, 0); @@ -248,8 +261,10 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu= _domain, struct mm_struct *mm) { int ret; + unsigned long flags; struct arm_smmu_ctx_desc *cd; struct arm_smmu_mmu_notifier *smmu_mn; + struct arm_smmu_master *master; =20 list_for_each_entry(smmu_mn, &smmu_domain->mmu_notifiers, list) { if (smmu_mn->mn.mm =3D=3D mm) { @@ -279,7 +294,12 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu= _domain, goto err_free_cd; } =20 - ret =3D arm_smmu_write_ctx_desc(smmu_domain, mm->pasid, cd); + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_for_each_entry(master, &smmu_domain->devices, domain_head) { + ret =3D arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, + master, mm->pasid, cd); + } + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); if (ret) goto err_put_notifier; =20 @@ -296,15 +316,23 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smm= u_domain, =20 static void arm_smmu_mmu_notifier_put(struct arm_smmu_mmu_notifier *smmu_m= n) { + unsigned long flags; struct mm_struct *mm =3D smmu_mn->mn.mm; struct arm_smmu_ctx_desc *cd =3D smmu_mn->cd; + struct arm_smmu_master *master; struct arm_smmu_domain *smmu_domain =3D smmu_mn->domain; =20 if (!refcount_dec_and_test(&smmu_mn->refs)) return; =20 list_del(&smmu_mn->list); - arm_smmu_write_ctx_desc(smmu_domain, mm->pasid, NULL); + + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_for_each_entry(master, &smmu_domain->devices, domain_head) { + arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, + mm->pasid, NULL); + } + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); =20 /* * If we went through clear(), we've already invalidated, and no diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index d79c6ef5d6ed4..b6f7cf60f8f3d 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -965,14 +965,13 @@ void arm_smmu_tlb_inv_asid(struct arm_smmu_device *sm= mu, u16 asid) arm_smmu_cmdq_issue_cmd_with_sync(smmu, &cmd); } =20 -static void arm_smmu_sync_cd(struct arm_smmu_domain *smmu_domain, +/* master may be null */ +static void arm_smmu_sync_cd(struct arm_smmu_master *master, int ssid, bool leaf) { size_t i; - unsigned long flags; - struct arm_smmu_master *master; struct arm_smmu_cmdq_batch cmds; - struct arm_smmu_device *smmu =3D smmu_domain->smmu; + struct arm_smmu_device *smmu; struct arm_smmu_cmdq_ent cmd =3D { .opcode =3D CMDQ_OP_CFGI_CD, .cfgi =3D { @@ -981,16 +980,15 @@ static void arm_smmu_sync_cd(struct arm_smmu_domain *= smmu_domain, }, }; =20 - cmds.num =3D 0; + if (!master) + return; =20 - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { - for (i =3D 0; i < master->num_streams; i++) { - cmd.cfgi.sid =3D master->streams[i].id; - arm_smmu_cmdq_batch_add(smmu, &cmds, &cmd); - } + smmu =3D master->smmu; + cmds.num =3D 0; + for (i =3D 0; i < master->num_streams; i++) { + cmd.cfgi.sid =3D master->streams[i].id; + arm_smmu_cmdq_batch_add(smmu, &cmds, &cmd); } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); =20 arm_smmu_cmdq_batch_submit(smmu, &cmds); } @@ -1020,16 +1018,18 @@ static void arm_smmu_write_cd_l1_desc(__le64 *dst, WRITE_ONCE(*dst, cpu_to_le64(val)); } =20 -static __le64 *arm_smmu_get_cd_ptr(struct arm_smmu_domain *smmu_domain, +/* master may be null */ +static __le64 *arm_smmu_get_cd_ptr(struct arm_smmu_device *smmu, + struct arm_smmu_s1_cfg *s1_cfg, + struct arm_smmu_master *master, u32 ssid) { __le64 *l1ptr; unsigned int idx; struct arm_smmu_l1_ctx_desc *l1_desc; - struct arm_smmu_device *smmu =3D smmu_domain->smmu; - struct arm_smmu_ctx_desc_cfg *cdcfg =3D &smmu_domain->s1_cfg.cdcfg; + struct arm_smmu_ctx_desc_cfg *cdcfg =3D &s1_cfg->cdcfg; =20 - if (smmu_domain->s1_cfg.s1fmt =3D=3D STRTAB_STE_0_S1FMT_LINEAR) + if (s1_cfg->s1fmt =3D=3D STRTAB_STE_0_S1FMT_LINEAR) return cdcfg->cdtab + ssid * CTXDESC_CD_DWORDS; =20 idx =3D ssid >> CTXDESC_SPLIT; @@ -1041,13 +1041,21 @@ static __le64 *arm_smmu_get_cd_ptr(struct arm_smmu_= domain *smmu_domain, l1ptr =3D cdcfg->cdtab + idx * CTXDESC_L1_DESC_DWORDS; arm_smmu_write_cd_l1_desc(l1ptr, l1_desc); /* An invalid L1CD can be cached */ - arm_smmu_sync_cd(smmu_domain, ssid, false); + arm_smmu_sync_cd(master, ssid, false); } idx =3D ssid & (CTXDESC_L2_ENTRIES - 1); return l1_desc->l2ptr + idx * CTXDESC_CD_DWORDS; } =20 -int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, +/* + * master must be provided if a CD sync is required but may be null otherw= ise + * (such as when the CD table isn't inserted into the STE yet, or is about= to + * be detached. + */ +int arm_smmu_write_ctx_desc(struct arm_smmu_device *smmu, + struct arm_smmu_s1_cfg *s1_cfg, + struct arm_smmu_master *master, + int ssid, struct arm_smmu_ctx_desc *cd) { /* @@ -1065,10 +1073,10 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain = *smmu_domain, int ssid, bool cd_live; __le64 *cdptr; =20 - if (WARN_ON(ssid >=3D (1 << smmu_domain->s1_cfg.s1cdmax))) + if (WARN_ON(ssid >=3D (1 << s1_cfg->s1cdmax))) return -E2BIG; =20 - cdptr =3D arm_smmu_get_cd_ptr(smmu_domain, ssid); + cdptr =3D arm_smmu_get_cd_ptr(smmu, s1_cfg, master, ssid); if (!cdptr) return -ENOMEM; =20 @@ -1092,11 +1100,11 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain = *smmu_domain, int ssid, cdptr[3] =3D cpu_to_le64(cd->mair); =20 /* - * STE is live, and the SMMU might read dwords of this CD in any - * order. Ensure that it observes valid values before reading - * V=3D1. + * STE may be live, and the SMMU might read dwords of this CD + * in any order. Ensure that it observes valid values before + * reading V=3D1. */ - arm_smmu_sync_cd(smmu_domain, ssid, true); + arm_smmu_sync_cd(master, ssid, true); =20 val =3D cd->tcr | #ifdef __BIG_ENDIAN @@ -1108,7 +1116,7 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *s= mmu_domain, int ssid, FIELD_PREP(CTXDESC_CD_0_ASID, cd->asid) | CTXDESC_CD_0_V; =20 - if (smmu_domain->stall_enabled) + if (s1_cfg->stall_enabled) val |=3D CTXDESC_CD_0_S; } =20 @@ -1122,7 +1130,7 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *s= mmu_domain, int ssid, * without first making the structure invalid. */ WRITE_ONCE(cdptr[0], cpu_to_le64(val)); - arm_smmu_sync_cd(smmu_domain, ssid, true); + arm_smmu_sync_cd(master, ssid, true); return 0; } =20 @@ -1136,6 +1144,7 @@ static int arm_smmu_init_s1_cfg(struct arm_smmu_maste= r *master, struct arm_smmu_ctx_desc_cfg *cdcfg =3D &cfg->cdcfg; =20 cfg->s1cdmax =3D master->ssid_bits; + cfg->stall_enabled =3D master->stall_enabled; max_contexts =3D 1 << cfg->s1cdmax; =20 if (!(smmu->features & ARM_SMMU_FEAT_2_LVL_CDTAB) || @@ -2093,8 +2102,6 @@ static int arm_smmu_domain_finalise_s1(struct arm_smm= u_domain *smmu_domain, if (ret) goto out_unlock; =20 - smmu_domain->stall_enabled =3D master->stall_enabled; - ret =3D arm_smmu_init_s1_cfg(master, cfg); if (ret) goto out_free_asid; @@ -2110,12 +2117,9 @@ static int arm_smmu_domain_finalise_s1(struct arm_sm= mu_domain *smmu_domain, CTXDESC_CD_0_TCR_EPD1 | CTXDESC_CD_0_AA64; cd->mair =3D pgtbl_cfg->arm_lpae_s1_cfg.mair; =20 - /* - * Note that this will end up calling arm_smmu_sync_cd() before - * the master has been added to the devices list for this domain. - * This isn't an issue because the STE hasn't been installed yet. - */ - ret =3D arm_smmu_write_ctx_desc(smmu_domain, 0, cd); + ret =3D arm_smmu_write_ctx_desc(smmu, cfg, + NULL /*Not attached to a master yet */, + 0, cd); if (ret) goto out_free_cd_tables; =20 @@ -2386,6 +2390,11 @@ static void arm_smmu_detach_dev(struct arm_smmu_mast= er *master) =20 master->domain =3D NULL; master->ats_enabled =3D false; + if (master->s1_cfg) + arm_smmu_write_ctx_desc( + master->smmu, master->s1_cfg, + NULL /* Skip sync since we detach the CD table next*/, + 0, NULL); master->s1_cfg =3D NULL; master->s2_cfg =3D NULL; arm_smmu_install_ste_for_dev(master); @@ -2435,7 +2444,8 @@ static int arm_smmu_attach_dev(struct iommu_domain *d= omain, struct device *dev) ret =3D -EINVAL; goto out_unlock; } else if (smmu_domain->stage =3D=3D ARM_SMMU_DOMAIN_S1 && - smmu_domain->stall_enabled !=3D master->stall_enabled) { + smmu_domain->s1_cfg.stall_enabled !=3D + master->stall_enabled) { ret =3D -EINVAL; goto out_unlock; } diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.h index 3c614fbe2b8b9..00a493442d6f9 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -595,6 +595,7 @@ struct arm_smmu_s1_cfg { struct arm_smmu_ctx_desc_cfg cdcfg; u8 s1fmt; u8 s1cdmax; + bool stall_enabled; }; =20 struct arm_smmu_s2_cfg { @@ -713,7 +714,6 @@ struct arm_smmu_domain { struct mutex init_mutex; /* Protects smmu pointer */ =20 struct io_pgtable_ops *pgtbl_ops; - bool stall_enabled; atomic_t nr_ats_masters; =20 enum arm_smmu_domain_stage stage; @@ -742,7 +742,9 @@ extern struct xarray arm_smmu_asid_xa; extern struct mutex arm_smmu_asid_lock; extern struct arm_smmu_ctx_desc quiet_cd; =20 -int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid, +int arm_smmu_write_ctx_desc(struct arm_smmu_device *smmu, + struct arm_smmu_s1_cfg *s1_cfg, + struct arm_smmu_master *smmu_master, int ssid, struct arm_smmu_ctx_desc *cd); void arm_smmu_tlb_inv_asid(struct arm_smmu_device *smmu, u16 asid); void arm_smmu_tlb_inv_range_asid(unsigned long iova, size_t size, int asid, --=20 2.41.0.162.gfafddb0af9-goog From nobody Sun Feb 8 07:15:01 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 86174EB64DC for ; Wed, 21 Jun 2023 06:44:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229727AbjFUGoh (ORCPT ); Wed, 21 Jun 2023 02:44:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40292 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230527AbjFUGoD (ORCPT ); Wed, 21 Jun 2023 02:44:03 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 370231BCF for ; Tue, 20 Jun 2023 23:43:57 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-bcef239a9fbso13159717276.1 for ; Tue, 20 Jun 2023 23:43:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1687329836; x=1689921836; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=6vzESexuWCV4LLT4ZlWlOvb8u8164/HANB5QcSOGct0=; b=svmsiobx2Wo1tGVdmspzGY8XKIT2RMMfZsVIGP8+D0VskhD3W00QvnB74yg6w3Znfg FF0PHmvC5XVe190m9JDChplAtynZBkJ4zmC/GZXSj4PHFQu9Po+4/dccCy/wbGuWhmxh se005ZBXaisFJn7BGyDl48Svnfz+/u/MRUw7ZsoPofZBjic/m2iZTrsM/fqSlADWGFY/ De6MKp6cEBEd3qxpL6wwFrJQp9hAuRrVmgyq3yIR5w/vn99QHwt8yL/0Jng0DUo0EAV9 NZlsTatB05xTsImbbonTm96Wvoi3kQc1kObhyM6VDa93ATuNWP6gZ4K7nJlIb1va4fXm sPbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687329836; x=1689921836; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=6vzESexuWCV4LLT4ZlWlOvb8u8164/HANB5QcSOGct0=; b=g8WEareeH0eyVpZANwkSYE5Xl8yxjHq6jXe0n0dny8ECzA0cJbuvxJZkqWZ4HAG6iz As4sF2PswoGzBvBazG2ao+qKT4RRp6KtKsr2SFGH6ksiC3zTJcwCCZGgqrs+0WNB87UP iqG6T3Lw/54wSkhLr6W6E8tJdaLycP0KqF+Gr5T9cLOs8YJh/xLPE5NFBCdcWlb2TnQ+ j1SjfX3B0RkcWdM89JG6Hjzqd+im+iD8nwaJDPdK7aR1fIT84SQFg1yJVdFHxBB5EaHR gNgjMdNxFJH9/0K+HwaSkb5nAVRaqk1dkfOnaoNm6eDzWFpmMPnDMxtFlFHyJnLlTFHn YgLQ== X-Gm-Message-State: AC+VfDyrOC9V3AQae0dMJLYuCh3QlMv4tFkEFdLIeHTSAX8axPcwOzTm A4BY24giDqzPY17OGm0PyDYf4aJs1Ugt X-Google-Smtp-Source: ACHHUZ5OSHRehe3DXZzs9l8Ry/Fyn0ESmjipNH2+kktnVhz8AkYddX0OFhLKRoI4AmtAbrv6Hza5MotMUTju X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:384f:f7da:c61d:5a3e]) (user=mshavit job=sendgmr) by 2002:a25:abee:0:b0:ba8:4ff5:3214 with SMTP id v101-20020a25abee000000b00ba84ff53214mr3393703ybi.2.1687329836534; Tue, 20 Jun 2023 23:43:56 -0700 (PDT) Date: Wed, 21 Jun 2023 14:37:17 +0800 In-Reply-To: <20230621063825.268890-1-mshavit@google.com> Mime-Version: 1.0 References: <20230621063825.268890-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230621063825.268890-6-mshavit@google.com> Subject: [PATCH v4 05/13] iommu/arm-smmu-v3: Use the master-owned s1_cfg From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Insert CDs for STAGE_1 domains into a CD table owned by the arm_smmu_master. Remove the CD table that was owned by arm_smmu_domain. Signed-off-by: Michael Shavit --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 43 ++++++--------------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 3 -- 2 files changed, 12 insertions(+), 34 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index b6f7cf60f8f3d..08f440fe1da6d 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2065,12 +2065,8 @@ static void arm_smmu_domain_free(struct iommu_domain= *domain) =20 /* Free the CD and ASID, if we allocated them */ if (smmu_domain->stage =3D=3D ARM_SMMU_DOMAIN_S1) { - struct arm_smmu_s1_cfg *cfg =3D &smmu_domain->s1_cfg; - /* Prevent SVA from touching the CD while we're freeing it */ mutex_lock(&arm_smmu_asid_lock); - if (cfg->cdcfg.cdtab) - arm_smmu_free_cd_tables(smmu, &cfg->cdcfg); arm_smmu_free_asid(&smmu_domain->cd); mutex_unlock(&arm_smmu_asid_lock); } else { @@ -2082,14 +2078,13 @@ static void arm_smmu_domain_free(struct iommu_domai= n *domain) kfree(smmu_domain); } =20 -static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, +static int arm_smmu_domain_finalise_cd(struct arm_smmu_domain *smmu_domain, struct arm_smmu_master *master, struct io_pgtable_cfg *pgtbl_cfg) { int ret; u32 asid; struct arm_smmu_device *smmu =3D smmu_domain->smmu; - struct arm_smmu_s1_cfg *cfg =3D &smmu_domain->s1_cfg; struct arm_smmu_ctx_desc *cd =3D &smmu_domain->cd; typeof(&pgtbl_cfg->arm_lpae_s1_cfg.tcr) tcr =3D &pgtbl_cfg->arm_lpae_s1_c= fg.tcr; =20 @@ -2102,10 +2097,6 @@ static int arm_smmu_domain_finalise_s1(struct arm_sm= mu_domain *smmu_domain, if (ret) goto out_unlock; =20 - ret =3D arm_smmu_init_s1_cfg(master, cfg); - if (ret) - goto out_free_asid; - cd->asid =3D (u16)asid; cd->ttbr =3D pgtbl_cfg->arm_lpae_s1_cfg.ttbr; cd->tcr =3D FIELD_PREP(CTXDESC_CD_0_TCR_T0SZ, tcr->tsz) | @@ -2117,19 +2108,9 @@ static int arm_smmu_domain_finalise_s1(struct arm_sm= mu_domain *smmu_domain, CTXDESC_CD_0_TCR_EPD1 | CTXDESC_CD_0_AA64; cd->mair =3D pgtbl_cfg->arm_lpae_s1_cfg.mair; =20 - ret =3D arm_smmu_write_ctx_desc(smmu, cfg, - NULL /*Not attached to a master yet */, - 0, cd); - if (ret) - goto out_free_cd_tables; - mutex_unlock(&arm_smmu_asid_lock); return 0; =20 -out_free_cd_tables: - arm_smmu_free_cd_tables(smmu, &cfg->cdcfg); -out_free_asid: - arm_smmu_free_asid(cd); out_unlock: mutex_unlock(&arm_smmu_asid_lock); return ret; @@ -2192,7 +2173,7 @@ static int arm_smmu_domain_finalise(struct iommu_doma= in *domain, ias =3D min_t(unsigned long, ias, VA_BITS); oas =3D smmu->ias; fmt =3D ARM_64_LPAE_S1; - finalise_stage_fn =3D arm_smmu_domain_finalise_s1; + finalise_stage_fn =3D arm_smmu_domain_finalise_cd; break; case ARM_SMMU_DOMAIN_NESTED: case ARM_SMMU_DOMAIN_S2: @@ -2439,20 +2420,20 @@ static int arm_smmu_attach_dev(struct iommu_domain = *domain, struct device *dev) } else if (smmu_domain->smmu !=3D smmu) { ret =3D -EINVAL; goto out_unlock; - } else if (smmu_domain->stage =3D=3D ARM_SMMU_DOMAIN_S1 && - master->ssid_bits !=3D smmu_domain->s1_cfg.s1cdmax) { - ret =3D -EINVAL; - goto out_unlock; - } else if (smmu_domain->stage =3D=3D ARM_SMMU_DOMAIN_S1 && - smmu_domain->s1_cfg.stall_enabled !=3D - master->stall_enabled) { - ret =3D -EINVAL; - goto out_unlock; } =20 master->domain =3D smmu_domain; if (smmu_domain->stage =3D=3D ARM_SMMU_DOMAIN_S1) { - master->s1_cfg =3D &smmu_domain->s1_cfg; + master->s1_cfg =3D &master->owned_s1_cfg; + ret =3D arm_smmu_write_ctx_desc( + smmu, + master->s1_cfg, NULL /*Not attached to a master yet */, + 0, &smmu_domain->cd); + if (ret) { + master->s1_cfg =3D NULL; + master->domain =3D NULL; + goto out_unlock; + } } else if (smmu_domain->stage =3D=3D ARM_SMMU_DOMAIN_S2 || smmu_domain->stage =3D=3D ARM_SMMU_DOMAIN_NESTED) { master->s2_cfg =3D &smmu_domain->s2_cfg; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.h index 00a493442d6f9..dff0fa8345462 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -718,10 +718,7 @@ struct arm_smmu_domain { =20 enum arm_smmu_domain_stage stage; union { - struct { struct arm_smmu_ctx_desc cd; - struct arm_smmu_s1_cfg s1_cfg; - }; struct arm_smmu_s2_cfg s2_cfg; }; =20 --=20 2.41.0.162.gfafddb0af9-goog From nobody Sun Feb 8 07:15:01 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 08E42EB64D7 for ; Wed, 21 Jun 2023 06:45:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230108AbjFUGpQ (ORCPT ); Wed, 21 Jun 2023 02:45:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40460 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231156AbjFUGoI (ORCPT ); Wed, 21 Jun 2023 02:44:08 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3E2481BF4 for ; Tue, 20 Jun 2023 23:44:05 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-bc77a2d3841so5687269276.3 for ; Tue, 20 Jun 2023 23:44:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1687329844; x=1689921844; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=kKyYB6nGtYqnelo6XDlWvQzY0JjgxL/I8n3xKSQjyKM=; b=1gaS0SkW/Sq1tXITKeGmUGTrKdPbcpqB9xL6/exaE20A1NUvR9wFlTNBFxrCnmmx1W noltPOrcY64SY3zBIRAqGGfbitthNieh+upEIdv0gNliNAenwXtXVAvyOqY984mtAd6s nZgSwJLTL85y1xBHpiEsiHarlo3Bmr5RbJEloL0e473xQwEsJxKBB34H4CbuybmXI0iF wQwBCGVX4lZWA1O+4ztFmOn8qil9C1LlaOpKaRxZ4fVPxqw4z+x6RhcaqWte8IpC0cAV lWLdyIWdY+62QfGdY9MFUSCJ+NVCLxGu5r6cyaSOI9Fao6Wur1O0Gop63zLuED1qwKea cevQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687329844; x=1689921844; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=kKyYB6nGtYqnelo6XDlWvQzY0JjgxL/I8n3xKSQjyKM=; b=dkNSv1PxXteX/8YVt7fWsO3ybQLWTTr6TCdeYayxDpIjpPtlFa0HQikbpfy6Pccd21 cCguyLkMmV6iLxp+DMTKVOnNZq83MX3kTCtGsexjH0ArnXIOOH+HnBDIV2kGPQdGqQxg CZW0xpAQncS4Dvdcu2q7ehbFTSCu5AIudDBQfp1L+tmH2iVIi/u7xVGz72nXFXy/D69n koz+4sSNIpBwWZafec5QOpJExSUXnNLyiUzo86xFIIfv6Tve9bCLQkIwJOAm/ZWkhO+j XjR4IB9oNYDjT93g3M3DENUCA5IPatpvfeCPTskulfQB0W1wVjxv7M9otKcLeGiBpcXu xS/A== X-Gm-Message-State: AC+VfDwZoy8EgHgVh7KHKvPoBSmcs0dZEhlP7/bRBil7o5e8wCqsq4fh CbLdOjlTG9hN1TdU9rfRUMopVwsIYZWo X-Google-Smtp-Source: ACHHUZ6RcCmWn2rv5cCpEtepK8VPy5yY5wftqIvcrh1WvBJEI4R+LQYSq6Xo61BWKeoCrTmvyiGy/yG28bPE X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:384f:f7da:c61d:5a3e]) (user=mshavit job=sendgmr) by 2002:a05:6902:1363:b0:bea:918f:2ef6 with SMTP id bt3-20020a056902136300b00bea918f2ef6mr1543181ybb.0.1687329844564; Tue, 20 Jun 2023 23:44:04 -0700 (PDT) Date: Wed, 21 Jun 2023 14:37:18 +0800 In-Reply-To: <20230621063825.268890-1-mshavit@google.com> Mime-Version: 1.0 References: <20230621063825.268890-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230621063825.268890-7-mshavit@google.com> Subject: [PATCH v4 06/13] iommu/arm-smmu-v3: Simplify arm_smmu_enable_ats From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" arm_smmu_enable_ats's call to inv_domain would trigger an invalidation for all masters that a domain is attached to everytime it's attached to another ATS-enabled master. It doesn't seem like those invalidations are necessary, and it's easier to reason about arm_smmu_enable_ats if it only issues invalidation commands for the current master. Signed-off-by: Michael Shavit --- v1->v2: Fix commit message wrapping --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index 08f440fe1da6d..dc7a59e87a2b4 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2286,7 +2286,7 @@ static void arm_smmu_enable_ats(struct arm_smmu_maste= r *master) pdev =3D to_pci_dev(master->dev); =20 atomic_inc(&smmu_domain->nr_ats_masters); - arm_smmu_atc_inv_domain(smmu_domain, 0, 0, 0); + arm_smmu_atc_inv_master(master); if (pci_enable_ats(pdev, stu)) dev_err(master->dev, "Failed to enable ATS (STU %zu)\n", stu); } --=20 2.41.0.162.gfafddb0af9-goog From nobody Sun Feb 8 07:15:01 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D6946EB64D7 for ; Wed, 21 Jun 2023 06:45:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230476AbjFUGpZ (ORCPT ); Wed, 21 Jun 2023 02:45:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39856 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230013AbjFUGoo (ORCPT ); Wed, 21 Jun 2023 02:44:44 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 09A001FE0 for ; Tue, 20 Jun 2023 23:44:13 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-bc8ea14f4eeso6912624276.0 for ; Tue, 20 Jun 2023 23:44:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1687329853; x=1689921853; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=3AqNtmxD8/oO/z6ww5oLsHi1wrWCEx0tKqFMiqVzP6A=; b=n2U7hoM7km5VMyO9tQ7aCYTdIMhARsT7MatmhovkwIXV2Qz88RLJgC5HnSksVvHsKm rPQaUditDHkJ6h0Ku6MbsaD2YvShF3Ix5mqqs/zYd+3vr4EOwnOWKy4lcLXuJ4/a7Tqw tvN+QaQUsh1WKWAakGfRT3yJSF+sxzPW0pwBBxxzj7GphA1Meqnip54EVYH5zl+5EATo +HWEH0NNDeX7WFiw2GsAMSdW5VFcK25Gyl6jzy7ccQmIXn9bqolKll5MbYerkBz2J37L /HOTx1oGr5IaHFiGYr+inO1jyJn+Fu1bfbbNlO6ZwGhe8h69qo1gwui0ApXeUHja4F/8 NDYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687329853; x=1689921853; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=3AqNtmxD8/oO/z6ww5oLsHi1wrWCEx0tKqFMiqVzP6A=; b=ONgxoGRQaCf7/U88Cv4lg/KV5/UxN5F+T6Ouv1oHxyAXwBBqsIZOAdcuRXJT6jcaqU UN8xaJ1CikvXpEVDAfKbcxJ+SCkQD+e4NR08qz2136877sd5WSwcY1AE4JPI47CK2wM2 RQ8UHY02Pe070n6DZdy1+6J9k1ZWE3sekdMykq+O2w18+zit9djtSdkNE8rSbqDAyJIi sGReCyp6IQyCv4PTtHuYBEx3pBlJ0/hZo3z6woid56AsqlW99o1Fj8PW74+E4rRSd8fX MwRfi/1GzCsgKc4LmYEnBMLrmg5ObzD1BpAn33REfBcYK3r0/AF1kIT0eoSOqij/KJtD rV9g== X-Gm-Message-State: AC+VfDws+qwjuSRKgd9vfS22THV2pnMeTol8EaOmUdid359s/HaqWaGE lE08MD1xUZheIWgHyv6AolHMRjGjtOtq X-Google-Smtp-Source: ACHHUZ7u4RLoF9ExTVtcKCsCN7ih9HUZf9zxl+WnFbdsFA9qkkCsufcSXAqh52YWtPGXzFxO5L7HrPptCqZc X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:384f:f7da:c61d:5a3e]) (user=mshavit job=sendgmr) by 2002:a25:7408:0:b0:ba8:929a:2073 with SMTP id p8-20020a257408000000b00ba8929a2073mr5832292ybc.1.1687329852794; Tue, 20 Jun 2023 23:44:12 -0700 (PDT) Date: Wed, 21 Jun 2023 14:37:19 +0800 In-Reply-To: <20230621063825.268890-1-mshavit@google.com> Mime-Version: 1.0 References: <20230621063825.268890-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230621063825.268890-8-mshavit@google.com> Subject: [PATCH v4 07/13] iommu/arm-smmu-v3: Keep track of attached ssids From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The arm-smmu-v3 driver keeps track of all masters that a domain is attached to so that it can re-write their STEs when the domain's ASID is upated by SVA. This tracking is also used to invalidate ATCs on all masters that a domain is attached to. This change introduces a new data structures to track all the CD entries that a domain is attached to. This change is a pre-requisite to allow domain attachment on non 0 SSIDs. Signed-off-by: Michael Shavit --- v3->v4: Remove reference to the master's domain accidentally re-introduced during a rebase. Make arm_smmu_atc_inv_domain static. v1->v2: Fix arm_smmu_atc_inv_cmd_set_ssid and other cosmetic changes --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 53 +++++++---- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 89 ++++++++++++------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 18 ++-- 3 files changed, 105 insertions(+), 55 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iomm= u/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 48fa8eb271a45..d07c08b53c5cf 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -51,6 +51,7 @@ arm_smmu_share_asid(struct mm_struct *mm, u16 asid) struct arm_smmu_device *smmu; struct arm_smmu_domain *smmu_domain; struct arm_smmu_master *master; + struct arm_smmu_attached_domain *attached_domain; =20 cd =3D xa_load(&arm_smmu_asid_xa, asid); if (!cd) @@ -82,11 +83,14 @@ arm_smmu_share_asid(struct mm_struct *mm, u16 asid) * be some overlap between use of both ASIDs, until we invalidate the * TLB. */ - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { - arm_smmu_write_ctx_desc(smmu, master->s1_cfg, master, 0, cd); + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, + domain_head) { + master =3D attached_domain->master; + arm_smmu_write_ctx_desc(smmu, master->s1_cfg, master, + attached_domain->ssid, cd); } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); =20 /* Invalidate TLB entries previously associated with that context */ arm_smmu_tlb_inv_asid(smmu, asid); @@ -210,7 +214,7 @@ static void arm_smmu_mm_invalidate_range(struct mmu_not= ifier *mn, if (!(smmu_domain->smmu->features & ARM_SMMU_FEAT_BTM)) arm_smmu_tlb_inv_range_asid(start, size, smmu_mn->cd->asid, PAGE_SIZE, false, smmu_domain); - arm_smmu_atc_inv_domain(smmu_domain, mm->pasid, start, size); + arm_smmu_atc_inv_domain_ssid(smmu_domain, mm->pasid, start, size); } =20 static void arm_smmu_mm_release(struct mmu_notifier *mn, struct mm_struct = *mm) @@ -218,6 +222,7 @@ static void arm_smmu_mm_release(struct mmu_notifier *mn= , struct mm_struct *mm) struct arm_smmu_mmu_notifier *smmu_mn =3D mn_to_smmu(mn); struct arm_smmu_domain *smmu_domain =3D smmu_mn->domain; struct arm_smmu_master *master; + struct arm_smmu_attached_domain *attached_domain; unsigned long flags; =20 mutex_lock(&sva_lock); @@ -230,15 +235,21 @@ static void arm_smmu_mm_release(struct mmu_notifier *= mn, struct mm_struct *mm) * DMA may still be running. Keep the cd valid to avoid C_BAD_CD events, * but disable translation. */ - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { - arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, - mm->pasid, &quiet_cd); + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, + domain_head) { + master =3D attached_domain->master; + /* + * SVA domains piggyback on the attached_domain with SSID 0. + */ + if (attached_domain->ssid =3D=3D 0) + arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, + master, mm->pasid, &quiet_cd); } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); =20 arm_smmu_tlb_inv_asid(smmu_domain->smmu, smmu_mn->cd->asid); - arm_smmu_atc_inv_domain(smmu_domain, mm->pasid, 0, 0); + arm_smmu_atc_inv_domain_ssid(smmu_domain, mm->pasid, 0, 0); =20 smmu_mn->cleared =3D true; mutex_unlock(&sva_lock); @@ -265,6 +276,7 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smmu_= domain, struct arm_smmu_ctx_desc *cd; struct arm_smmu_mmu_notifier *smmu_mn; struct arm_smmu_master *master; + struct arm_smmu_attached_domain *attached_domain; =20 list_for_each_entry(smmu_mn, &smmu_domain->mmu_notifiers, list) { if (smmu_mn->mn.mm =3D=3D mm) { @@ -294,12 +306,14 @@ arm_smmu_mmu_notifier_get(struct arm_smmu_domain *smm= u_domain, goto err_free_cd; } =20 - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, + domain_head) { + master =3D attached_domain->master; ret =3D arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, mm->pasid, cd); } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); if (ret) goto err_put_notifier; =20 @@ -319,6 +333,7 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_m= mu_notifier *smmu_mn) unsigned long flags; struct mm_struct *mm =3D smmu_mn->mn.mm; struct arm_smmu_ctx_desc *cd =3D smmu_mn->cd; + struct arm_smmu_attached_domain *attached_domain; struct arm_smmu_master *master; struct arm_smmu_domain *smmu_domain =3D smmu_mn->domain; =20 @@ -327,12 +342,14 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu= _mmu_notifier *smmu_mn) =20 list_del(&smmu_mn->list); =20 - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, + domain_head) { + master =3D attached_domain->master; arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, mm->pasid, NULL); } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); =20 /* * If we went through clear(), we've already invalidated, and no @@ -340,7 +357,7 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_m= mu_notifier *smmu_mn) */ if (!smmu_mn->cleared) { arm_smmu_tlb_inv_asid(smmu_domain->smmu, cd->asid); - arm_smmu_atc_inv_domain(smmu_domain, mm->pasid, 0, 0); + arm_smmu_atc_inv_domain_ssid(smmu_domain, mm->pasid, 0, 0); } =20 /* Frees smmu_mn */ diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index dc7a59e87a2b4..65e2dfd28b7d8 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1711,7 +1711,14 @@ static irqreturn_t arm_smmu_combined_irq_handler(int= irq, void *dev) } =20 static void -arm_smmu_atc_inv_to_cmd(int ssid, unsigned long iova, size_t size, +arm_smmu_atc_inv_cmd_set_ssid(int ssid, struct arm_smmu_cmdq_ent *cmd) +{ + cmd->substream_valid =3D !!ssid; + cmd->atc.ssid =3D ssid; +} + +static void +arm_smmu_atc_inv_to_cmd(unsigned long iova, size_t size, struct arm_smmu_cmdq_ent *cmd) { size_t log2_span; @@ -1736,8 +1743,8 @@ arm_smmu_atc_inv_to_cmd(int ssid, unsigned long iova,= size_t size, */ *cmd =3D (struct arm_smmu_cmdq_ent) { .opcode =3D CMDQ_OP_ATC_INV, - .substream_valid =3D !!ssid, - .atc.ssid =3D ssid, + .substream_valid =3D false, + .atc.ssid =3D 0, }; =20 if (!size) { @@ -1783,8 +1790,7 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_ma= ster *master) struct arm_smmu_cmdq_ent cmd; struct arm_smmu_cmdq_batch cmds; =20 - arm_smmu_atc_inv_to_cmd(0, 0, 0, &cmd); - + arm_smmu_atc_inv_to_cmd(0, 0, &cmd); cmds.num =3D 0; for (i =3D 0; i < master->num_streams; i++) { cmd.atc.sid =3D master->streams[i].id; @@ -1794,13 +1800,19 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_= master *master) return arm_smmu_cmdq_batch_submit(master->smmu, &cmds); } =20 -int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, - unsigned long iova, size_t size) +/* + * If ssid is non-zero, issue atc invalidations with the given ssid instea= d of + * the one the domain is attached to. This is used by SVA since it's pasid + * attachments aren't recorded in smmu_domain yet. + */ +int arm_smmu_atc_inv_domain_ssid(struct arm_smmu_domain *smmu_domain, int = ssid, + unsigned long iova, size_t size) { int i; unsigned long flags; struct arm_smmu_cmdq_ent cmd; struct arm_smmu_master *master; + struct arm_smmu_attached_domain *attached_domain; struct arm_smmu_cmdq_batch cmds; =20 if (!(smmu_domain->smmu->features & ARM_SMMU_FEAT_ATS)) @@ -1823,25 +1835,37 @@ int arm_smmu_atc_inv_domain(struct arm_smmu_domain = *smmu_domain, int ssid, if (!atomic_read(&smmu_domain->nr_ats_masters)) return 0; =20 - arm_smmu_atc_inv_to_cmd(ssid, iova, size, &cmd); + arm_smmu_atc_inv_to_cmd(iova, size, &cmd); =20 cmds.num =3D 0; =20 - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_for_each_entry(master, &smmu_domain->devices, domain_head) { + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, + domain_head) { + master =3D attached_domain->master; if (!master->ats_enabled) continue; + if (ssid !=3D 0) + arm_smmu_atc_inv_cmd_set_ssid(ssid, &cmd); + else + arm_smmu_atc_inv_cmd_set_ssid(attached_domain->ssid, &cmd); =20 for (i =3D 0; i < master->num_streams; i++) { cmd.atc.sid =3D master->streams[i].id; arm_smmu_cmdq_batch_add(smmu_domain->smmu, &cmds, &cmd); } } - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); =20 return arm_smmu_cmdq_batch_submit(smmu_domain->smmu, &cmds); } =20 +static int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, + unsigned long iova, size_t size) +{ + return arm_smmu_atc_inv_domain_ssid(smmu_domain, 0, iova, size); +} + /* IO_PGTABLE API */ static void arm_smmu_tlb_inv_context(void *cookie) { @@ -1863,7 +1887,7 @@ static void arm_smmu_tlb_inv_context(void *cookie) cmd.tlbi.vmid =3D smmu_domain->s2_cfg.vmid; arm_smmu_cmdq_issue_cmd_with_sync(smmu, &cmd); } - arm_smmu_atc_inv_domain(smmu_domain, 0, 0, 0); + arm_smmu_atc_inv_domain(smmu_domain, 0, 0); } =20 static void __arm_smmu_tlb_inv_range(struct arm_smmu_cmdq_ent *cmd, @@ -1951,7 +1975,7 @@ static void arm_smmu_tlb_inv_range_domain(unsigned lo= ng iova, size_t size, * Unfortunately, this can't be leaf-only since we may have * zapped an entire table. */ - arm_smmu_atc_inv_domain(smmu_domain, 0, iova, size); + arm_smmu_atc_inv_domain(smmu_domain, iova, size); } =20 void arm_smmu_tlb_inv_range_asid(unsigned long iova, size_t size, int asid, @@ -2031,8 +2055,8 @@ static struct iommu_domain *arm_smmu_domain_alloc(uns= igned type) return NULL; =20 mutex_init(&smmu_domain->init_mutex); - INIT_LIST_HEAD(&smmu_domain->devices); - spin_lock_init(&smmu_domain->devices_lock); + INIT_LIST_HEAD(&smmu_domain->attached_domains); + spin_lock_init(&smmu_domain->attached_domains_lock); INIT_LIST_HEAD(&smmu_domain->mmu_notifiers); =20 return &smmu_domain->domain; @@ -2270,12 +2294,12 @@ static bool arm_smmu_ats_supported(struct arm_smmu_= master *master) return dev_is_pci(dev) && pci_ats_supported(to_pci_dev(dev)); } =20 -static void arm_smmu_enable_ats(struct arm_smmu_master *master) +static void arm_smmu_enable_ats(struct arm_smmu_master *master, + struct arm_smmu_domain *smmu_domain) { size_t stu; struct pci_dev *pdev; struct arm_smmu_device *smmu =3D master->smmu; - struct arm_smmu_domain *smmu_domain =3D master->domain; =20 /* Don't enable ATS at the endpoint if it's not enabled in the STE */ if (!master->ats_enabled) @@ -2291,10 +2315,9 @@ static void arm_smmu_enable_ats(struct arm_smmu_mast= er *master) dev_err(master->dev, "Failed to enable ATS (STU %zu)\n", stu); } =20 -static void arm_smmu_disable_ats(struct arm_smmu_master *master) +static void arm_smmu_disable_ats(struct arm_smmu_master *master, + struct arm_smmu_domain *smmu_domain) { - struct arm_smmu_domain *smmu_domain =3D master->domain; - if (!master->ats_enabled) return; =20 @@ -2358,18 +2381,17 @@ static void arm_smmu_disable_pasid(struct arm_smmu_= master *master) static void arm_smmu_detach_dev(struct arm_smmu_master *master) { unsigned long flags; - struct arm_smmu_domain *smmu_domain =3D master->domain; + struct arm_smmu_domain *smmu_domain =3D master->non_pasid_domain.domain; =20 if (!smmu_domain) return; =20 - arm_smmu_disable_ats(master); + arm_smmu_disable_ats(master, smmu_domain); =20 - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_del(&master->domain_head); - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_del(&master->non_pasid_domain.domain_head); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); =20 - master->domain =3D NULL; master->ats_enabled =3D false; if (master->s1_cfg) arm_smmu_write_ctx_desc( @@ -2378,6 +2400,7 @@ static void arm_smmu_detach_dev(struct arm_smmu_maste= r *master) 0, NULL); master->s1_cfg =3D NULL; master->s2_cfg =3D NULL; + master->non_pasid_domain.domain =3D NULL; arm_smmu_install_ste_for_dev(master); } =20 @@ -2422,7 +2445,6 @@ static int arm_smmu_attach_dev(struct iommu_domain *d= omain, struct device *dev) goto out_unlock; } =20 - master->domain =3D smmu_domain; if (smmu_domain->stage =3D=3D ARM_SMMU_DOMAIN_S1) { master->s1_cfg =3D &master->owned_s1_cfg; ret =3D arm_smmu_write_ctx_desc( @@ -2431,7 +2453,6 @@ static int arm_smmu_attach_dev(struct iommu_domain *d= omain, struct device *dev) 0, &smmu_domain->cd); if (ret) { master->s1_cfg =3D NULL; - master->domain =3D NULL; goto out_unlock; } } else if (smmu_domain->stage =3D=3D ARM_SMMU_DOMAIN_S2 || @@ -2449,13 +2470,17 @@ static int arm_smmu_attach_dev(struct iommu_domain = *domain, struct device *dev) if (smmu_domain->stage !=3D ARM_SMMU_DOMAIN_BYPASS) master->ats_enabled =3D arm_smmu_ats_supported(master); =20 + master->non_pasid_domain.master =3D master; + master->non_pasid_domain.domain =3D smmu_domain; + master->non_pasid_domain.ssid =3D 0; arm_smmu_install_ste_for_dev(master); =20 - spin_lock_irqsave(&smmu_domain->devices_lock, flags); - list_add(&master->domain_head, &smmu_domain->devices); - spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_add(&master->non_pasid_domain.domain_head, + &smmu_domain->attached_domains); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); =20 - arm_smmu_enable_ats(master); + arm_smmu_enable_ats(master, smmu_domain); =20 out_unlock: mutex_unlock(&smmu_domain->init_mutex); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.h index dff0fa8345462..6929590530367 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -682,11 +682,19 @@ struct arm_smmu_stream { struct rb_node node; }; =20 +/* List of {masters, ssid} that a domain is attached to */ +struct arm_smmu_attached_domain { + struct list_head domain_head; + struct arm_smmu_domain *domain; + struct arm_smmu_master *master; + int ssid; +}; + /* SMMU private data for each master */ struct arm_smmu_master { struct arm_smmu_device *smmu; struct device *dev; - struct arm_smmu_domain *domain; + struct arm_smmu_attached_domain non_pasid_domain; struct list_head domain_head; struct arm_smmu_stream *streams; struct arm_smmu_s1_cfg owned_s1_cfg; @@ -724,8 +732,8 @@ struct arm_smmu_domain { =20 struct iommu_domain domain; =20 - struct list_head devices; - spinlock_t devices_lock; + struct list_head attached_domains; + spinlock_t attached_domains_lock; =20 struct list_head mmu_notifiers; }; @@ -748,8 +756,8 @@ void arm_smmu_tlb_inv_range_asid(unsigned long iova, si= ze_t size, int asid, size_t granule, bool leaf, struct arm_smmu_domain *smmu_domain); bool arm_smmu_free_asid(struct arm_smmu_ctx_desc *cd); -int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, int ssid, - unsigned long iova, size_t size); +int arm_smmu_atc_inv_domain_ssid(struct arm_smmu_domain *smmu_domain, int = ssid, + unsigned long iova, size_t size); =20 #ifdef CONFIG_ARM_SMMU_V3_SVA bool arm_smmu_sva_supported(struct arm_smmu_device *smmu); --=20 2.41.0.162.gfafddb0af9-goog From nobody Sun Feb 8 07:15:01 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ABB0FEB64D7 for ; Wed, 21 Jun 2023 06:45:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230212AbjFUGpp (ORCPT ); Wed, 21 Jun 2023 02:45:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39452 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231267AbjFUGo7 (ORCPT ); Wed, 21 Jun 2023 02:44:59 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1260B2112 for ; Tue, 20 Jun 2023 23:44:22 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-bc7267224adso10882628276.0 for ; Tue, 20 Jun 2023 23:44:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1687329861; x=1689921861; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=C8az5zJDH0ESFjwiqnExbJyBaF+tF0BTQf7Bxk21/Xo=; b=MmssWY+3YFktxIF/WA4KOY5DVa4UBDzQVE2mTxNFaVyKAMTQcPmzrUyKnRow1u5/1i 0meUrWB+7qdjGxGk/BZXzmzb1pQ53nSbLZW7KEXzmJ1wTWPeYpDaHRWw5MR6gg5fRS/t ku0AYHWl3On0+y1JE0YaBf7Aeap5dO2iKcztXzsyi5LQbiU3nBTbOOBGFhj+kVO1nPS/ L1ltlHWhVDA8Qr/1MoVmRNWP3cN11BDGjlQjTXU+uttVYyoXzTzGdapoPcBYaZqSXJL0 P6GwQQiyXHV7CzI5aqwk/8XCza5dBcBmpLoo5ytAswHLv7lm0GTilChwtaq7QlU0zcFb bo5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687329861; x=1689921861; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=C8az5zJDH0ESFjwiqnExbJyBaF+tF0BTQf7Bxk21/Xo=; b=daZGyYb/OLJiyxJkNjW3gKexX752r6/FAkASZygWU3wLmUSk4q0p6kwsxz32bZpIyH uvMvL+uP02nGF3ikYyXkTr3ghy0H+eEO/XMUh038ZY0ANC979352nGkAwhF9i3qqcO0l 6oDOaqmXkdo6/gDqTxsQh49W38c4Yco0tHaVsuJLW/5+ybl+hYipUtB+vrbZTcVjrFY3 mpIqYJYhR90heXt0k29vpsvcZPADHwIhdU/tK6Vt7nR3uKIcGdVS5HS7rCnE9vm1ytrc zCRhgHEUdpiYzF+/fG9dOb2ujAFV50xSw0tFgJjPLRoXnjnKYrRUXYQWdQeuobdDhysR 4Rnw== X-Gm-Message-State: AC+VfDzShRkFrVXtmQwJ4+FCS3Rd63bGicmUJ9PleoM+GsXXazN7hUww q83ZwgQv95DjGVBhqk7NcZdmYE1JlqT/ X-Google-Smtp-Source: ACHHUZ7mhXQYdCxoNYKF5SfDVv2FuZ4r9FDAvWLFLi6rVg6yhDNcK8dk0uoxOmNJIbocwU2FBeocvwwAg5dq X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:384f:f7da:c61d:5a3e]) (user=mshavit job=sendgmr) by 2002:a05:6902:92:b0:bc4:a660:528f with SMTP id h18-20020a056902009200b00bc4a660528fmr3348263ybs.5.1687329861096; Tue, 20 Jun 2023 23:44:21 -0700 (PDT) Date: Wed, 21 Jun 2023 14:37:20 +0800 In-Reply-To: <20230621063825.268890-1-mshavit@google.com> Mime-Version: 1.0 References: <20230621063825.268890-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230621063825.268890-9-mshavit@google.com> Subject: [PATCH v4 08/13] iommu/arm-smmu-v3: Add helper for atc invalidation From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This will be used to invalidate ATC entries made on an SSID for a master when detaching a domain with pasid. Signed-off-by: Michael Shavit --- v1->v2: Make use of arm_smmu_atc_inv_cmd_set_ssid --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index 65e2dfd28b7d8..0a5e875abda86 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1784,13 +1784,15 @@ arm_smmu_atc_inv_to_cmd(unsigned long iova, size_t = size, cmd->atc.size =3D log2_span; } =20 -static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) +static int arm_smmu_atc_inv_master_ssid(struct arm_smmu_master *master, + int ssid) { int i; struct arm_smmu_cmdq_ent cmd; struct arm_smmu_cmdq_batch cmds; =20 arm_smmu_atc_inv_to_cmd(0, 0, &cmd); + arm_smmu_atc_inv_cmd_set_ssid(ssid, &cmd); cmds.num =3D 0; for (i =3D 0; i < master->num_streams; i++) { cmd.atc.sid =3D master->streams[i].id; @@ -1800,6 +1802,11 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_m= aster *master) return arm_smmu_cmdq_batch_submit(master->smmu, &cmds); } =20 +static int arm_smmu_atc_inv_master(struct arm_smmu_master *master) +{ + return arm_smmu_atc_inv_master_ssid(master, 0); +} + /* * If ssid is non-zero, issue atc invalidations with the given ssid instea= d of * the one the domain is attached to. This is used by SVA since it's pasid --=20 2.41.0.162.gfafddb0af9-goog From nobody Sun Feb 8 07:15:01 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 686ADEB64D7 for ; Wed, 21 Jun 2023 06:46:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230512AbjFUGqD (ORCPT ); Wed, 21 Jun 2023 02:46:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40626 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230489AbjFUGpe (ORCPT ); Wed, 21 Jun 2023 02:45:34 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CECFA26A2 for ; Tue, 20 Jun 2023 23:44:30 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-bb2a7308f21so7094005276.2 for ; Tue, 20 Jun 2023 23:44:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1687329869; x=1689921869; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=yj2G77LxXK3CUewBPucFF3sEDs9aMy8klQel7QUDOOo=; b=AVFwDUMdX9bm/1NAs/fP2460ErtAUK5gKa5D2fW3RzoiQ2i5K3I13ge21qxKjeNd3a wB3JqjD4MBapxFZNlWKOZkn9Z9Ps2U2DFeRXbm2iHFMcA5NOtca3EuKojlb8xFQiTMnh t5RdMnxU8ESVO6NCh1NGlXkPJJz7WM411mxVBP6tBx+vRZqeoCiln6p7c0lfLfpsWl0e jsz1vaaObWTpYfa5mpP0xLM3YRK6mcqHMxKnRxN1QOk+1DFiRqYKb10H3sXpY73yTVRF XbVFXrzPdekEKW6Dm35Vpy78ijKrkWqacRdb80QMb/w30C1UyJObegvDjmL/TzhUiTpb Hj7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687329869; x=1689921869; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=yj2G77LxXK3CUewBPucFF3sEDs9aMy8klQel7QUDOOo=; b=mB/4F1XBHPJ8cYgXZqM7L8EVUtxG3VsWLvTR6xQxYokZl4SihEJHn/kJ1kLtK5nnK7 w4BZt7jdm6b2MSXXnQq6vCZzASBXXFY0rhBlY8CF45A4C8Drt6hlI3PJkGHkoiJVWsgZ SxENxNSbBCeHM5v8RZq+8Um/BIrUWvE9/XeXlOZwhPh5BsOdtfmtdRSChnFd9z3Vo+6A wSD54sKOnuOyh5KkO+uNWRDFCTT4yyQjUiqR5NjCUaVPqCsjE+WoH6Z1ERltPjixPa17 TPJ17JlJjUTCx5bS7a0G+8tonT0K/j2tnKN3xEe50incADUdVAG5Lg8mUHYszkd7LLyL KfUw== X-Gm-Message-State: AC+VfDzOL8qA4vSPWo8/USJ2z7Ln7s8wOOk/LlfCLgnmIzWArqMohk5X WEe6qW/Zmn0odaKXzqlFSN1PBAdTlwKf X-Google-Smtp-Source: ACHHUZ57iaPQJ78bTLZUACxkmHc2UhtZaM+a2NgbrKeErayEpPeu5gHJ+UjoM5BH6Ni/ze6h+cPt+NjdULSA X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:384f:f7da:c61d:5a3e]) (user=mshavit job=sendgmr) by 2002:a25:ce07:0:b0:bad:99d:f088 with SMTP id x7-20020a25ce07000000b00bad099df088mr5471090ybe.11.1687329869352; Tue, 20 Jun 2023 23:44:29 -0700 (PDT) Date: Wed, 21 Jun 2023 14:37:21 +0800 In-Reply-To: <20230621063825.268890-1-mshavit@google.com> Mime-Version: 1.0 References: <20230621063825.268890-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230621063825.268890-10-mshavit@google.com> Subject: [PATCH v4 09/13] iommu/arm-smmu-v3: Implement set_dev_pasid From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This change enables the use of the iommu_attach_dev_pasid API for UNMANAGED domains. The primary use-case is to allow in-kernel users of the iommu API to manage domains with PASID. This change also allows for future support of pasid in the DMA api. Signed-off-by: Michael Shavit --- v1->v2: Add missing atc invalidation when detaching with pasid --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 167 +++++++++++++++++--- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 1 + 2 files changed, 149 insertions(+), 19 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index 0a5e875abda86..b928997d35ed3 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2173,6 +2173,10 @@ static int arm_smmu_domain_finalise_s2(struct arm_sm= mu_domain *smmu_domain, return 0; } =20 +/* + * master may be null for domain types that are finalized before being att= ached + * to a master. + */ static int arm_smmu_domain_finalise(struct iommu_domain *domain, struct arm_smmu_master *master) { @@ -2369,6 +2373,11 @@ static int arm_smmu_enable_pasid(struct arm_smmu_mas= ter *master) return 0; } =20 +static bool arm_smmu_master_has_pasid_domains(struct arm_smmu_master *mast= er) +{ + return master->nr_attached_pasid_domains > 0; +} + static void arm_smmu_disable_pasid(struct arm_smmu_master *master) { struct pci_dev *pdev; @@ -2411,6 +2420,28 @@ static void arm_smmu_detach_dev(struct arm_smmu_mast= er *master) arm_smmu_install_ste_for_dev(master); } =20 +/* + * Once attached for the first time, a domain can no longer be attached to= any + * master with a distinct upstream SMMU. + */ +static int arm_smmu_prepare_domain_for_smmu(struct arm_smmu_device *smmu, + struct arm_smmu_domain *smmu_domain) +{ + int ret =3D 0; + + mutex_lock(&smmu_domain->init_mutex); + if (!smmu_domain->smmu) { + smmu_domain->smmu =3D smmu; + ret =3D arm_smmu_domain_finalise(&smmu_domain->domain, NULL); + if (ret) + smmu_domain->smmu =3D NULL; + } else if (smmu_domain->smmu !=3D smmu) { + ret =3D -EINVAL; + } + mutex_unlock(&smmu_domain->init_mutex); + return ret; +} + static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device = *dev) { int ret =3D 0; @@ -2426,6 +2457,10 @@ static int arm_smmu_attach_dev(struct iommu_domain *= domain, struct device *dev) master =3D dev_iommu_priv_get(dev); smmu =3D master->smmu; =20 + ret =3D arm_smmu_prepare_domain_for_smmu(smmu, smmu_domain); + if (ret) + return ret; + /* * Checking that SVA is disabled ensures that this device isn't bound to * any mm, and can be safely detached from its old domain. Bonds cannot @@ -2436,22 +2471,18 @@ static int arm_smmu_attach_dev(struct iommu_domain = *domain, struct device *dev) return -EBUSY; } =20 - arm_smmu_detach_dev(master); - - mutex_lock(&smmu_domain->init_mutex); - - if (!smmu_domain->smmu) { - smmu_domain->smmu =3D smmu; - ret =3D arm_smmu_domain_finalise(domain, master); - if (ret) { - smmu_domain->smmu =3D NULL; - goto out_unlock; - } - } else if (smmu_domain->smmu !=3D smmu) { - ret =3D -EINVAL; - goto out_unlock; + /* + * Attaching a bypass or stage 2 domain would break any domains attached + * with pasid. Attaching an S1 domain should be feasible but requires + * more complicated logic to handle. + */ + if (arm_smmu_master_has_pasid_domains(master)) { + dev_err(dev, "cannot attach - domain attached with pasid\n"); + return -EBUSY; } =20 + arm_smmu_detach_dev(master); + if (smmu_domain->stage =3D=3D ARM_SMMU_DOMAIN_S1) { master->s1_cfg =3D &master->owned_s1_cfg; ret =3D arm_smmu_write_ctx_desc( @@ -2460,7 +2491,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *d= omain, struct device *dev) 0, &smmu_domain->cd); if (ret) { master->s1_cfg =3D NULL; - goto out_unlock; + return ret; } } else if (smmu_domain->stage =3D=3D ARM_SMMU_DOMAIN_S2 || smmu_domain->stage =3D=3D ARM_SMMU_DOMAIN_NESTED) { @@ -2489,11 +2520,75 @@ static int arm_smmu_attach_dev(struct iommu_domain = *domain, struct device *dev) =20 arm_smmu_enable_ats(master, smmu_domain); =20 -out_unlock: - mutex_unlock(&smmu_domain->init_mutex); return ret; } =20 +static int arm_smmu_set_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ + int ret =3D 0; + unsigned long flags; + struct iommu_fwspec *fwspec =3D dev_iommu_fwspec_get(dev); + struct arm_smmu_device *smmu; + struct arm_smmu_domain *smmu_domain =3D to_smmu_domain(domain); + struct arm_smmu_attached_domain *attached_domain; + struct arm_smmu_master *master; + + if (!fwspec) + return -ENOENT; + + master =3D dev_iommu_priv_get(dev); + smmu =3D master->smmu; + + ret =3D arm_smmu_prepare_domain_for_smmu(smmu, smmu_domain); + if (ret) + return ret; + + if (pasid =3D=3D 0) { + dev_err(dev, "pasid 0 is reserved for the device's primary domain\n"); + return -ENODEV; + } + + if (smmu_domain->stage !=3D ARM_SMMU_DOMAIN_S1) { + dev_err(dev, "set_dev_pasid only supports stage 1 domains\n"); + return -EINVAL; + } + + if (!master->s1_cfg || master->s2_cfg) + return -EBUSY; + + attached_domain =3D kzalloc(sizeof(*attached_domain), GFP_KERNEL); + if (!attached_domain) + return -ENOMEM; + + attached_domain->master =3D master; + attached_domain->domain =3D smmu_domain; + attached_domain->ssid =3D pasid; + + master->nr_attached_pasid_domains +=3D 1; + /* + * arm_smmu_share_asid may update the cd's asid value and write the + * ctx_desc for every attached_domains in the list. There's a potential + * race here regardless of whether we first write the ctx_desc or + * first insert into the domain's list. Grabbing the asic_lock prevents + * SVA from changing the cd's ASID while the cd is being attached. + */ + mutex_lock(&arm_smmu_asid_lock); + ret =3D arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, + pasid, &smmu_domain->cd); + if (ret) { + mutex_unlock(&arm_smmu_asid_lock); + kfree(attached_domain); + } + + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_add(&attached_domain->domain_head, &smmu_domain->attached_domains); + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); + mutex_unlock(&arm_smmu_asid_lock); + + return 0; +} + static int arm_smmu_map_pages(struct iommu_domain *domain, unsigned long i= ova, phys_addr_t paddr, size_t pgsize, size_t pgcount, int prot, gfp_t gfp, size_t *mapped) @@ -2739,6 +2834,15 @@ static void arm_smmu_release_device(struct device *d= ev) =20 if (WARN_ON(arm_smmu_master_sva_enabled(master))) iopf_queue_remove_device(master->smmu->evtq.iopf, dev); + if (WARN_ON(master->nr_attached_pasid_domains !=3D 0)) { + /* + * TODO: Do we need to handle this case? + * This requires a mechanism to obtain all the pasid domains + * that this master is attached to so that we can clean up the + * domain's attached_domain list. + */ + } + arm_smmu_detach_dev(master); arm_smmu_free_cd_tables(master->smmu, &master->owned_s1_cfg.cdcfg); arm_smmu_disable_pasid(master); @@ -2874,12 +2978,36 @@ static int arm_smmu_def_domain_type(struct device *= dev) static void arm_smmu_remove_dev_pasid(struct device *dev, ioasid_t pasid) { struct iommu_domain *domain; + struct arm_smmu_master *master =3D dev_iommu_priv_get(dev); + struct arm_smmu_domain *smmu_domain; + struct arm_smmu_attached_domain *attached_domain; + unsigned long flags; =20 - domain =3D iommu_get_domain_for_dev_pasid(dev, pasid, IOMMU_DOMAIN_SVA); + if (!master || pasid =3D=3D 0) + return; + + domain =3D iommu_get_domain_for_dev_pasid(dev, pasid, 0); if (WARN_ON(IS_ERR(domain)) || !domain) return; + if (domain->type =3D=3D IOMMU_DOMAIN_SVA) + return arm_smmu_sva_remove_dev_pasid(domain, dev, pasid); =20 - arm_smmu_sva_remove_dev_pasid(domain, dev, pasid); + smmu_domain =3D to_smmu_domain(domain); + mutex_lock(&arm_smmu_asid_lock); + spin_lock_irqsave(&smmu_domain->attached_domains_lock, flags); + list_for_each_entry(attached_domain, &smmu_domain->attached_domains, doma= in_head) { + if (attached_domain->master !=3D master || + attached_domain->ssid !=3D pasid) + continue; + list_del(&attached_domain->domain_head); + break; + } + spin_unlock_irqrestore(&smmu_domain->attached_domains_lock, flags); + arm_smmu_write_ctx_desc(master->smmu, master->s1_cfg, master, pasid, + NULL); + arm_smmu_atc_inv_master_ssid(master, pasid); + master->nr_attached_pasid_domains -=3D 1; + mutex_unlock(&arm_smmu_asid_lock); } =20 static struct iommu_ops arm_smmu_ops =3D { @@ -2899,6 +3027,7 @@ static struct iommu_ops arm_smmu_ops =3D { .owner =3D THIS_MODULE, .default_domain_ops =3D &(const struct iommu_domain_ops) { .attach_dev =3D arm_smmu_attach_dev, + .set_dev_pasid =3D arm_smmu_set_dev_pasid, .map_pages =3D arm_smmu_map_pages, .unmap_pages =3D arm_smmu_unmap_pages, .flush_iotlb_all =3D arm_smmu_flush_iotlb_all, diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.h index 6929590530367..48795a7287b69 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -707,6 +707,7 @@ struct arm_smmu_master { bool iopf_enabled; struct list_head bonds; unsigned int ssid_bits; + unsigned int nr_attached_pasid_domains; }; =20 /* SMMU private data for an IOMMU domain */ --=20 2.41.0.162.gfafddb0af9-goog From nobody Sun Feb 8 07:15:01 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25C52EB64DC for ; Wed, 21 Jun 2023 06:46:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231201AbjFUGqZ (ORCPT ); Wed, 21 Jun 2023 02:46:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40362 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231164AbjFUGpu (ORCPT ); Wed, 21 Jun 2023 02:45:50 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EF1681BD0 for ; Tue, 20 Jun 2023 23:44:39 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-bfec07e5eb0so736401276.2 for ; Tue, 20 Jun 2023 23:44:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1687329877; x=1689921877; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=tvomj7iW2UACU6/yECNbFWEXmb4EH1aV19zctsWiNw0=; b=bFIfZORz4qlyvHdesSBGaoACwrXyukUmDFr+sNMMwiGcbhJJOqX5vCINTCGVK/kqb1 M2pwDMPHBCg5EsNV8co3V/w8l6vhX4vTin9L9Y5GtkOrDe1gld2dyDd8Gs1HqLfNwdN1 G1FtBdPsy2NHnG2IC3hIdVsnPTWng2eGQ+qtojfzCdm1ORIt9R40o+eakI5Ec4ojnwqT F2VattLO/4nUSbZD2O/jFgdRuiN0pvkwRjTfp2vimZsBjB7peBN0fhLflu4T05WVZiO1 hJkx9W1oX0Pcb7kL4Ccd0fi2GEjiJaL7AxA3aaF5kFJsUz+3Ce+yO5GZBwoTHZZ/M+Ot GroQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687329877; x=1689921877; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=tvomj7iW2UACU6/yECNbFWEXmb4EH1aV19zctsWiNw0=; b=fwba4Jk3WNHBFx8nCdnOkJ1roBJuHKqxT/UBVVwH7ANOVfaAiPImnTVorp/4rNN+Cy u3wMvNWOp9qJoCMTLZ8mfAh/xwLHLhbDeNZ4Tdx4RvwvWTvrDW8qyXLqw6tNnCTTVkD9 yY0CsB6BngJbfsYTGlGW8VQBcbdbVnEAFZL/73cM+4CK4Gz598gO0rupEp3cIPNjv1JJ 89jE8sBrljRUIcnpfk2ceab4E/elJiDTiWmef89JBhY8yVt8yGOLeYYGExxSqTY1m+9q Gb2wXcmfl9H+t02fexZtEdr5fCHbE6HnpMyCs6B8OIA1DIcHX2LZYLu3o7XwkSvH46PK q/Yw== X-Gm-Message-State: AC+VfDzLV7Sx2H3enPRzQISEDtf0OJVO/RagjNrp7X3mRccC6MIRW2lt a1rulwyBFRqO6NReiDsFy0nlrRwAFePA X-Google-Smtp-Source: ACHHUZ4Hv0OtvqUjdHD4vH+y719Y6q1CiVIktPp2laFj3DNmpEvjIkRNUnFgjTd048rLwQnXunTIxIImH8Cz X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:384f:f7da:c61d:5a3e]) (user=mshavit job=sendgmr) by 2002:a25:d658:0:b0:bbb:8c13:ce26 with SMTP id n85-20020a25d658000000b00bbb8c13ce26mr5885974ybg.11.1687329877614; Tue, 20 Jun 2023 23:44:37 -0700 (PDT) Date: Wed, 21 Jun 2023 14:37:22 +0800 In-Reply-To: <20230621063825.268890-1-mshavit@google.com> Mime-Version: 1.0 References: <20230621063825.268890-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230621063825.268890-11-mshavit@google.com> Subject: [PATCH v4 10/13] iommu/arm-smmu-v3-sva: Remove bond refcount From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The iommu-sva framework checks if a bond between a device and mm already exists and handles refcounting at the iommu_domain level. __arm_smmu_sva_bind is therefore only called once for a device/mm pair. Signed-off-by: Michael Shavit --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 12 +----------- 1 file changed, 1 insertion(+), 11 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iomm= u/arm/arm-smmu-v3/arm-smmu-v3-sva.c index d07c08b53c5cf..20301d0a2c0b0 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -29,7 +29,6 @@ struct arm_smmu_bond { struct mm_struct *mm; struct arm_smmu_mmu_notifier *smmu_mn; struct list_head list; - refcount_t refs; }; =20 #define sva_to_bond(handle) \ @@ -377,21 +376,12 @@ __arm_smmu_sva_bind(struct device *dev, struct mm_str= uct *mm) if (!master || !master->sva_enabled) return ERR_PTR(-ENODEV); =20 - /* If bind() was already called for this {dev, mm} pair, reuse it. */ - list_for_each_entry(bond, &master->bonds, list) { - if (bond->mm =3D=3D mm) { - refcount_inc(&bond->refs); - return &bond->sva; - } - } - bond =3D kzalloc(sizeof(*bond), GFP_KERNEL); if (!bond) return ERR_PTR(-ENOMEM); =20 bond->mm =3D mm; bond->sva.dev =3D dev; - refcount_set(&bond->refs, 1); =20 bond->smmu_mn =3D arm_smmu_mmu_notifier_get(smmu_domain, mm); if (IS_ERR(bond->smmu_mn)) { @@ -570,7 +560,7 @@ void arm_smmu_sva_remove_dev_pasid(struct iommu_domain = *domain, } } =20 - if (!WARN_ON(!bond) && refcount_dec_and_test(&bond->refs)) { + if (!WARN_ON(!bond)) { list_del(&bond->list); arm_smmu_mmu_notifier_put(bond->smmu_mn); kfree(bond); --=20 2.41.0.162.gfafddb0af9-goog From nobody Sun Feb 8 07:15:01 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9BCEDEB64D7 for ; Wed, 21 Jun 2023 06:46:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231190AbjFUGqq (ORCPT ); Wed, 21 Jun 2023 02:46:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39676 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229709AbjFUGp6 (ORCPT ); Wed, 21 Jun 2023 02:45:58 -0400 Received: from mail-yw1-x114a.google.com (mail-yw1-x114a.google.com [IPv6:2607:f8b0:4864:20::114a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A11821BEF for ; Tue, 20 Jun 2023 23:44:49 -0700 (PDT) Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-57000a7480bso77142067b3.0 for ; Tue, 20 Jun 2023 23:44:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1687329886; x=1689921886; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=1HNFofEgftQ/0viD8hP2C9StxNG8tEEj8VIgof4alao=; b=NwyrXz6ubgbYeu5se4OnmTGRl/ECZnN/7JXUGfb3yfJHIleuEhl/vBOWZdfzMER6Gx BiPVvWA9Zp+4QaWrnn9OAzgPxD8EtCxqeits8Wu66pkUEHOGaiwDQhRW/J+yNdDMsAlF y8tREQZuazgzoLW1lJOdMAgC3jROf3yU6f6/ooGUM/ed0cdMK69TTIuHL8pi8+hdgxg4 sMmuWLwUHHqnGxYMofKOeij/4hhaVO7yrHVtyCzzPIP3fz7HtdUP47sUq7HkSrrLt0u4 AKIy0MweTpJJB47zOUsszP8BDsLy6b518MCCeqGT4lRkc+iJWu0D/oSX+S9wtNgHByFL IhfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687329886; x=1689921886; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=1HNFofEgftQ/0viD8hP2C9StxNG8tEEj8VIgof4alao=; b=SKzXZyh35Cr1YMOXR+EXdWO+xgDMmPS4D+8LsxokkAy6uZucnRKFvds54lw/JQJO6U ojMWq/I3jaBbY+VjcboINOYJIHGyGXr9tJ46fPamF7MB30q2MjM0PmSG8ooBkVUI3ZQM YmWIsU20MqR13EUuszf8hj2aPxaqwl3pq5JNcezMuYfxRzrmDmicTx2WVOf0+CsbfDdj iBLlT26z0ChNQuTOoCZr/VaKzkMhQb5OT0KtmhIHXAAK9hMG2q8LFIgmRXsGWWpVJJLO T5O0kc4XzdY3oYKVnaJHDEnREI5tcusSIvwWfJQgN/dwJfTVJZFngMEDM4qT13zPgn0u iwBg== X-Gm-Message-State: AC+VfDwsf8GxCU26TY7vyDuOscecljKee145KOosNz4qjy8AdZKYRfun eYNJdxHh51kY9GyiNpaVBh9E4/e5ZnL/ X-Google-Smtp-Source: ACHHUZ7QGJG0D7z7/43AEtJ6RwcS6kiSb09I0mWwsorRzmgb/ZEaTmvkUNbdIb93dJYPVoc71NOppCwfYFjW X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:384f:f7da:c61d:5a3e]) (user=mshavit job=sendgmr) by 2002:a81:bc4d:0:b0:56f:2841:aa24 with SMTP id b13-20020a81bc4d000000b0056f2841aa24mr6215155ywl.10.1687329885894; Tue, 20 Jun 2023 23:44:45 -0700 (PDT) Date: Wed, 21 Jun 2023 14:37:23 +0800 In-Reply-To: <20230621063825.268890-1-mshavit@google.com> Mime-Version: 1.0 References: <20230621063825.268890-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230621063825.268890-12-mshavit@google.com> Subject: [PATCH v4 11/13] iommu/arm-smmu-v3-sva: Clean unused iommu_sva From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The __arm_smmu_sva_bind function returned an unused iommu_sva handle that can be removed. Signed-off-by: Michael Shavit --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 18 ++++++------------ 1 file changed, 6 insertions(+), 12 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iomm= u/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 20301d0a2c0b0..650c9c9ad52f1 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -25,7 +25,6 @@ struct arm_smmu_mmu_notifier { #define mn_to_smmu(mn) container_of(mn, struct arm_smmu_mmu_notifier, mn) =20 struct arm_smmu_bond { - struct iommu_sva sva; struct mm_struct *mm; struct arm_smmu_mmu_notifier *smmu_mn; struct list_head list; @@ -364,8 +363,7 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu_m= mu_notifier *smmu_mn) arm_smmu_free_shared_cd(cd); } =20 -static struct iommu_sva * -__arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) +static int __arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) { int ret; struct arm_smmu_bond *bond; @@ -374,14 +372,13 @@ __arm_smmu_sva_bind(struct device *dev, struct mm_str= uct *mm) struct arm_smmu_domain *smmu_domain =3D to_smmu_domain(domain); =20 if (!master || !master->sva_enabled) - return ERR_PTR(-ENODEV); + return -ENODEV; =20 bond =3D kzalloc(sizeof(*bond), GFP_KERNEL); if (!bond) - return ERR_PTR(-ENOMEM); + return -ENOMEM; =20 bond->mm =3D mm; - bond->sva.dev =3D dev; =20 bond->smmu_mn =3D arm_smmu_mmu_notifier_get(smmu_domain, mm); if (IS_ERR(bond->smmu_mn)) { @@ -390,11 +387,11 @@ __arm_smmu_sva_bind(struct device *dev, struct mm_str= uct *mm) } =20 list_add(&bond->list, &master->bonds); - return &bond->sva; + return 0; =20 err_free_bond: kfree(bond); - return ERR_PTR(ret); + return ret; } =20 bool arm_smmu_sva_supported(struct arm_smmu_device *smmu) @@ -572,13 +569,10 @@ static int arm_smmu_sva_set_dev_pasid(struct iommu_do= main *domain, struct device *dev, ioasid_t id) { int ret =3D 0; - struct iommu_sva *handle; struct mm_struct *mm =3D domain->mm; =20 mutex_lock(&sva_lock); - handle =3D __arm_smmu_sva_bind(dev, mm); - if (IS_ERR(handle)) - ret =3D PTR_ERR(handle); + ret =3D __arm_smmu_sva_bind(dev, mm); mutex_unlock(&sva_lock); =20 return ret; --=20 2.41.0.162.gfafddb0af9-goog From nobody Sun Feb 8 07:15:01 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 820C2EB64D8 for ; Wed, 21 Jun 2023 06:46:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231156AbjFUGq5 (ORCPT ); Wed, 21 Jun 2023 02:46:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40014 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231145AbjFUGqS (ORCPT ); Wed, 21 Jun 2023 02:46:18 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 728401992 for ; Tue, 20 Jun 2023 23:45:00 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-bce829aca1bso6009205276.3 for ; Tue, 20 Jun 2023 23:45:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1687329893; x=1689921893; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Iaiq8BTRsoloLG7rldnwk3G52sAWylq1kN+2LzQv5tM=; b=f2p4RRe6xTN7igEKKPHZGsDF92MIFONOcBrriUfaGfiU3ABD3qhOoJJOHan/Szh8sJ Kph4EWS3o0wtAioESPtVkWwEA9xFadSpa4206f5f3djBGbzfWOB1yGO3McRAFLPpCABM fvhSEzmduefkQzfFvvyoq4Ij4xlBU9yXRK8Lkbeizarjd2Ow4YKI2mXssopkMc4dGBkD uAd3/uApx4hTbP0KqhvPL1hTILkWXfRdIqXtfMtN7PIPEbOyvMIFFJeq1AN2pA1kVws8 6R0dGFwCrLR2eZbdJbjNe6bcjYYwXVRTcsXyD48CL4OOgvxLnLwqOdLIqmLyHh29irfJ cesA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687329894; x=1689921894; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Iaiq8BTRsoloLG7rldnwk3G52sAWylq1kN+2LzQv5tM=; b=LjkyYlGoa17PLbGE6pIDhuH0GgDfJzIzBP/S25H6MG/AsPb7BvpWIAZ6TYAkZ2RejR ico26b8FfVIrhtnEpo3ckvlvyUtjLa7hhaFXsJoxDKO7v8TBq8iTR10ih5VeQCiJOXyz NdZFxO3jpwSMq1tNwWNJavmH5UGdYOyISCWcmRVcpGQ6T+kZNBjGzmw9nKhqEPPo6j1j JC7T3UCj8MFtG0J4XUQhi5Bz5Ci12zU2ntnJjLkF7xgYhiB4hjq2Fe6JD6IH5P1cYyE7 j31nTvFZD7xoobdi2Uw9hNLIdGaBqiARAXU1+PyEb8SpUagNNk8yISHaQ8rTWMBT+ubw Ijzg== X-Gm-Message-State: AC+VfDyMkCeqpcZ1eWNjfBJZ2t78tKAoPzbSszAH6wl18VOEc01lDDrD 0ibfrkT7oybOLFmyWbzDwTj+ITLdJjhj X-Google-Smtp-Source: ACHHUZ6DIszFBg7Jz5/qr1W0AVeQNCApk6Onqbv0M7i291vX3d/TmnbsUBoL2GYXPlET6GLZPqDom3YGK+aI X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:384f:f7da:c61d:5a3e]) (user=mshavit job=sendgmr) by 2002:a25:882:0:b0:bfe:ea69:91b4 with SMTP id 124-20020a250882000000b00bfeea6991b4mr167148ybi.4.1687329893822; Tue, 20 Jun 2023 23:44:53 -0700 (PDT) Date: Wed, 21 Jun 2023 14:37:24 +0800 In-Reply-To: <20230621063825.268890-1-mshavit@google.com> Mime-Version: 1.0 References: <20230621063825.268890-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230621063825.268890-13-mshavit@google.com> Subject: [PATCH v4 12/13] iommu/arm-smmu-v3-sva: Remove arm_smmu_bond From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" There's a 1:1 relationship between arm_smmu_bond and the iommu_domain used in set_dev_pasid/remove_dev_pasid. arm_smmu_bond has become an unnecessary complication. It's more natural to store any needed information at the iommu_domain container level. Signed-off-by: Michael Shavit --- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 69 +++++++------------ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 1 - drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 2 +- 3 files changed, 24 insertions(+), 48 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iomm= u/arm/arm-smmu-v3/arm-smmu-v3-sva.c index 650c9c9ad52f1..b615a85e6a54e 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -24,14 +24,13 @@ struct arm_smmu_mmu_notifier { =20 #define mn_to_smmu(mn) container_of(mn, struct arm_smmu_mmu_notifier, mn) =20 -struct arm_smmu_bond { - struct mm_struct *mm; +struct arm_smmu_sva_domain { + struct iommu_domain iommu_domain; struct arm_smmu_mmu_notifier *smmu_mn; - struct list_head list; }; =20 -#define sva_to_bond(handle) \ - container_of(handle, struct arm_smmu_bond, sva) +#define to_sva_domain(domain) \ + container_of(domain, struct arm_smmu_sva_domain, iommu_domain) =20 static DEFINE_MUTEX(sva_lock); =20 @@ -363,10 +362,10 @@ static void arm_smmu_mmu_notifier_put(struct arm_smmu= _mmu_notifier *smmu_mn) arm_smmu_free_shared_cd(cd); } =20 -static int __arm_smmu_sva_bind(struct device *dev, struct mm_struct *mm) +static int __arm_smmu_sva_bind(struct device *dev, + struct arm_smmu_sva_domain *sva_domain, + struct mm_struct *mm) { - int ret; - struct arm_smmu_bond *bond; struct arm_smmu_master *master =3D dev_iommu_priv_get(dev); struct iommu_domain *domain =3D iommu_get_domain_for_dev(dev); struct arm_smmu_domain *smmu_domain =3D to_smmu_domain(domain); @@ -374,24 +373,14 @@ static int __arm_smmu_sva_bind(struct device *dev, st= ruct mm_struct *mm) if (!master || !master->sva_enabled) return -ENODEV; =20 - bond =3D kzalloc(sizeof(*bond), GFP_KERNEL); - if (!bond) - return -ENOMEM; - - bond->mm =3D mm; - - bond->smmu_mn =3D arm_smmu_mmu_notifier_get(smmu_domain, mm); - if (IS_ERR(bond->smmu_mn)) { - ret =3D PTR_ERR(bond->smmu_mn); - goto err_free_bond; + sva_domain->smmu_mn =3D arm_smmu_mmu_notifier_get(smmu_domain, + mm); + if (IS_ERR(sva_domain->smmu_mn)) { + sva_domain->smmu_mn =3D NULL; + return PTR_ERR(sva_domain->smmu_mn); } - - list_add(&bond->list, &master->bonds); + master->nr_attached_sva_domains +=3D 1; return 0; - -err_free_bond: - kfree(bond); - return ret; } =20 bool arm_smmu_sva_supported(struct arm_smmu_device *smmu) @@ -521,7 +510,7 @@ int arm_smmu_master_enable_sva(struct arm_smmu_master *= master) int arm_smmu_master_disable_sva(struct arm_smmu_master *master) { mutex_lock(&sva_lock); - if (!list_empty(&master->bonds)) { + if (master->nr_attached_sva_domains !=3D 0) { dev_err(master->dev, "cannot disable SVA, device is bound\n"); mutex_unlock(&sva_lock); return -EBUSY; @@ -545,23 +534,12 @@ void arm_smmu_sva_notifier_synchronize(void) void arm_smmu_sva_remove_dev_pasid(struct iommu_domain *domain, struct device *dev, ioasid_t id) { - struct mm_struct *mm =3D domain->mm; - struct arm_smmu_bond *bond =3D NULL, *t; + struct arm_smmu_sva_domain *sva_domain =3D to_sva_domain(domain); struct arm_smmu_master *master =3D dev_iommu_priv_get(dev); =20 mutex_lock(&sva_lock); - list_for_each_entry(t, &master->bonds, list) { - if (t->mm =3D=3D mm) { - bond =3D t; - break; - } - } - - if (!WARN_ON(!bond)) { - list_del(&bond->list); - arm_smmu_mmu_notifier_put(bond->smmu_mn); - kfree(bond); - } + master->nr_attached_sva_domains -=3D 1; + arm_smmu_mmu_notifier_put(sva_domain->smmu_mn); mutex_unlock(&sva_lock); } =20 @@ -572,7 +550,7 @@ static int arm_smmu_sva_set_dev_pasid(struct iommu_doma= in *domain, struct mm_struct *mm =3D domain->mm; =20 mutex_lock(&sva_lock); - ret =3D __arm_smmu_sva_bind(dev, mm); + ret =3D __arm_smmu_sva_bind(dev, to_sva_domain(domain), mm); mutex_unlock(&sva_lock); =20 return ret; @@ -590,12 +568,11 @@ static const struct iommu_domain_ops arm_smmu_sva_dom= ain_ops =3D { =20 struct iommu_domain *arm_smmu_sva_domain_alloc(void) { - struct iommu_domain *domain; + struct arm_smmu_sva_domain *sva_domain; =20 - domain =3D kzalloc(sizeof(*domain), GFP_KERNEL); - if (!domain) + sva_domain =3D kzalloc(sizeof(*sva_domain), GFP_KERNEL); + if (!sva_domain) return NULL; - domain->ops =3D &arm_smmu_sva_domain_ops; - - return domain; + sva_domain->iommu_domain.ops =3D &arm_smmu_sva_domain_ops; + return &sva_domain->iommu_domain; } diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.c index b928997d35ed3..2ed6c9e0df241 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2784,7 +2784,6 @@ static struct iommu_device *arm_smmu_probe_device(str= uct device *dev) =20 master->dev =3D dev; master->smmu =3D smmu; - INIT_LIST_HEAD(&master->bonds); dev_iommu_priv_set(dev, master); =20 ret =3D arm_smmu_insert_master(smmu, master); diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/ar= m/arm-smmu-v3/arm-smmu-v3.h index 48795a7287b69..3525d60668c23 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -705,7 +705,7 @@ struct arm_smmu_master { bool stall_enabled; bool sva_enabled; bool iopf_enabled; - struct list_head bonds; + unsigned int nr_attached_sva_domains; unsigned int ssid_bits; unsigned int nr_attached_pasid_domains; }; --=20 2.41.0.162.gfafddb0af9-goog From nobody Sun Feb 8 07:15:01 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C39D0EB64D8 for ; Wed, 21 Jun 2023 06:47:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229897AbjFUGrE (ORCPT ); Wed, 21 Jun 2023 02:47:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40338 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230170AbjFUGqT (ORCPT ); Wed, 21 Jun 2023 02:46:19 -0400 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [IPv6:2607:f8b0:4864:20::1049]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5DE631FC4 for ; Tue, 20 Jun 2023 23:45:02 -0700 (PDT) Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-25e89d51bf2so2581661a91.0 for ; Tue, 20 Jun 2023 23:45:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1687329902; x=1689921902; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=GFfBSOUvUPqFSx6n4vEtFQq3Hc8Ha+kDvCzi3+3mgos=; b=t4HhyrUZx343POQssx8xWB3bNqEwvVz5axTthBdUwUeHP98YwM0/jOGBkI8RNJ6CfV q813TeRnvgUNnvQ5yy+drxSkGdqSae/ZdfJL2CLBJbfVgemKrHUzyxNImNn9WE4R5pN1 QgALlayI2ddPtiW6CPlWgIpz9CmCsE/e1EJarXWy6qGQnc+hIy5agSCGPnJEUO2/Lcx0 w31wnEnCEJrXmNRkg29aSfwTjpHDwxP6w/ngMobxEgWSJDGOmFnCq3Hf+OQierGh/m/V vjKc3DIyPkMR9sdzlVpuqyKBLwgRtHl253JAEGraYWPxylZDrHFJArJHYf6fcfRkfMfq s4QQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687329902; x=1689921902; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=GFfBSOUvUPqFSx6n4vEtFQq3Hc8Ha+kDvCzi3+3mgos=; b=GCKtJzarPQ3nonHalz0+J0b9efmwtlITGzAt09JU1Ja4BmKwmM/Lcp4uaR7vPIhr2N e7k9YOrYY2Tdpu8k7ICM4X35APA+BUrCP6CKgxHxTJAHG4qWcSHQrIdy8WQKnqKqQmg4 TUoCLvGNNJTlniK/9KqXNTAgFozaR1L+zazEwswGq9f58g4GGr6U35Z5XxshsGTdzyMH akh2NmdnnyJaNmM25M6E9eLiI4QjUgP7OE6ypAxmNkodg8aTk99Kbt3V/2EVUbfKD9Rd b5nCsS02k56VZ3K79ynJhbiWnnXjV8FcW5sLIpD4PRYY2A30853rYUn2cIBUnkDGReS5 pwfQ== X-Gm-Message-State: AC+VfDwAr2r3LDkDU/dvcl+GmPHNtFbzcSpO6xebca/HobIPsO8uf8Gq R/xQ8juK0SKheN9LTuOMkNtR84lHZSMq X-Google-Smtp-Source: ACHHUZ4sre6FKQd6JetGrp6VVOp9i7yh8bqwdCg0x/qqo75bj1C1/wd2ryfmXfIMYXHnMZ1hGUQI3eC1XGFb X-Received: from mshavit.ntc.corp.google.com ([2401:fa00:95:20c:384f:f7da:c61d:5a3e]) (user=mshavit job=sendgmr) by 2002:a17:90a:fa06:b0:25b:d689:72a4 with SMTP id cm6-20020a17090afa0600b0025bd68972a4mr1992026pjb.4.1687329901714; Tue, 20 Jun 2023 23:45:01 -0700 (PDT) Date: Wed, 21 Jun 2023 14:37:25 +0800 In-Reply-To: <20230621063825.268890-1-mshavit@google.com> Mime-Version: 1.0 References: <20230621063825.268890-1-mshavit@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230621063825.268890-14-mshavit@google.com> Subject: [PATCH v4 13/13] iommu/arm-smmu-v3-sva: Add check when enabling sva From: Michael Shavit To: Will Deacon , Robin Murphy , Joerg Roedel Cc: Michael Shavit , jean-philippe@linaro.org, nicolinc@nvidia.com, jgg@nvidia.com, baolu.lu@linux.intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" SVA domains can only be attached when the master's STEs have a stage 1 domain. Signed-off-by: Michael Shavit --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iomm= u/arm/arm-smmu-v3/arm-smmu-v3-sva.c index b615a85e6a54e..e2a91f20f0906 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c @@ -499,9 +499,15 @@ int arm_smmu_master_enable_sva(struct arm_smmu_master = *master) int ret; =20 mutex_lock(&sva_lock); + + if (!master->s1_cfg) { + ret =3D -EBUSY; + goto unlock; + } ret =3D arm_smmu_master_sva_enable_iopf(master); if (!ret) master->sva_enabled =3D true; +unlock: mutex_unlock(&sva_lock); =20 return ret; --=20 2.41.0.162.gfafddb0af9-goog