From nobody Mon Jun 8 06:36:46 2026 Received: from mail-dl1-f73.google.com (mail-dl1-f73.google.com [74.125.82.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3ED44405840 for ; Fri, 5 Jun 2026 18:59:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.82.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780685981; cv=none; b=b67yoEjG+721PdK9Jsj6YBdfD9m38yvA0HoQVqfQSYJpRluZHnBLoCNqQn5hlCqOtZCWZErebc24rugxojz9d2bmSP9QrytlDo/BfQ8RWB4bp/UsGxzV6xCOsW7PNtP9/2GlY+sfhCkVm8+mS1yi6r4RDWJLOAK2J3sq0V0cLgw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780685981; c=relaxed/simple; bh=8sP1mFvvrrrYgljNGbpHcBy0tVKVMx/WfQdTGt5Mjiw=; h=Date:Mime-Version:Message-ID:Subject:From:To:Cc:Content-Type; b=E4KdNZnorKB1XzwrPzQ89rVHp2DyTSjzxYve1r10ZzoVuRpE770+OX0aPbegiJdSh1uh8mU0fYJ2V4dsLvjI3GqBelHDmkcSfAmqVzUxin1Ph79IUCXj3yErKV2059DQ3/rY9AxKOURi5Kv/1ngU12mruE8KqqQeRyx1zQK+QWQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--loganodell.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=uQb10v2X; arc=none smtp.client-ip=74.125.82.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--loganodell.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="uQb10v2X" Received: by mail-dl1-f73.google.com with SMTP id a92af1059eb24-137ea73393cso4486252c88.0 for ; Fri, 05 Jun 2026 11:59:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1780685971; x=1781290771; darn=vger.kernel.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=E2G8CRf+JIwMyLPHamjVNAYiccLdKeAW/w/Ijs0l4Ss=; b=uQb10v2XuVNOovf0lVAwYhjtWDOLVATKQ/W94J7XGg4I6TGKrsGLvVsE+3fc39vaqQ IUQEGN8dU7kl4spoWekJBmpWQVXsvGUsuAtJvfcjXDMCj5Rch/iiMwrZoQdVyTto76W8 7Qt73NeN//27+cSLbaTS22YsNUWNKps75B8ku2hW51+dmBP8zHRowPYG/Tg2yAIZG4kj g1RS89djPKM/oVcOJuleyRNo6Uqf9sAb1uBTFW5fA3wyjsAEVb0Rd7DWNZpfToyVl0Fz hPZfCykn/xRy1ob0aD44hyJKDMOjKXnwuHSDeG9bnCRSjTEts4ffqvIrNboJH+K/YPoB Eytw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780685971; x=1781290771; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=E2G8CRf+JIwMyLPHamjVNAYiccLdKeAW/w/Ijs0l4Ss=; b=Vp92lZpGnA9joR52Rc55MLX0s+nZlhMx3HaHTuqjrTxe89atAQh0GaAYCtHYO+Hz6p 1PRGCR78pWamuQaWONyNvqgVhV1tC50YqRrJ7LwY9c8qLbiTRHsvHaRT6NgzNSy6NT3v Nv19CQwNYUPEb1ApQ2i+Woa/1tTV4tEBbu0AEUS4G8j5ucqEhwk2hazJuaMdYHni/cWs 1gx2ZRazJXm6Ct+gNu1aY/x7nWh+7b4l4inDSkqzGffWC68WiyTIJjcFQOhTCoKlTON1 HXo8Nc7ZeZXraq8CjpwBPwuJBgn4CQ+EhpYdlsWu9R7GIRyVlUuqQCefUxF//rojdp0f exwQ== X-Forwarded-Encrypted: i=1; AFNElJ/lSjPFHUPyurALlPplpxckzjH7q4/MlPVzuqz/bRpBQX2yTCo55OaABxiHPaZZk327xflch7jECM+HYX0=@vger.kernel.org X-Gm-Message-State: AOJu0YyoZRA2ydYPOaiue5QE3VpHx9K2W3FF3K8UU+dYt7k3HuuGQ9D8 LqyKlAJrudn1zVjusNJ1n8nu49tN790w+71HTBChuSQkt+qRPkt9dEBzi7MtehYYFKOVnIOtvam +ppnF9/jiTY0Dzps0NQagWA== X-Received: from dlbsw11.prod.google.com ([2002:a05:7022:3a8b:b0:138:700:1c7c]) (user=loganodell job=prod-delivery.src-stubby-dispatcher) by 2002:a05:7022:6b99:b0:137:e532:f53f with SMTP id a92af1059eb24-1380672bf32mr2439937c88.34.1780685970807; Fri, 05 Jun 2026 11:59:30 -0700 (PDT) Date: Fri, 5 Jun 2026 11:59:26 -0700 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 X-Mailer: git-send-email 2.54.0.1064.gd145956f57-goog Message-ID: <20260605185927.917517-1-loganodell@google.com> Subject: [PATCH] iommu/iova: Move CPU magazine init to first insert From: Logan Odell To: robin.murphy@arm.com Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, mclapinski@google.com, skhawaja@google.com, Logan Odell Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" A large amount of memory may be allocated for these magazines on machines with a lot of IOMMU groups and CPU cores. Not all may be used as some devices may be unused or be bound to drivers that do not use the DMA-API. Furthermore, some drivers may not use all levels or CPUs. Move the initialization of the loaded and prev magazines for each CPU on the first attempt to try to insert a freed IOVA to them. Signed-off-by: Logan Odell --- drivers/iommu/iova.c | 27 ++++++++++++++++++--------- 1 file changed, 18 insertions(+), 9 deletions(-) diff --git a/drivers/iommu/iova.c b/drivers/iommu/iova.c index 021daf6528de..1a4bbf45dbb9 100644 --- a/drivers/iommu/iova.c +++ b/drivers/iommu/iova.c @@ -621,6 +621,9 @@ iova_magazine_free_pfns(struct iova_magazine *mag, stru= ct iova_domain *iovad) unsigned long flags; int i; =20 + if (!mag) + return; + spin_lock_irqsave(&iovad->iova_rbtree_lock, flags); =20 for (i =3D 0 ; i < mag->size; ++i) { @@ -737,14 +740,7 @@ int iova_domain_init_rcaches(struct iova_domain *iovad) } for_each_possible_cpu(cpu) { cpu_rcache =3D per_cpu_ptr(rcache->cpu_rcaches, cpu); - spin_lock_init(&cpu_rcache->lock); - cpu_rcache->loaded =3D iova_magazine_alloc(GFP_KERNEL); - cpu_rcache->prev =3D iova_magazine_alloc(GFP_KERNEL); - if (!cpu_rcache->loaded || !cpu_rcache->prev) { - ret =3D -ENOMEM; - goto out_err; - } } } =20 @@ -777,8 +773,20 @@ static bool __iova_rcache_insert(struct iova_domain *i= ovad, cpu_rcache =3D raw_cpu_ptr(rcache->cpu_rcaches); spin_lock_irqsave(&cpu_rcache->lock, flags); =20 + if (!cpu_rcache->loaded) { + cpu_rcache->loaded =3D iova_magazine_alloc(GFP_ATOMIC | __GFP_NOWARN); + if (!cpu_rcache->loaded) + goto unlock; + } + if (!iova_magazine_full(cpu_rcache->loaded)) { can_insert =3D true; + } else if (!cpu_rcache->prev) { + cpu_rcache->prev =3D iova_magazine_alloc(GFP_ATOMIC | __GFP_NOWARN); + if (!cpu_rcache->prev) + goto unlock; + swap(cpu_rcache->prev, cpu_rcache->loaded); + can_insert =3D true; } else if (!iova_magazine_full(cpu_rcache->prev)) { swap(cpu_rcache->prev, cpu_rcache->loaded); can_insert =3D true; @@ -799,6 +807,7 @@ static bool __iova_rcache_insert(struct iova_domain *io= vad, if (can_insert) iova_magazine_push(cpu_rcache->loaded, iova_pfn); =20 +unlock: spin_unlock_irqrestore(&cpu_rcache->lock, flags); =20 return can_insert; @@ -831,9 +840,9 @@ static unsigned long __iova_rcache_get(struct iova_rcac= he *rcache, cpu_rcache =3D raw_cpu_ptr(rcache->cpu_rcaches); spin_lock_irqsave(&cpu_rcache->lock, flags); =20 - if (!iova_magazine_empty(cpu_rcache->loaded)) { + if (cpu_rcache->loaded && !iova_magazine_empty(cpu_rcache->loaded)) { has_pfn =3D true; - } else if (!iova_magazine_empty(cpu_rcache->prev)) { + } else if (cpu_rcache->prev && !iova_magazine_empty(cpu_rcache->prev)) { swap(cpu_rcache->prev, cpu_rcache->loaded); has_pfn =3D true; } else { --=20 2.54.0.1064.gd145956f57-goog