From nobody Fri Jul 24 04:53:54 2026 Received: from mail-pf1-f178.google.com (mail-pf1-f178.google.com [209.85.210.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 70B2A47D44D for ; Thu, 23 Jul 2026 17:36:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784828213; cv=none; b=I5n7Aa1JYWI9CZG009HumztSRZgsPnZtnLkBYOSMUptrBvsraPHl6SAxY4fBoO2QUhEKyS/YnVc2uL5hJWU3HFRT+rwrVURbSoktlQY1xZCQ0T1KuhXnDNBQfhIRKdjXRv79BIgZ9IBQicg9Rnd5U3LamDtDDMK9APIwsR/pA4Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784828213; c=relaxed/simple; bh=e05SOoAaH6rdxtJDrz7A9LUM75TsCH+ORPFdPnvT+/o=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=miAVqdbW+yWhOjRu8HEoMYEGEVBPg0YfOjUujI5MI64Z8SD04O81sgDFcio7IQgI7BVdKmFBtS45L/v5huhj2v9U4+3VPy3/01jAgc/qvG/ZVhAxdzw5XaJCXf5t7miTH0SgPkgY7D6BKpTvTdu+Ll2SAqkvBVeUKUGIx3iEwp8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=JWCH55cr; arc=none smtp.client-ip=209.85.210.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="JWCH55cr" Received: by mail-pf1-f178.google.com with SMTP id d2e1a72fcca58-848479c9bd5so707632b3a.3 for ; Thu, 23 Jul 2026 10:36:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1784828206; x=1785433006; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=JmjPr4ajWT+pNjYdNv6NRB3T+nqQJNeM5CFnnbec2W4=; b=JWCH55crn8LHQSD8Tn4CAevzsLUpwWnSdnZeyG870/I9I5y9gva8B8QK3cPltZ9PbK V0gz1+md4VLJ397Byzz35/w0SIWrCGZp1Hyh8F9E2prAohCeBD1/LYBgRDgqIFAwmGIY uzdEw7egtqC3BkEJk1k0A/haWNBgtxeCleOVkZkFHqAM5xMlboL6f6jXr3mQuWO7oGB2 3amYn7QUvpw8j8djFCksHHPvTI5INbW7UFY5aM+a0Yb304dAu6+svDVRJY7v8CDRt/E3 z9VV3Tp0A6GV84NXNdJ3nkgK7G4d0MesKW/2my9wwjV4g2czqyk6vAIZp0NKUODh10MZ SEaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784828206; x=1785433006; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=JmjPr4ajWT+pNjYdNv6NRB3T+nqQJNeM5CFnnbec2W4=; b=lODbFaP5woEevh+GeLgB7mQhl8xF6Kxw9CZwqa1krBbUUgREpqRUD6FTjCOLxpwKre BCoP6oUf+dZqvfCJhquH/EZYIKR1+U9veMQ6bAtU9JOpXJgKtgkvZgdp6f9G426PVd/z wxELYmGntofRGWRu/9hTlI0HpYCTpk6PSPA/EzwQkOs40qfp+I0cOoARnY3d12oRuBXq aWr3qbwGek6Eb8LuC/fGXfWeiEv/x+1duQS6dnrv7JqcU0x3+22Tb3oEaHER9SPslYTg XoXiAFRnpc7Al/Eer//HeNPC68V2fYOJ0tdqyFWDoQmYj4Aviao3lzLMQyPBBjCh7zK0 hnrg== X-Forwarded-Encrypted: i=1; AHgh+RoPnM8Xq0ErxPRPvNHFPF7urAhtoMzIRdYPsCZVZeSa0iF5aas5wmF8z0AxHuS4Buyu3vTjvTnFhUtgJ08=@vger.kernel.org X-Gm-Message-State: AOJu0YwywjDMgzt8tg9MbKhO+4nfG7wan52DJRnHZWcGwkulUgyOznD2 RI2WCCGMTon1iZG6C7Swq87xSPAb+JZSM8nwD5LhFtYzm8jOcFAFRyhd X-Gm-Gg: AR+sD13L4sULp/jJuF6U4Twv/t0rDbrxK0uYNwYY15pdIT1nVYojjA2veAenKUFtWOV fremNfJgLyHsqRGPcmcGWN4GgK/upgutSfV4LDmmstedaanqV97W7zIEwGdCqfN4b9cC1ALTaSA zLTyueBP2/QkE9L5pHjCObFUAVJua4q6f43fKqs3OmG1M8m2vMCKG51Bc3ZbeNR6sK3d6x5NI1y AC2mVKB5K+mGBt7CA7dyU+GF5f60Fn2F8poe79zrOwhx59BdzfwmoudVZpb3xwpDETUGezOuTU2 Ff/fwh8kAMKP1jiAUn51YD4yYzSK/KhLHnpetGETBa7oiFQvhPd3VRyGFtYLu5E1/3FJzQYjoNv eALU1oOUpLS6zmm1dy6zgHfKX6BWy/SKoVDbYYYTiVDcAtmLYIoq1d5cnWmiGrkhqLFb2/hHyFU OZTxNy9gEh7xuAgnvF/+AizxUy0kfLJkG8E8/68T55j1nuprJp X-Received: by 2002:a05:6a00:3cc4:b0:845:e7ee:eae7 with SMTP id d2e1a72fcca58-84e2b7fc1f8mr4822860b3a.5.1784828206217; Thu, 23 Jul 2026 10:36:46 -0700 (PDT) Received: from [192.168.0.160] (c-98-225-44-182.hsd1.wa.comcast.net. [98.225.44.182]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-84e20622dedsm2691612b3a.11.2026.07.23.10.36.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Jul 2026 10:36:45 -0700 (PDT) From: Stanislav Kinsburskii Date: Thu, 23 Jul 2026 10:36:33 -0700 Subject: [PATCH v11 1/8] mm/hmm: move page fault handling out of walk callbacks Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260723-hmm-v10-v11-1-c55b003a4b61@gmail.com> References: <20260723-hmm-v10-v11-0-c55b003a4b61@gmail.com> In-Reply-To: <20260723-hmm-v10-v11-0-c55b003a4b61@gmail.com> To: Jason Gunthorpe , Leon Romanovsky , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jonathan Corbet , Shuah Khan , Shuah Khan , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Long Li , Lyude Paul , Danilo Krummrich , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Simona Vetter , Min Ma , Lizhi Hou , Oded Gabbay , skinsburskii@gmail.com Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-hyperv@vger.kernel.org, dri-devel@lists.freedesktop.org, nouveau@lists.freedesktop.org, linux-rdma@vger.kernel.org, Jason Gunthorpe X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1784828202; l=9376; i=skinsburskii@gmail.com; s=20260722; h=from:subject:message-id; bh=e05SOoAaH6rdxtJDrz7A9LUM75TsCH+ORPFdPnvT+/o=; b=gwyU/77AJ6dKu0lZRnFlWCTHfMteTC7dUTtNdZcgpayFHK/4UuYZFQAVucZpomng7uyaHw8h7 T8/k5JdDqvADw77M1tYI9WkF5Vk2uNwAOeuvh4Jkjbevj0cWSnQtBpg X-Developer-Key: i=skinsburskii@gmail.com; a=ed25519; pk=bDpriHBYgeTdkIDweZDCemxsU93neJBOCn3YLIuJpnE= hmm_range_fault() currently triggers page faults from inside the page-table walk callbacks: hmm_vma_walk_pmd(), hmm_vma_walk_pud(), hmm_vma_walk_hugetlb_entry() and the pte-level helper all call hmm_vma_fault(), which in turn calls handle_mm_fault() while the walker still holds nested locks. The pte spinlock is dropped explicitly by each caller, and the hugetlb path manually drops and retakes hugetlb_vma_lock_read around the fault to dodge a deadlock against the walk framework's unconditional unlock. This layering does not extend cleanly to fault handlers that may release mmap_lock (VM_FAULT_RETRY, VM_FAULT_COMPLETED). If the lock is dropped while walk_page_range() is mid-traversal, the VMA can be freed before the walk framework's matching hugetlb_vma_unlock_read(), turning that unlock into a use-after-free. Split the responsibilities the way get_user_pages() does. Walk callbacks become inspect-only: when they detect a range that needs to be faulted in, they record it in struct hmm_vma_walk and return a private sentinel (HMM_FAULT_PENDING). The outer loop in hmm_range_fault() then drops out of walk_page_range(), invokes a new helper hmm_do_fault() that calls handle_mm_fault() with only mmap_lock held, and restarts the walk so the now-present entries are collected into hmm_pfns. No functional change for existing callers. As a side effect the hugetlb callback no longer needs the hugetlb_vma_{un}lock_read dance, and every fault-path exit from the callbacks now releases the pte spinlock on a single, common path. This refactor is also a precursor for adding an unlockable variant of hmm_range_fault() in a follow-up patch. Reviewed-by: Jason Gunthorpe Signed-off-by: Stanislav Kinsburskii --- mm/hmm.c | 118 ++++++++++++++++++++++++++++++++++++++++-------------------= ---- 1 file changed, 75 insertions(+), 43 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index e5c1f4deed24..bc9361a715fa 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -33,8 +33,17 @@ struct hmm_vma_walk { struct hmm_range *range; unsigned long last; + unsigned long end; + unsigned int required_fault; }; =20 +/* + * Internal sentinel returned by walk callbacks when they need a page faul= t. + * The callback stores end/required_fault in hmm_vma_walk; the outer loop + * consumes the sentinel and never propagates it to the caller. + */ +#define HMM_FAULT_PENDING -EAGAIN + enum { HMM_NEED_FAULT =3D 1 << 0, HMM_NEED_WRITE_FAULT =3D 1 << 1, @@ -60,37 +69,25 @@ static int hmm_pfns_fill(unsigned long addr, unsigned l= ong end, } =20 /* - * hmm_vma_fault() - fault in a range lacking valid pmd or pte(s) - * @addr: range virtual start address (inclusive) - * @end: range virtual end address (exclusive) - * @required_fault: HMM_NEED_* flags - * @walk: mm_walk structure - * Return: -EBUSY after page fault, or page fault error + * hmm_record_fault() - record a range that needs to be faulted in * - * This function will be called whenever pmd_none() or pte_none() returns = true, - * or whenever there is no page directory covering the virtual address ran= ge. + * Called by the walk callbacks when they discover that part of the range + * needs a page fault. The callback records what to fault and returns + * HMM_FAULT_PENDING; the outer loop in hmm_range_fault() drops back out of + * walk_page_range() and invokes handle_mm_fault() from a context where no + * page-table or hugetlb_vma_lock is held. */ -static int hmm_vma_fault(unsigned long addr, unsigned long end, - unsigned int required_fault, struct mm_walk *walk) +static int hmm_record_fault(unsigned long addr, unsigned long end, + unsigned int required_fault, + struct mm_walk *walk) { struct hmm_vma_walk *hmm_vma_walk =3D walk->private; - struct vm_area_struct *vma =3D walk->vma; - unsigned int fault_flags =3D FAULT_FLAG_REMOTE; =20 WARN_ON_ONCE(!required_fault); hmm_vma_walk->last =3D addr; - - if (required_fault & HMM_NEED_WRITE_FAULT) { - if (!(vma->vm_flags & VM_WRITE)) - return -EPERM; - fault_flags |=3D FAULT_FLAG_WRITE; - } - - for (; addr < end; addr +=3D PAGE_SIZE) - if (handle_mm_fault(vma, addr, fault_flags, NULL) & - VM_FAULT_ERROR) - return -EFAULT; - return -EBUSY; + hmm_vma_walk->end =3D end; + hmm_vma_walk->required_fault =3D required_fault; + return HMM_FAULT_PENDING; } =20 static unsigned int hmm_pte_need_fault(const struct hmm_vma_walk *hmm_vma_= walk, @@ -174,7 +171,7 @@ static int hmm_vma_walk_hole(unsigned long addr, unsign= ed long end, return hmm_pfns_fill(addr, end, range, HMM_PFN_ERROR); } if (required_fault) - return hmm_vma_fault(addr, end, required_fault, walk); + return hmm_record_fault(addr, end, required_fault, walk); return hmm_pfns_fill(addr, end, range, 0); } =20 @@ -209,7 +206,7 @@ static int hmm_vma_handle_pmd(struct mm_walk *walk, uns= igned long addr, required_fault =3D hmm_range_need_fault(hmm_vma_walk, hmm_pfns, npages, cpu_flags); if (required_fault) - return hmm_vma_fault(addr, end, required_fault, walk); + return hmm_record_fault(addr, end, required_fault, walk); =20 pfn =3D pmd_pfn(pmd) + ((addr & ~PMD_MASK) >> PAGE_SHIFT); for (i =3D 0; addr < end; addr +=3D PAGE_SIZE, i++, pfn++) { @@ -328,7 +325,7 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, uns= igned long addr, fault: pte_unmap(ptep); /* Fault any virtual address we were asked to fault */ - return hmm_vma_fault(addr, end, required_fault, walk); + return hmm_record_fault(addr, end, required_fault, walk); } =20 #ifdef CONFIG_ARCH_HAS_PMD_SOFTLEAVES @@ -371,7 +368,7 @@ static int hmm_vma_handle_absent_pmd(struct mm_walk *wa= lk, unsigned long start, npages, 0); if (required_fault) { if (softleaf_is_device_private(entry)) - return hmm_vma_fault(addr, end, required_fault, walk); + return hmm_record_fault(addr, end, required_fault, walk); else return -EFAULT; } @@ -517,7 +514,7 @@ static int hmm_vma_walk_pud(pud_t *pudp, unsigned long = start, unsigned long end, npages, cpu_flags); if (required_fault) { spin_unlock(ptl); - return hmm_vma_fault(addr, end, required_fault, walk); + return hmm_record_fault(addr, end, required_fault, walk); } =20 pfn =3D pud_pfn(pud) + ((addr & ~PUD_MASK) >> PAGE_SHIFT); @@ -564,21 +561,8 @@ static int hmm_vma_walk_hugetlb_entry(pte_t *pte, unsi= gned long hmask, required_fault =3D hmm_pte_need_fault(hmm_vma_walk, pfn_req_flags, cpu_flags); if (required_fault) { - int ret; - spin_unlock(ptl); - hugetlb_vma_unlock_read(vma); - /* - * Avoid deadlock: drop the vma lock before calling - * hmm_vma_fault(), which will itself potentially take and - * drop the vma lock. This is also correct from a - * protection point of view, because there is no further - * use here of either pte or ptl after dropping the vma - * lock. - */ - ret =3D hmm_vma_fault(addr, end, required_fault, walk); - hugetlb_vma_lock_read(vma); - return ret; + return hmm_record_fault(addr, end, required_fault, walk); } =20 pfn =3D pte_pfn(entry) + ((start & ~hmask) >> PAGE_SHIFT); @@ -637,6 +621,44 @@ static const struct mm_walk_ops hmm_walk_ops =3D { .walk_lock =3D PGWALK_RDLOCK, }; =20 +/* + * hmm_do_fault - fault in a range recorded by a walk callback + * + * Called from the outer loop in hmm_range_fault() after a callback + * returned HMM_FAULT_PENDING. At this point we hold only mmap_lock; + * the page-table spinlock and any hugetlb_vma_lock acquired by the walk + * framework have already been released by the unwind. + * + * Returns -EBUSY on success (all pages faulted, caller should re-walk). + * Returns a negative errno on failure. + */ +static int hmm_do_fault(struct mm_struct *mm, + struct hmm_vma_walk *hmm_vma_walk) +{ + unsigned long addr =3D hmm_vma_walk->last; + unsigned long end =3D hmm_vma_walk->end; + unsigned int required_fault =3D hmm_vma_walk->required_fault; + unsigned int fault_flags =3D FAULT_FLAG_REMOTE; + struct vm_area_struct *vma; + + vma =3D vma_lookup(mm, addr); + if (!vma) + return -EFAULT; + + if (required_fault & HMM_NEED_WRITE_FAULT) { + if (!(vma->vm_flags & VM_WRITE)) + return -EPERM; + fault_flags |=3D FAULT_FLAG_WRITE; + } + + for (; addr < end; addr +=3D PAGE_SIZE) + if (handle_mm_fault(vma, addr, fault_flags, NULL) & + VM_FAULT_ERROR) + return -EFAULT; + + return -EBUSY; +} + /** * hmm_range_fault - try to fault some address in a virtual address range * @range: argument structure @@ -674,6 +696,16 @@ int hmm_range_fault(struct hmm_range *range) return -EBUSY; ret =3D walk_page_range(mm, hmm_vma_walk.last, range->end, &hmm_walk_ops, &hmm_vma_walk); + /* + * When HMM_FAULT_PENDING is returned a walk callback + * recorded a range that needs handle_mm_fault(); + * hmm_do_fault() runs the fault outside walk_page_range() + * (so no page-table or hugetlb_vma_lock is held) and + * returns -EBUSY so the loop re-walks and picks up the + * now-present entries. + */ + if (ret =3D=3D HMM_FAULT_PENDING) + ret =3D hmm_do_fault(mm, &hmm_vma_walk); /* * When -EBUSY is returned the loop restarts with * hmm_vma_walk.last set to an address that has not been stored --=20 2.43.0 From nobody Fri Jul 24 04:53:54 2026 Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E8CCD3845CB for ; Thu, 23 Jul 2026 17:36:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784828217; cv=none; b=bokWL00axunXbU2pEOrD7Wyv120Vn3EWMi1vEmS01fkEwe5ZRq7izCj3V+o3NGn+0dzY+I+w7qlm+wzrq4fkZKnf4EBatWyzgUo+GFBWJOWlcjInF+cDfH/LLo26+jJ7wvOv2B1zz0zY8JWxOpz5s3d2zCaCQWVnSFPGyNAG47s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784828217; c=relaxed/simple; bh=yBveMdLuawThiS1dhv7++wFLml+sRao0RpfraHKbFjw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=bdcDFbaR38iLy+uN2Z/W/W8AQrlNMX59uec74DOHu2I7QWuiyI30SqRWD9Y9qaCPrv/Z3zFDdEUJHu4NAemqolzmDCi+HAnGw6q0btEfiW1Y7ge78WFZ3qku/0OcEAQSkMil912fRQr4soiyQ7XDaPhadxm3gsZKCldDSvJGtsA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=BxbRgTf6; arc=none smtp.client-ip=209.85.210.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="BxbRgTf6" Received: by mail-pf1-f174.google.com with SMTP id d2e1a72fcca58-8485b358552so974715b3a.2 for ; Thu, 23 Jul 2026 10:36:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1784828208; x=1785433008; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=G6LIBYIGv1cFbOC8TwREAPf03viSAAKKc4OU4ULLyfI=; b=BxbRgTf62BynctK6VlKZAEeI71qoJVF8z90NTAarvGJoC859at/SbGkiVr8Ywd8xki OyOl3lBfKd3/eCOXRUGkqnFRTirfII1ek8psqgJ2LTDMqgSwk2ktIdbt9B0CzcDqMA6O 8ylYWv8jDHbtWe6vkm6qhTja1MJojWD2eM1R6BzqLWzTli+7mUcIeFIFOTgFfcqBW8yL V/tfrLH0ioVX5VkB+dcYnbmWh51jYG7ZxMyMWu77zi1wgQKfHWIuLhzGx+3xNOtPcxZ5 liPXFw8sBnuFFQTHVl6Vo8dHXlCbJWAnlEDPE0R67+t+sVXyghj7fMVesloMm6lOXfCb kYJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784828208; x=1785433008; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=G6LIBYIGv1cFbOC8TwREAPf03viSAAKKc4OU4ULLyfI=; b=c2GwzxYXjzyu/oA9g+lSmmBOaC6e6uYxr6zlu4vrSv90i1zi8EHeoMpeyF21CyO6WO kJIMdKCM8QcoPSbqQKKT5LXjjb87PwNfPDqRCmcZ8ZFfRoos8WdmL9SNR3EPrCftiAFT p6lyhWtrLzWUR0llsPoDVotpEm4JYq7vtBraJKAxZUX/nVnUvqEBfYmaQRw0/gKCkSwG Mm9mxLLjb3okBfruvI6jV3eaTyOm9Q0qAs79uwhLIoNgFfgKZOzcWqjvb6ntEUOW2WXN RocBllTjWVSIRZ6oJNpeRQAmttm5GblHrC2QMvh62LJZdwyxY4fFY9RP7scUXUAy77/e DjVw== X-Forwarded-Encrypted: i=1; AHgh+Rq/dHFGEqDUOj75w+afV9YWIKBsWaHeKx5GvWkARXRQ5b6FDEYGebWdWROQcK6wEuQwgkBZn9VRtjfT+mQ=@vger.kernel.org X-Gm-Message-State: AOJu0Yy53NKILxKqRCr0ywZNFghTK2sF6aVeyABwn/nY6N+uk7I7Rt0A u6AXs/7kB5O/h1JCUk2t3QAu01mdj8QUUMdsNsTJuBC6pLQXE72Qe42w X-Gm-Gg: AR+sD12a8dq1/CfhxKOVciL80kvCF3Z7ZWjyHHds84m3TWgG8CIaqAj9WI2FVqXWVmZ igi8GzoyxafStiLvj30CZT9BEHHRhP06mfcG2sfJkRY1/rrGcVu89eD0OALDDHZnIlelFeg90j2 WGA1Amb/pumbOXsuWvETBqrfUAgZDG7aWi/fz+KVtWOO0mFZ6lJlT+41BKTxETq2Por8+Jlz0Xt LBbYMfvsUUteE/gmF61yDsqDmcZzIXDE6wxyXUyjYBayi7aHh9xLW6691d23it3nZvKBpKbMcCQ 6rMkw9J8AQNU/mGmcJNQrdTcgoFORmHstDQyf/DXiHxd32Lo3q2JGElpXxe893DwURmBKoJBmZy UBuAhbG0VXfwMeCNFXK94i8akDxX5wciRV18EmReMPRIzoKz3h1GgCuxzPOwKohvmNV+RJqmQuv sd4DwjHiOQ2iSFJAig0GXBfEQSaSYrq0wOl5PA+yJTo1qGn2I4 X-Received: by 2002:a05:6a00:12c9:b0:848:2f84:735 with SMTP id d2e1a72fcca58-84e2c29544emr4641567b3a.72.1784828208296; Thu, 23 Jul 2026 10:36:48 -0700 (PDT) Received: from [192.168.0.160] (c-98-225-44-182.hsd1.wa.comcast.net. [98.225.44.182]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-84e20622dedsm2691612b3a.11.2026.07.23.10.36.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Jul 2026 10:36:47 -0700 (PDT) From: Stanislav Kinsburskii Date: Thu, 23 Jul 2026 10:36:34 -0700 Subject: [PATCH v11 2/8] mm/hmm: add hmm_range_fault_unlocked_timeout() for mmap lock-drop support Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260723-hmm-v10-v11-2-c55b003a4b61@gmail.com> References: <20260723-hmm-v10-v11-0-c55b003a4b61@gmail.com> In-Reply-To: <20260723-hmm-v10-v11-0-c55b003a4b61@gmail.com> To: Jason Gunthorpe , Leon Romanovsky , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jonathan Corbet , Shuah Khan , Shuah Khan , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Long Li , Lyude Paul , Danilo Krummrich , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Simona Vetter , Min Ma , Lizhi Hou , Oded Gabbay , skinsburskii@gmail.com Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-hyperv@vger.kernel.org, dri-devel@lists.freedesktop.org, nouveau@lists.freedesktop.org, linux-rdma@vger.kernel.org X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1784828202; l=17665; i=skinsburskii@gmail.com; s=20260722; h=from:subject:message-id; bh=yBveMdLuawThiS1dhv7++wFLml+sRao0RpfraHKbFjw=; b=RqFsLiAaylM2q4Ton3CfyZiP4afaKY3FunE4ZxEM2QGEfK5IrWSX3Gu0Dee2EG09uG7N8SK9X 4pTZwMsiwkKBqZYwUTKoY+fzCcabt6gN+J5Gz5Let4XgmBZfY5Ojrar X-Developer-Key: i=skinsburskii@gmail.com; a=ed25519; pk=bDpriHBYgeTdkIDweZDCemxsU93neJBOCn3YLIuJpnE= hmm_range_fault() requires the caller to hold the mmap read lock for the duration of the call. This is incompatible with mappings whose fault handler may release the mmap lock, notably userfaultfd-managed regions, where handle_mm_fault() can return VM_FAULT_RETRY or VM_FAULT_COMPLETED after dropping the lock. Drivers that need to populate device page tables for such mappings have no way to do so today. Add hmm_range_fault_unlocked_timeout() for callers that do not need to hold mmap_lock across any work outside the HMM fault itself. The helper takes mmap_read_lock_killable() internally, calls the common HMM fault implementation, and releases the lock before returning if it is still held. The timeout is specified in jiffies; passing 0 retries indefinitely, while a non-zero timeout makes the helper return -EBUSY when the retry budget expires. The retry deadline is set before refreshing the notifier sequence and acquiring mmap_lock, so contended mmap_lock acquisition is included in the retry budget. After acquiring mmap_lock, the helper also rejects unstable address spaces before walking page tables. When handle_mm_fault() drops mmap_lock, or when the range is invalidated, hmm_range_fault_unlocked_timeout() refreshes range->notifier_seq and retries the walk internally. If the lock was dropped, the retry deadline is also restarted because a lock-dropping fault handler made progress. Ordinary -EBUSY retries keep the existing deadline, preserving the caller's timeout policy for repeated mmu-notifier invalidations. The caller only needs to perform the usual post-success mmu_interval_read_retry() check while holding its update lock before consuming the pfns. If mmap_lock acquisition is interrupted or a fatal signal is pending during retry handling, -EINTR is returned instead. The common implementation conditionally sets FAULT_FLAG_ALLOW_RETRY and FAULT_FLAG_KILLABLE only for hmm_range_fault_unlocked_timeout(). The existing hmm_range_fault() path still passes no locked state, does not allow handle_mm_fault() to drop mmap_lock, and remains a thin wrapper preserving the existing API contract for current callers. The previous refactor that moved page fault handling out of the page-table walk callbacks is what makes this change small. Faults now run after walk_page_range() has unwound, with only mmap_lock held, so dropping it does not interact with the walker's pte spinlock or hugetlb_vma_lock. Hugetlb regions therefore participate in the unlocked path uniformly with PTE- and PMD-level mappings; no special case is required. Documentation/mm/hmm.rst is updated with a description of the new API and the recommended caller pattern. Signed-off-by: Stanislav Kinsburskii --- Documentation/mm/hmm.rst | 79 +++++++++++++++------- include/linux/hmm.h | 2 + mm/hmm.c | 168 +++++++++++++++++++++++++++++++++++++++----= ---- 3 files changed, 197 insertions(+), 52 deletions(-) diff --git a/Documentation/mm/hmm.rst b/Documentation/mm/hmm.rst index 7d61b7a8b65b..e021218ada58 100644 --- a/Documentation/mm/hmm.rst +++ b/Documentation/mm/hmm.rst @@ -156,42 +156,57 @@ During the ops->invalidate() callback the device driv= er must perform the update action to the range (mark range read only, or fully unmap, etc.). T= he device must complete the update before the driver callback returns. =20 -When the device driver wants to populate a range of virtual addresses, it = can -use:: +When the device driver wants to populate a range of virtual addresses, the +normal interface is:: =20 - int hmm_range_fault(struct hmm_range *range); + int hmm_range_fault_unlocked_timeout(struct hmm_range *range, + unsigned long timeout); =20 It will trigger a page fault on missing or read-only entries if write acce= ss is requested (see below). Page faults use the generic mm page fault code path= just -like a CPU page fault. The usage pattern is:: +like a CPU page fault. + +The caller must not hold ``mmap_read_lock`` before the call. +``hmm_range_fault_unlocked_timeout()`` takes the mmap read lock internally= and +allows ``handle_mm_fault()`` to drop it during fault handling. This is req= uired +for VMAs whose fault handlers may release the mmap lock, for example regio= ns +managed by ``userfaultfd``. + +If the mmap lock is dropped or the range is invalidated, the function refr= eshes +``range->notifier_seq`` and restarts the walk internally. ``-EINTR`` is re= turned +if mmap lock acquisition is interrupted or a fatal signal is pending during +retry handling. + +The timeout is specified in jiffies; passing ``0`` means retry indefinitel= y. The +timeout exists to preserve caller policy for repeated mmu-notifier invalid= ation +and is checked between retry attempts. HMM does not interrupt page fault +handling when the timeout expires, but returns ``-EBUSY`` if the retry bud= get is +exhausted before a stable range is obtained. + +The usage pattern is:: =20 int driver_populate_range(...) { struct hmm_range range; + unsigned long timeout; ... =20 + timeout =3D msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT); range.notifier =3D &interval_sub; range.start =3D ...; range.end =3D ...; range.hmm_pfns =3D ...; =20 - if (!mmget_not_zero(interval_sub->notifier.mm)) + if (!mmget_not_zero(interval_sub.mm)) return -EFAULT; =20 again: - range.notifier_seq =3D mmu_interval_read_begin(&interval_sub); - mmap_read_lock(mm); - ret =3D hmm_range_fault(&range); - if (ret) { - mmap_read_unlock(mm); - if (ret =3D=3D -EBUSY) - goto again; - return ret; - } - mmap_read_unlock(mm); + ret =3D hmm_range_fault_unlocked_timeout(&range, timeout); + if (ret) + goto out_put; =20 take_lock(driver->update); - if (mmu_interval_read_retry(&ni, range.notifier_seq) { + if (mmu_interval_read_retry(range.notifier, range.notifier_seq)) { release_lock(driver->update); goto again; } @@ -200,13 +215,31 @@ like a CPU page fault. The usage pattern is:: * under the update lock */ =20 release_lock(driver->update); - return 0; + ret =3D 0; + + out_put: + mmput(interval_sub.mm); + return ret; } =20 The driver->update lock is the same lock that the driver takes inside its invalidate() callback. That lock must be held before calling mmu_interval_read_retry() to avoid any race with a concurrent CPU page tab= le -update. +update. The retry check must use the same notifier and sequence number sto= red +in ``range`` by ``hmm_range_fault_unlocked_timeout()``. + +Holding the mmap lock across HMM faults +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Most callers should use ``hmm_range_fault_unlocked_timeout()``. If a driver +really needs to hold the mmap lock across work outside HMM, it can use:: + + int hmm_range_fault(struct hmm_range *range); + +The mmap lock must be held by the caller and will remain held on return. T= his +interface cannot support VMAs whose fault handlers need to drop the mmap l= ock. +New callers should prefer ``hmm_range_fault_unlocked_timeout()`` unless th= ey +have a specific requirement to keep the mmap lock held across the call. =20 Leverage default_flags and pfn_flags_mask =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D @@ -221,8 +254,8 @@ permission, it sets:: range->default_flags =3D HMM_PFN_REQ_FAULT; range->pfn_flags_mask =3D 0; =20 -and calls hmm_range_fault() as described above. This will fill fault all p= ages -in the range with at least read permission. +and calls the HMM range fault helper as described above. This will fault +all pages in the range with at least read permission. =20 Now let's say the driver wants to do the same except for one page in the r= ange for which it wants to have write permission. Now driver set:: @@ -236,9 +269,9 @@ address =3D=3D range->start + (index_of_write << PAGE_S= HIFT) it will fault with write permission i.e., if the CPU pte does not have write permission set t= hen HMM will call handle_mm_fault(). =20 -After hmm_range_fault completes the flag bits are set to the current state= of -the page tables, ie HMM_PFN_VALID | HMM_PFN_WRITE will be set if the page = is -writable. +After the HMM range fault helper completes the flag bits are set to the +current state of the page tables, ie HMM_PFN_VALID | HMM_PFN_WRITE will be +set if the page is writable. =20 =20 Represent and manage device memory from core kernel point of view diff --git a/include/linux/hmm.h b/include/linux/hmm.h index db75ffc949a7..6f04e3932f5b 100644 --- a/include/linux/hmm.h +++ b/include/linux/hmm.h @@ -123,6 +123,8 @@ struct hmm_range { * Please see Documentation/mm/hmm.rst for how to use the range API. */ int hmm_range_fault(struct hmm_range *range); +int hmm_range_fault_unlocked_timeout(struct hmm_range *range, + unsigned long timeout); =20 /* * HMM_RANGE_DEFAULT_TIMEOUT - default timeout (ms) when waiting for a ran= ge diff --git a/mm/hmm.c b/mm/hmm.c index bc9361a715fa..2b05c53b82dc 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -17,6 +17,7 @@ #include #include #include +#include #include #include #include @@ -32,6 +33,7 @@ =20 struct hmm_vma_walk { struct hmm_range *range; + bool *locked; unsigned long last; unsigned long end; unsigned int required_fault; @@ -44,6 +46,14 @@ struct hmm_vma_walk { */ #define HMM_FAULT_PENDING -EAGAIN =20 +/* + * Internal sentinel returned by hmm_do_fault() when handle_mm_fault() + * completes a page fault with the mmap lock dropped. hmm_do_fault() sets + * *locked =3D false; the outer loop consumes the sentinel and never propa= gates + * it to the caller. + */ +#define HMM_FAULT_UNLOCKED -ENOLCK + enum { HMM_NEED_FAULT =3D 1 << 0, HMM_NEED_WRITE_FAULT =3D 1 << 1, @@ -73,9 +83,9 @@ static int hmm_pfns_fill(unsigned long addr, unsigned lon= g end, * * Called by the walk callbacks when they discover that part of the range * needs a page fault. The callback records what to fault and returns - * HMM_FAULT_PENDING; the outer loop in hmm_range_fault() drops back out of - * walk_page_range() and invokes handle_mm_fault() from a context where no - * page-table or hugetlb_vma_lock is held. + * HMM_FAULT_PENDING; the outer loop in hmm_range_fault_locked() drops + * back out of walk_page_range() and invokes handle_mm_fault() from a cont= ext + * where no page-table or hugetlb_vma_lock is held. */ static int hmm_record_fault(unsigned long addr, unsigned long end, unsigned int required_fault, @@ -624,7 +634,7 @@ static const struct mm_walk_ops hmm_walk_ops =3D { /* * hmm_do_fault - fault in a range recorded by a walk callback * - * Called from the outer loop in hmm_range_fault() after a callback + * Called from the outer loop in hmm_range_fault_locked() after a callback * returned HMM_FAULT_PENDING. At this point we hold only mmap_lock; * the page-table spinlock and any hugetlb_vma_lock acquired by the walk * framework have already been released by the unwind. @@ -641,6 +651,9 @@ static int hmm_do_fault(struct mm_struct *mm, unsigned int fault_flags =3D FAULT_FLAG_REMOTE; struct vm_area_struct *vma; =20 + if (hmm_vma_walk->locked) + fault_flags |=3D FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE; + vma =3D vma_lookup(mm, addr); if (!vma) return -EFAULT; @@ -651,37 +664,34 @@ static int hmm_do_fault(struct mm_struct *mm, fault_flags |=3D FAULT_FLAG_WRITE; } =20 - for (; addr < end; addr +=3D PAGE_SIZE) - if (handle_mm_fault(vma, addr, fault_flags, NULL) & - VM_FAULT_ERROR) - return -EFAULT; + for (; addr < end; addr +=3D PAGE_SIZE) { + vm_fault_t ret; + + ret =3D handle_mm_fault(vma, addr, fault_flags, NULL); + + if (ret & (VM_FAULT_COMPLETED | VM_FAULT_RETRY)) { + *hmm_vma_walk->locked =3D false; + return HMM_FAULT_UNLOCKED; + } + + if (ret & VM_FAULT_ERROR) { + int err =3D vm_fault_to_errno(ret, 0); + + if (WARN_ON(!err)) + err =3D -EINVAL; + + return err; + } + } =20 return -EBUSY; } =20 -/** - * hmm_range_fault - try to fault some address in a virtual address range - * @range: argument structure - * - * Returns 0 on success or one of the following error codes: - * - * -EINVAL: Invalid arguments or mm or virtual address is in an invalid vma - * (e.g., device file vma). - * -ENOMEM: Out of memory. - * -EPERM: Invalid permission (e.g., asking for write and range is read - * only). - * -EBUSY: The range has been invalidated and the caller needs to wait for - * the invalidation to finish. - * -EFAULT: A page was requested to be valid and could not be made val= id - * ie it has no backing VMA or it is illegal to access - * - * This is similar to get_user_pages(), except that it can read the page t= ables - * without mutating them (ie causing faults). - */ -int hmm_range_fault(struct hmm_range *range) +static int hmm_range_fault_locked(struct hmm_range *range, bool *locked) { struct hmm_vma_walk hmm_vma_walk =3D { .range =3D range, + .locked =3D locked, .last =3D range->start, }; struct mm_struct *mm =3D range->notifier->mm; @@ -704,8 +714,14 @@ int hmm_range_fault(struct hmm_range *range) * returns -EBUSY so the loop re-walks and picks up the * now-present entries. */ - if (ret =3D=3D HMM_FAULT_PENDING) + if (ret =3D=3D HMM_FAULT_PENDING) { ret =3D hmm_do_fault(mm, &hmm_vma_walk); + if (ret =3D=3D HMM_FAULT_UNLOCKED) { + if (fatal_signal_pending(current)) + return -EINTR; + return -EBUSY; + } + } /* * When -EBUSY is returned the loop restarts with * hmm_vma_walk.last set to an address that has not been stored @@ -715,8 +731,102 @@ int hmm_range_fault(struct hmm_range *range) } while (ret =3D=3D -EBUSY); return ret; } + +/** + * hmm_range_fault - try to fault some address in a virtual address range + * @range: argument structure + * + * Returns 0 on success or one of the following error codes: + * + * -EINVAL: Invalid arguments or mm or virtual address is in an invalid vma + * (e.g., device file vma). + * -ENOMEM: Out of memory. + * -EPERM: Invalid permission (e.g., asking for write and range is read + * only). + * -EBUSY: The range has been invalidated and the caller needs to wait for + * the invalidation to finish. + * -EFAULT: A page was requested to be valid and could not be made val= id + * ie it has no backing VMA or it is illegal to access + * + * This is similar to get_user_pages(), except that it can read the page t= ables + * without mutating them (ie causing faults). + * + * The mmap lock must be held by the caller and will remain held on return. + * New users should prefer hmm_range_fault_unlocked_timeout() unless they + * specifically need to keep the mmap lock held across the call. This help= er + * cannot support VMAs whose fault handlers need to drop the mmap lock. + */ +int hmm_range_fault(struct hmm_range *range) +{ + return hmm_range_fault_locked(range, NULL); +} EXPORT_SYMBOL(hmm_range_fault); =20 +/** + * hmm_range_fault_unlocked_timeout - fault in a range with a retry timeout + * @range: argument structure + * @timeout: timeout in jiffies for internal -EBUSY retries, or 0 to retry + * indefinitely + * + * The caller must not hold the mmap lock. The function takes the mmap read + * lock internally and allows handle_mm_fault() to drop it during faults. = If + * the mmap lock is dropped or the range is invalidated, the function refr= eshes + * range->notifier_seq and restarts the walk internally. + * + * Passing 0 for @timeout retries indefinitely. A non-zero @timeout is a c= aller + * policy limit for repeated mmu-notifier invalidation retries. HMM does n= ot + * interrupt page fault handling when the timeout expires, but returns -EB= USY + * if the retry budget is exhausted before a stable range is obtained. + * + * Returns 0 on success or one of the error codes documented for + * hmm_range_fault(). -EINTR is returned if mmap_lock acquisition is + * interrupted or a fatal signal is pending during retry handling. + */ +int hmm_range_fault_unlocked_timeout(struct hmm_range *range, + unsigned long timeout) +{ + struct mm_struct *mm =3D range->notifier->mm; + unsigned long deadline =3D 0; + bool locked =3D false; + int ret; + + do { + /* + * If the previous fault dropped mmap_lock, then the fault + * handler made progress. Restart the retry timeout in that + * case, but keep the existing deadline for ordinary -EBUSY + * retries. + */ + if (timeout && !locked) + deadline =3D jiffies + timeout; + + range->notifier_seq =3D + mmu_interval_read_begin(range->notifier); + + ret =3D mmap_read_lock_killable(mm); + if (ret) + return ret; + + if (check_stable_address_space(mm)) { + mmap_read_unlock(mm); + return -EFAULT; + } + + if (timeout && time_after(jiffies, deadline)) { + mmap_read_unlock(mm); + return -EBUSY; + } + + locked =3D true; + ret =3D hmm_range_fault_locked(range, &locked); + if (locked) + mmap_read_unlock(mm); + } while (ret =3D=3D -EBUSY); + + return ret; +} +EXPORT_SYMBOL(hmm_range_fault_unlocked_timeout); + /** * hmm_dma_map_alloc - Allocate HMM map structure * @dev: device to allocate structure for --=20 2.43.0 From nobody Fri Jul 24 04:53:54 2026 Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8FE4D47D455 for ; Thu, 23 Jul 2026 17:36:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784828220; cv=none; b=fL0ck7rHrsSzkWUQTJCyLrwhXYn6hMzWZ6BBvsMHBpQB4kKjLl/f/tiaq9szwDoXoosYnRQApWhqD9VfOUw1adtgcWEyV4beRqyGoh3WBjTS4bfYN7b+kz94ctGN79f3a/epyW+eVTnqhyx2uOjhL1wXDeu5rjuANsP8MJOEMVQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784828220; c=relaxed/simple; bh=AMv7G8JGzmbmB+WwCDHthj1N45IFPqE+bBj6HkmdaG8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=KQFpMjj9p2rHItZg7Mie3qeSuOpC7PRWwzOdc7BMYcfJB9WHOLGm2kjSAFDxlsoovBuiu1+D1EdmpbIqGBUcTXvW8GdtzYlpLzNnoeCqaFVLqnR2BnfEzUYNdA9Lsvk0p6tSbKH6/+HH5nSWCH6QuQ15b3ppubP3p31QGemMGOU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=XfueAbTf; arc=none smtp.client-ip=209.85.210.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="XfueAbTf" Received: by mail-pf1-f170.google.com with SMTP id d2e1a72fcca58-845b6d9bf39so906457b3a.1 for ; Thu, 23 Jul 2026 10:36:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1784828210; x=1785433010; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=3cb8/RR6ht/AFgcmZ+ZVlj45R/ECVd/hJBDF8v5cizE=; b=XfueAbTfTPhV87X8D7Nuag/56w+7XjWr+BROHbjwoo5uYlUZrPLutzsB8uBAUTHy58 d/6Hwesdu94XiitvSctXXWz4AFQ+OgLqT6clkOybBGaiHMNAmOM8WeXNgcy0YwQZsEbU D9Xe0//MkbHTGZO5YODNWDbVVgZARd0FmNsK+EteJnGFp0cPcCUOrhb72UCeX7KVs52H n7DIO3t2OvYoVY7ZaciWzRZ7vpHEoW+Rlk9zMBz6I/Uj0x6i1VdgaBANMy3w4nJeeHV0 LvbLm4P12TqISp0jbvs3LvrPMi3CgI2K9kD790TPzea1jKPFBeANvSWlAE2dLkXTGYDa 9txw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784828210; x=1785433010; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=3cb8/RR6ht/AFgcmZ+ZVlj45R/ECVd/hJBDF8v5cizE=; b=TCcchlCSJccgklm8pqzd1cyUCmgxS/TOuw8ovxextVycQNwrjNT1FQ9kDzC8OQjTZT yWtm3/cfHFJlyyzzTtxCsHXyRwa7QPKvFsx64KYAt41rq4i1owCfuu98UtS93jCP/Mfv yW/ePcm8NJNS7Sdd/0M9ULLC5NEwR4G8/MMCUAkvpUeE8yzpkNDRLExMCdh+zph5OV1m ZQP89Zu3mxUnRMgEOB6jGiRoGI6TvdvOeSt6bHwaAihsQMlz6Jkb2UquTVu8Yv1X+NCu 95F8EGIMWM04ZjPILpBbnBvlxRaTihkH8MDFhRLeJSjwASwo66Q3pNRuj0gCVZqqvzyk 4l8Q== X-Forwarded-Encrypted: i=1; AHgh+Rogfd4NUEjHi9kVWYXsydgkK0UeZZBT86B49qK5d/9VyERo5Zrzw6MYwIIkHz6s+CXF4gaEhCqdBPZK5Tg=@vger.kernel.org X-Gm-Message-State: AOJu0YwnblKWjg3wlldv05dEORVJKtuvzh4NPPr6RD0PuW/2FHlptnJ0 vZnj0Mkpq8WfKw12PeMQyWV+s49qzHjR42EMh/W3ekH36L4SRgQYe8ZS X-Gm-Gg: AR+sD10I2mNesMF9DWa42WhJduf4u2tZ5+b4JtBpyujLMq6b5jHEK/28uoFIaI1T3uk 1BcE8gV8eUhEMkZ5bc38asRN9tm3YzXMoGZqiAWMwkIfr5krV4f3J/V73htOIDQLHfA3xG7EJLL rtRJSQY5i3DzB4SMi/DC+z+n5yf37+AmvoeiBzoxiIxtqv0KGL+ACIMuf9aQREtc54a6fOh/ojH N3483EvfuICCdVxJyzwe8OyDMpeej0SzMAZ4/GPQhiXU8S5zud1w7uFKdexcftZ9dEEmoCJ/JoS 0OpzDJTVOwAX+mwGFmADIcYxm+IliXLgLhpRyivAt+PzeAOO/zFrGVjCRyNV2DM8A1bhMR7EE67 HwyfgMcTHCja/LUaWt4E8maw7NSlZQ85QaBpN+aQVTKpiw6UY5CJXXCw6RlzbUnA1RK7vxnJESR rx0S51r0oH69t4Tf9QRLKOiXKIBqTtfgtvKzGh4DegRUlj7GtE X-Received: by 2002:a05:6a00:1743:b0:848:2a69:97be with SMTP id d2e1a72fcca58-84e2e83fdd3mr3652722b3a.10.1784828209948; Thu, 23 Jul 2026 10:36:49 -0700 (PDT) Received: from [192.168.0.160] (c-98-225-44-182.hsd1.wa.comcast.net. [98.225.44.182]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-84e20622dedsm2691612b3a.11.2026.07.23.10.36.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Jul 2026 10:36:49 -0700 (PDT) From: Stanislav Kinsburskii Date: Thu, 23 Jul 2026 10:36:35 -0700 Subject: [PATCH v11 3/8] selftests/mm: add HMM test for mmap lock-dropping faults Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260723-hmm-v10-v11-3-c55b003a4b61@gmail.com> References: <20260723-hmm-v10-v11-0-c55b003a4b61@gmail.com> In-Reply-To: <20260723-hmm-v10-v11-0-c55b003a4b61@gmail.com> To: Jason Gunthorpe , Leon Romanovsky , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jonathan Corbet , Shuah Khan , Shuah Khan , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Long Li , Lyude Paul , Danilo Krummrich , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Simona Vetter , Min Ma , Lizhi Hou , Oded Gabbay , skinsburskii@gmail.com Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-hyperv@vger.kernel.org, dri-devel@lists.freedesktop.org, nouveau@lists.freedesktop.org, linux-rdma@vger.kernel.org X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1784828202; l=9280; i=skinsburskii@gmail.com; s=20260722; h=from:subject:message-id; bh=AMv7G8JGzmbmB+WwCDHthj1N45IFPqE+bBj6HkmdaG8=; b=BuqNY2TIJTKZ+JDj0LwhTCH9DL3tOw+s1oGAQTVqbv/O0Es16a2w6TtDqCrW+/bkwYm1G/T+y TC27ldxQkp6C6wCnvMt0iWL55Wyx6EropXrTYx43zSnmS/uz5vWN/I1 X-Developer-Key: i=skinsburskii@gmail.com; a=ed25519; pk=bDpriHBYgeTdkIDweZDCemxsU93neJBOCn3YLIuJpnE= Add test_hmm coverage for the HMM lock-dropping fault path. The test module gets a new HMM_DMIRROR_READ_UNLOCKED ioctl that calls hmm_range_fault_unlocked_timeout() with a timeout of 0, exercising the unbounded retry mode while allowing the mmap lock to be dropped during fault handling. Add a userfaultfd_read selftest that registers an anonymous mapping with UFFDIO_REGISTER_MODE_MISSING, services the faults from a handler thread with UFFDIO_COPY, and verifies that HMM can read back the data supplied by the handler. This exercises the path where handle_mm_fault() drops mmap_lock and hmm_range_fault_unlocked_timeout() restarts the walk internally. Assisted-by: GitHub-Copilot:claude-opus-4.6 Signed-off-by: Stanislav Kinsburskii --- lib/test_hmm.c | 107 ++++++++++++++++++++++- lib/test_hmm_uapi.h | 1 + tools/testing/selftests/mm/hmm-tests.c | 150 +++++++++++++++++++++++++++++= ++++ 3 files changed, 257 insertions(+), 1 deletion(-) diff --git a/lib/test_hmm.c b/lib/test_hmm.c index 45c0cb992218..6205fb313bd0 100644 --- a/lib/test_hmm.c +++ b/lib/test_hmm.c @@ -389,6 +389,67 @@ static int dmirror_range_fault(struct dmirror *dmirror, return ret; } =20 +static int dmirror_range_fault_unlocked(struct dmirror *dmirror, + struct hmm_range *range, + unsigned long timeout) +{ + int ret; + + while (true) { + ret =3D hmm_range_fault_unlocked_timeout(range, timeout); + if (ret) + goto out; + + mutex_lock(&dmirror->mutex); + if (mmu_interval_read_retry(range->notifier, + range->notifier_seq)) { + mutex_unlock(&dmirror->mutex); + continue; + } + break; + } + + ret =3D dmirror_do_fault(dmirror, range); + + mutex_unlock(&dmirror->mutex); +out: + return ret; +} + +static int dmirror_fault_unlocked(struct dmirror *dmirror, + unsigned long start, + unsigned long end, bool write, + unsigned long timeout) +{ + struct mm_struct *mm =3D dmirror->notifier.mm; + unsigned long addr; + unsigned long pfns[32]; + struct hmm_range range =3D { + .notifier =3D &dmirror->notifier, + .hmm_pfns =3D pfns, + .pfn_flags_mask =3D 0, + .default_flags =3D + HMM_PFN_REQ_FAULT | (write ? HMM_PFN_REQ_WRITE : 0), + .dev_private_owner =3D dmirror->mdevice, + }; + int ret =3D 0; + + if (!mmget_not_zero(mm)) + return -EFAULT; + + for (addr =3D start; addr < end; addr =3D range.end) { + range.start =3D addr; + range.end =3D min(addr + (ARRAY_SIZE(pfns) << PAGE_SHIFT), end); + + ret =3D dmirror_range_fault_unlocked(dmirror, &range, timeout); + if (ret) + break; + } + + mmput(mm); + return ret; +} + static int dmirror_fault(struct dmirror *dmirror, unsigned long start, unsigned long end, bool write) { @@ -488,6 +549,48 @@ static int dmirror_read(struct dmirror *dmirror, struc= t hmm_dmirror_cmd *cmd) return ret; } =20 +static int dmirror_read_unlocked(struct dmirror *dmirror, + struct hmm_dmirror_cmd *cmd, + unsigned long timeout) +{ + struct dmirror_bounce bounce; + unsigned long start, end; + unsigned long size =3D cmd->npages << PAGE_SHIFT; + int ret; + + start =3D cmd->addr; + end =3D start + size; + if (end < start) + return -EINVAL; + + ret =3D dmirror_bounce_init(&bounce, start, size); + if (ret) + return ret; + + while (1) { + mutex_lock(&dmirror->mutex); + ret =3D dmirror_do_read(dmirror, start, end, &bounce); + mutex_unlock(&dmirror->mutex); + if (ret !=3D -ENOENT) + break; + + start =3D cmd->addr + (bounce.cpages << PAGE_SHIFT); + ret =3D dmirror_fault_unlocked(dmirror, start, end, false, timeout); + if (ret) + break; + cmd->faults++; + } + + if (ret =3D=3D 0) { + if (copy_to_user(u64_to_user_ptr(cmd->ptr), bounce.ptr, + bounce.size)) + ret =3D -EFAULT; + } + cmd->cpages =3D bounce.cpages; + dmirror_bounce_fini(&bounce); + return ret; +} + static int dmirror_do_write(struct dmirror *dmirror, unsigned long start, unsigned long end, struct dmirror_bounce *bounce) { @@ -1572,7 +1675,9 @@ static long dmirror_fops_unlocked_ioctl(struct file *= filp, dmirror->flags =3D cmd.npages; ret =3D 0; break; - + case HMM_DMIRROR_READ_UNLOCKED: + ret =3D dmirror_read_unlocked(dmirror, &cmd, 0); + break; default: return -EINVAL; } diff --git a/lib/test_hmm_uapi.h b/lib/test_hmm_uapi.h index f94c6d457338..ea9b0ec404fb 100644 --- a/lib/test_hmm_uapi.h +++ b/lib/test_hmm_uapi.h @@ -38,6 +38,7 @@ struct hmm_dmirror_cmd { #define HMM_DMIRROR_CHECK_EXCLUSIVE _IOWR('H', 0x06, struct hmm_dmirror_cm= d) #define HMM_DMIRROR_RELEASE _IOWR('H', 0x07, struct hmm_dmirror_cmd) #define HMM_DMIRROR_FLAGS _IOWR('H', 0x08, struct hmm_dmirror_cmd) +#define HMM_DMIRROR_READ_UNLOCKED _IOWR('H', 0x09, struct hmm_dmirror_cmd) =20 #define HMM_DMIRROR_FLAG_FAIL_ALLOC (1ULL << 0) =20 diff --git a/tools/testing/selftests/mm/hmm-tests.c b/tools/testing/selftes= ts/mm/hmm-tests.c index 6fccbdab02ee..5acb728666f8 100644 --- a/tools/testing/selftests/mm/hmm-tests.c +++ b/tools/testing/selftests/mm/hmm-tests.c @@ -29,6 +29,10 @@ #include #include #include +#include +#include +#include +#include =20 /* * This is a private UAPI to the kernel test module so it isn't exported @@ -2952,4 +2956,150 @@ TEST_F_TIMEOUT(hmm, benchmark_thp_migration, 120) &thp_results, ®ular_results); } } +/* + * Test that HMM can fault in pages backed by userfaultfd using the + * hmm_range_fault_unlocked_timeout() path with no timeout. This exercises + * the lock-drop retry logic in the HMM framework. + */ +struct uffd_thread_args { + int uffd; + int stop_fd; + void *page_buffer; + unsigned long page_size; +}; + +static void *uffd_handler_thread(void *arg) +{ + struct uffd_thread_args *args =3D arg; + struct uffd_msg msg; + struct uffdio_copy copy; + struct pollfd pollfd[2]; + int ret; + + pollfd[0].fd =3D args->uffd; + pollfd[0].events =3D POLLIN; + pollfd[1].fd =3D args->stop_fd; + pollfd[1].events =3D POLLIN; + + while (1) { + ret =3D poll(pollfd, 2, -1); + if (ret <=3D 0) + break; + if (pollfd[1].revents) + break; + if (!(pollfd[0].revents & POLLIN)) + break; + + ret =3D read(args->uffd, &msg, sizeof(msg)); + if (ret !=3D sizeof(msg)) + break; + + if (msg.event !=3D UFFD_EVENT_PAGEFAULT) + break; + + /* Fill the page with a known pattern */ + memset(args->page_buffer, 0xAB, args->page_size); + + copy.dst =3D msg.arg.pagefault.address & ~(args->page_size - 1); + copy.src =3D (unsigned long)args->page_buffer; + copy.len =3D args->page_size; + copy.mode =3D 0; + copy.copy =3D 0; + + ret =3D ioctl(args->uffd, UFFDIO_COPY, ©); + if (ret < 0) + break; + } + + return NULL; +} + +TEST_F(hmm, userfaultfd_read) +{ + struct hmm_buffer *buffer; + struct uffd_thread_args uffd_args; + unsigned long npages; + unsigned long size; + unsigned long i; + unsigned char *ptr; + pthread_t thread; + int uffd; + int stop_fd; + int ret; + struct uffdio_api api; + struct uffdio_register reg; + uint64_t stop =3D 1; + ssize_t nwrite; + + npages =3D 4; + size =3D npages << self->page_shift; + + /* Create userfaultfd */ + uffd =3D syscall(__NR_userfaultfd, O_CLOEXEC | O_NONBLOCK); + if (uffd < 0) + SKIP(return, "userfaultfd not available"); + + api.api =3D UFFD_API; + api.features =3D 0; + ret =3D ioctl(uffd, UFFDIO_API, &api); + ASSERT_EQ(ret, 0); + + buffer =3D malloc(sizeof(*buffer)); + ASSERT_NE(buffer, NULL); + + buffer->fd =3D -1; + buffer->size =3D size; + buffer->mirror =3D malloc(size); + ASSERT_NE(buffer->mirror, NULL); + + /* Create anonymous mapping */ + buffer->ptr =3D mmap(NULL, size, + PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS, + -1, 0); + ASSERT_NE(buffer->ptr, MAP_FAILED); + + /* Register the region with userfaultfd */ + reg.range.start =3D (unsigned long)buffer->ptr; + reg.range.len =3D size; + reg.mode =3D UFFDIO_REGISTER_MODE_MISSING; + ret =3D ioctl(uffd, UFFDIO_REGISTER, ®); + ASSERT_EQ(ret, 0); + + /* Set up the handler thread */ + uffd_args.uffd =3D uffd; + stop_fd =3D eventfd(0, EFD_CLOEXEC); + ASSERT_GE(stop_fd, 0); + uffd_args.stop_fd =3D stop_fd; + uffd_args.page_buffer =3D malloc(self->page_size); + ASSERT_NE(uffd_args.page_buffer, NULL); + uffd_args.page_size =3D self->page_size; + + ret =3D pthread_create(&thread, NULL, uffd_handler_thread, &uffd_args); + ASSERT_EQ(ret, 0); + + /* + * Use the unlocked read path which allows the mmap lock to be + * dropped during the fault, enabling userfaultfd resolution. + */ + ret =3D hmm_dmirror_cmd(self->fd, HMM_DMIRROR_READ_UNLOCKED, + buffer, npages); + ASSERT_EQ(ret, 0); + ASSERT_EQ(buffer->cpages, npages); + + /* Verify the device read the data filled by the uffd handler */ + ptr =3D buffer->mirror; + for (i =3D 0; i < size; ++i) + ASSERT_EQ(ptr[i], (unsigned char)0xAB); + + nwrite =3D write(stop_fd, &stop, sizeof(stop)); + ASSERT_EQ(nwrite, sizeof(stop)); + pthread_join(thread, NULL); + close(stop_fd); + free(uffd_args.page_buffer); + close(uffd); + hmm_buffer_free(buffer); +} + + TEST_HARNESS_MAIN --=20 2.43.0 From nobody Fri Jul 24 04:53:54 2026 Received: from mail-pf1-f180.google.com (mail-pf1-f180.google.com [209.85.210.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DE88647D466 for ; Thu, 23 Jul 2026 17:36:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784828221; cv=none; b=fSjrWaPDuQKXdPAnh+PNJoOgoIt56Zp5g5DIsWEvYvXVLuqSqY+A/yd7GmM70NY9+2jEeN+YvZRwiz6eNxlP0H2sV0AC2I8TuoIqJ/FN8qCt80s7/Y9mlqhrq5orC4Dd99qfntGIBHWTIjq/e8YEFHr+yqpZkmRL0nmZjRUexEo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784828221; c=relaxed/simple; bh=ZyAhBAfWZePcXjSsL7QqvsrSYU5bhX/lu/bgMYHTJjg=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=TW7ag8U+ydY39hAOBDyZyrt1Pr0sOLnkvGARwrq5zvpKVWC9+7RYxOL64luAxeQ5wPk+BkImgYFpHblj9A5sreMYE5R0HZ9UJqRVBhO+g8IYSBIv4FI/PZsn/sMV5x+3KKp8jeWSTxvy35hKLzDMYcTVz+cU5qezkqa3Cg5bdTY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=TxDQV4ad; arc=none smtp.client-ip=209.85.210.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="TxDQV4ad" Received: by mail-pf1-f180.google.com with SMTP id d2e1a72fcca58-8485b358552so974784b3a.2 for ; Thu, 23 Jul 2026 10:36:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1784828211; x=1785433011; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=3slUHnl834IP76VoxdiRinKrcbFYlUZTVsrXFJZ4JdY=; b=TxDQV4adhqRRAekWKez2o8M9b2r1N8AqJJwAYODXxBXkLYFJQkJEdchOdxcj8bhvrM VAUQBDtVhqqz1QYGH9SkvlUvzw+PSD3qHq/CtSUz95oRVoBgIZpRGtNy6YzFO9ze8Ahc D4l4Fj9JpYgfSAKb8NbAQvV/P1dINo9cElUfZDImdi5bzBGJ7Fvl6qInx+o8cYy8zH4P YC8f3mFup4oQ0/yGL2PUs92s0+La/W9BKGjKEKhVG4LS1cmmM4WVeeWQmY1pWGdtRoWA 5WtKrrraSVTNyA8JVVUb05BWJAXZ+Pe8F61u/GCnMpmHA+/hpoSvoIuCXn+I4zdGUo46 XGIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784828211; x=1785433011; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=3slUHnl834IP76VoxdiRinKrcbFYlUZTVsrXFJZ4JdY=; b=QnD6/FODz9ORHjDnpJnztj72eSrloAoeaR20bRAR4NqdRSvRLabef3e915X5j5/AjJ nrvIZSVri5Gi9irq3TYaVLPY7jfYpgJs/U/TLBOEB4zjCdsGH2HDmh7DLGADSbg1LXc/ q/icelUBSV/TOc1WEC7O8bW5Qy6FSU5pojy78LkMh1CRvDSY2wjiVVp4cw5rQlytKBtZ /SdlDffDQ3b66wVw1sJgOWLcDzBUfH+I7eneuVxTnAeJVObMbTnv4j4C8kQMWXq1qyK1 j55Lxpi+tIkaODZeOVRLkB+rJnakycFOv/Guq92vnzj+pQUEfV9PIFCr6fKlM4u+wWTb FLcQ== X-Forwarded-Encrypted: i=1; AHgh+RrKyX1JIep8ExK9cSd12WCdVUie/rVHpYOtOqwxMQyk6zhqD4jJGaZU5wXTUs0F6IK+CB/k+xZbfP6ia2M=@vger.kernel.org X-Gm-Message-State: AOJu0YzPyQyk4zp/wtECoOyu2HOiIfc/SxCUkVsxG75SCZcqlPvmhJF+ TdY+a9IV5eS091149PDw+L30hZZABqzbSHUeUKGESMX6TnxGySVRwunp X-Gm-Gg: AR+sD105ArCvokTxNVx46KwSLzzJ0ZCCJJE7+ASftu4NL3+SYNDL1HHgNuJhik0R1A2 h6HrVbXGiyZ97zctfJvDBgcJIGwRT9d04raYhKnvsoiqe9lRI+pclcFoVXLfpScU6kaRcgcWnBU k4ANkDvKaOfV5JlDKgrOpcwgBQLLpygdEiHINOr29/HugZfipe2m2mEoJZvw3vsw/mOV+QO9nPC gvHxapRMtwr0pgwO/pJn4cpzRmQeXxf0LvJPNUMvuJRaUm3ac9WujylFbtbJqk8WoOTNbonpKAi YhK3sWy7tAh3BLVfKCuQipONzoHzH50f7BWQQWcGhpILRq2ilTIeWwQC4YUfceNJdHvgPex+/vf sBy8OBriaE6I0gIhPG2S33s6wKNplzUXbSJPEEooGhS00a5fcRd30q5rxjz8PgJMo261wQKQE6h 77yuVtV+xMhqd3SBs8n7Y6YistbLypH74ggjRpZZWT6A0CIJMBzozpx27NY2Y= X-Received: by 2002:a05:6a00:27aa:b0:848:2d1d:836f with SMTP id d2e1a72fcca58-84e2bd6923fmr4526185b3a.28.1784828211456; Thu, 23 Jul 2026 10:36:51 -0700 (PDT) Received: from [192.168.0.160] (c-98-225-44-182.hsd1.wa.comcast.net. [98.225.44.182]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-84e20622dedsm2691612b3a.11.2026.07.23.10.36.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Jul 2026 10:36:51 -0700 (PDT) From: Stanislav Kinsburskii Date: Thu, 23 Jul 2026 10:36:36 -0700 Subject: [PATCH v11 4/8] mshv: Use hmm_range_fault_unlocked_timeout() for region faults Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260723-hmm-v10-v11-4-c55b003a4b61@gmail.com> References: <20260723-hmm-v10-v11-0-c55b003a4b61@gmail.com> In-Reply-To: <20260723-hmm-v10-v11-0-c55b003a4b61@gmail.com> To: Jason Gunthorpe , Leon Romanovsky , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jonathan Corbet , Shuah Khan , Shuah Khan , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Long Li , Lyude Paul , Danilo Krummrich , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Simona Vetter , Min Ma , Lizhi Hou , Oded Gabbay , skinsburskii@gmail.com Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-hyperv@vger.kernel.org, dri-devel@lists.freedesktop.org, nouveau@lists.freedesktop.org, linux-rdma@vger.kernel.org, Jason Gunthorpe X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1784828202; l=3646; i=skinsburskii@gmail.com; s=20260722; h=from:subject:message-id; bh=ZyAhBAfWZePcXjSsL7QqvsrSYU5bhX/lu/bgMYHTJjg=; b=U8sSsRw+6XM+qGDxpQuoe99V7LeZ+M1zXxyPfZetIcbxZBFVX+M/0/c0psrN63mtn0Gyjy1RN 9IByi6+fiCLCWN+9DASYqtkcc69AC9zezCPbmBdSfWzMLLq1QDXujHn X-Developer-Key: i=skinsburskii@gmail.com; a=ed25519; pk=bDpriHBYgeTdkIDweZDCemxsU93neJBOCn3YLIuJpnE= MSHV currently faults movable memory regions by taking mmap_read_lock() around hmm_range_fault(). That prevents the fault path from handling VMAs whose fault handlers need to drop mmap_lock, such as userfaultfd-backed mappings. Use hmm_range_fault_unlocked_timeout() instead. Passing a timeout of 0 preserves MSHV's existing unbounded retry behavior while letting the HMM helper own mmap_lock acquisition and refresh range->notifier_seq internally before walking the range. After the fault succeeds, MSHV still takes mreg_mutex and checks mmu_interval_read_retry() before installing the pages into the region, so the existing invalidation synchronization is preserved. Fold the small fault-and-lock helper into mshv_region_range_fault(), since the remaining retry path is just the standard "fault, take the driver lock, check the interval notifier sequence" pattern. Reviewed-by: Jason Gunthorpe Signed-off-by: Stanislav Kinsburskii --- drivers/hv/mshv_regions.c | 54 +++++++++----------------------------------= ---- 1 file changed, 10 insertions(+), 44 deletions(-) diff --git a/drivers/hv/mshv_regions.c b/drivers/hv/mshv_regions.c index 6d65e5b42152..dddaade31b5d 100644 --- a/drivers/hv/mshv_regions.c +++ b/drivers/hv/mshv_regions.c @@ -381,46 +381,6 @@ int mshv_region_get(struct mshv_mem_region *region) return kref_get_unless_zero(®ion->mreg_refcount); } =20 -/** - * mshv_region_hmm_fault_and_lock - Handle HMM faults and lock the memory = region - * @region: Pointer to the memory region structure - * @range: Pointer to the HMM range structure - * - * This function performs the following steps: - * 1. Reads the notifier sequence for the HMM range. - * 2. Acquires a read lock on the memory map. - * 3. Handles HMM faults for the specified range. - * 4. Releases the read lock on the memory map. - * 5. If successful, locks the memory region mutex. - * 6. Verifies if the notifier sequence has changed during the operation. - * If it has, releases the mutex and returns -EBUSY to match with - * hmm_range_fault() return code for repeating. - * - * Return: 0 on success, a negative error code otherwise. - */ -static int mshv_region_hmm_fault_and_lock(struct mshv_mem_region *region, - struct hmm_range *range) -{ - int ret; - - range->notifier_seq =3D mmu_interval_read_begin(range->notifier); - mmap_read_lock(region->mreg_mni.mm); - ret =3D hmm_range_fault(range); - mmap_read_unlock(region->mreg_mni.mm); - if (ret) - return ret; - - mutex_lock(®ion->mreg_mutex); - - if (mmu_interval_read_retry(range->notifier, range->notifier_seq)) { - mutex_unlock(®ion->mreg_mutex); - cond_resched(); - return -EBUSY; - } - - return 0; -} - /** * mshv_region_range_fault - Handle memory range faults for a given region. * @region: Pointer to the memory region structure. @@ -452,13 +412,19 @@ static int mshv_region_range_fault(struct mshv_mem_re= gion *region, range.start =3D region->start_uaddr + page_offset * HV_HYP_PAGE_SIZE; range.end =3D range.start + page_count * HV_HYP_PAGE_SIZE; =20 - do { - ret =3D mshv_region_hmm_fault_and_lock(region, &range); - } while (ret =3D=3D -EBUSY); - +again: + ret =3D hmm_range_fault_unlocked_timeout(&range, 0); if (ret) goto out; =20 + mutex_lock(®ion->mreg_mutex); + + if (mmu_interval_read_retry(range.notifier, range.notifier_seq)) { + mutex_unlock(®ion->mreg_mutex); + cond_resched(); + goto again; + } + for (i =3D 0; i < page_count; i++) region->mreg_pages[page_offset + i] =3D hmm_pfn_to_page(pfns[i]); =20 --=20 2.43.0 From nobody Fri Jul 24 04:53:54 2026 Received: from mail-pf1-f171.google.com (mail-pf1-f171.google.com [209.85.210.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B124547D474 for ; Thu, 23 Jul 2026 17:36:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784828222; cv=none; b=KAf8IApNWI3rsw26hRFEJtFy52N2b/hkuolIKs15xR1R3FQZIaJ6CFpSNpgvS0mwGQ6+8tVgZ4JKWI1XKRlTsHcd7bnOna2mEKCXygGDre/V0OLhfKBuQt50Nv2Yev9N1/rtqD64uUCZtaiMlOTdiwxHCfbHMTTUCMz2MWKuu04= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784828222; c=relaxed/simple; bh=KaVy0LDD2WZ56zIlG4jDVljfxrz2j27JL0AnOSACkpk=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=bY/qGlv/JuGkq1Qvf8o6P04W1nn1rvyVXZCZdYRU7K+9aOcGcfFXo8/ZaQ7Wd1AwPbtmADZO+q9y9xtuLwriGs4I//t9FRfetJ8b6VttIvLlmcWn1ybdA/klhXFeP+uHd/iLQgL73QxzM0UQ2g7I0XZsaA/+59+KmDSOzIChPK4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=F5I5dTg9; arc=none smtp.client-ip=209.85.210.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="F5I5dTg9" Received: by mail-pf1-f171.google.com with SMTP id d2e1a72fcca58-848595b338cso1031768b3a.0 for ; Thu, 23 Jul 2026 10:36:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1784828213; x=1785433013; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=WVIK4CTS2ebWqFuX16f+6TzH9lK4l3CZHr+WHLsMqrs=; b=F5I5dTg9VBzF8IrLEaasAUXZbc35tqzwAdoCUQ+NS+RPMYG4ODlhVr8S+6B/UuWGme 5nCKQSaPomN4EIrV/rrHdA4D3MkjfWEDRdyrsclWwLhfDPIRXUVSsAxe3lrjbkWlC6Wk BFMwS7j5ueHyF+Fhk3FNrDSgn2jSv+P3bE/+VgkKRpme6Zk/lOCjvB9UBmt684//KJmj Mti36tLu0CGwY65yJWBr0wu7SJonL83xRNavFj3yrK/7rnlAbNNGMD8VpGeX7fUQSJgo QUbVnGlGAiMX0eI8uW+iDZL7L91QE/E48bsOvhKI7fvifHhKAGII41iYSw2p6v6APLyT 5CGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784828213; x=1785433013; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=WVIK4CTS2ebWqFuX16f+6TzH9lK4l3CZHr+WHLsMqrs=; b=RPkrqMorgehRqFe9kpsCiTEbgGKSfq+81TtLxQYLSfPVtsIwYNzakoDhuHC6HPMbce FqmpFKQaLc6r7mxJ+9DJp2/AlajsNWiLF3SP77LKXY71x2p3lRa0hbxtJYKunxxs6/Hw 87AJz5BpDgsmhr4JRUD0coCL9gt69VJEdzMmjUTw1mwzYMU/v0XB0Dn1neneqSL/0sLW O/mEf9Ss+p1emcOE7RybDiJGyLM2MCmtExuULl+mVfCqQqppRaY5dbCEhOOssTS89uLC 0J9g/HDtcvheOXN7NDU0CtMyj8f0Apxb0+AONtkw4yZe0XmfRmLqRrcfnWMUsRJzwmWB TaCA== X-Forwarded-Encrypted: i=1; AHgh+RoqcEpRVpqYP538c6MqC8bhg5/llfcu1f56oQNsclxfbWJfTFWehX6VvT+C2ImkbLHk6+8P+Rnj0CUkyM0=@vger.kernel.org X-Gm-Message-State: AOJu0Yxu+GbsOBW64vW0y1XgAZu7YCqB1cRe5R035O74CyfE7W4GNRyn Tt3gSmBN/vGvuGL2msJbecGRk67nb43Iatsy/+EZlLNNA2WCZBIELgNF X-Gm-Gg: AR+sD10zzmjfuAAJQ6xmVsUXpzDpMObxPl9/CGWTZ2QI2NR0rRo08ieymAeBo8Luamg yb0WqIThesuHFO0RCach7ET5DyW/Yhq/dJIBI1q2fJbnBrnrHcTS+tTElMFuV7JRD9CzjwjfGHv vB1iozeBZyUtq9rBeFcUkILaLbR9eRvUoM6ZX1by0gE5HZM3Mt2N6gufc4VRK2obAPr/tZTtgdM z68dGfSok22/GEp4ffH1bdYcxn8gy3uxivNjW4bRzjqHSw/5WeI/nEdFyd4GXgbH2ptzQD4zq2y aDahdEybMSygQytqiryeQDdGIwDWpKfsGI0e95NioFxKW0SEJWI04DZzHi65yeFh5YML7icnPXO aVZzmN2WvFvRNeXQwOlYMC9s/eVdNWMexTIBW2wOp04ixf4Pdq+xsAr1oZ8QmD6YD5h6fR7vmtp IkCFIczXEDcTwq3kzk7Wo4yLSE6UlmAQuzpqhCCEm6JuWGJxqa X-Received: by 2002:a05:6a00:278a:b0:848:42d0:bc91 with SMTP id d2e1a72fcca58-84e2b7f4d38mr4894542b3a.12.1784828213088; Thu, 23 Jul 2026 10:36:53 -0700 (PDT) Received: from [192.168.0.160] (c-98-225-44-182.hsd1.wa.comcast.net. [98.225.44.182]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-84e20622dedsm2691612b3a.11.2026.07.23.10.36.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Jul 2026 10:36:52 -0700 (PDT) From: Stanislav Kinsburskii Date: Thu, 23 Jul 2026 10:36:37 -0700 Subject: [PATCH v11 5/8] drm/nouveau: Use hmm_range_fault_unlocked_timeout() for SVM faults Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260723-hmm-v10-v11-5-c55b003a4b61@gmail.com> References: <20260723-hmm-v10-v11-0-c55b003a4b61@gmail.com> In-Reply-To: <20260723-hmm-v10-v11-0-c55b003a4b61@gmail.com> To: Jason Gunthorpe , Leon Romanovsky , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jonathan Corbet , Shuah Khan , Shuah Khan , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Long Li , Lyude Paul , Danilo Krummrich , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Simona Vetter , Min Ma , Lizhi Hou , Oded Gabbay , skinsburskii@gmail.com Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-hyperv@vger.kernel.org, dri-devel@lists.freedesktop.org, nouveau@lists.freedesktop.org, linux-rdma@vger.kernel.org, Jason Gunthorpe X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1784828202; l=2163; i=skinsburskii@gmail.com; s=20260722; h=from:subject:message-id; bh=KaVy0LDD2WZ56zIlG4jDVljfxrz2j27JL0AnOSACkpk=; b=KV2nZSPyVg1Dt5uSSiSffiKfkGgb7W3sWOI6CSqvJIxSZb1CjTMqYYugO/lEA7iz8WZFfaBr6 LbFOvRd/oG3AQDmUr73ZTQ/tVrKY3HQM715K46T+2IVDrVm2X1nPXe2 X-Developer-Key: i=skinsburskii@gmail.com; a=ed25519; pk=bDpriHBYgeTdkIDweZDCemxsU93neJBOCn3YLIuJpnE= nouveau_range_fault() takes mmap_read_lock() only to call hmm_range_fault(). It also keeps a single HMM_RANGE_DEFAULT_TIMEOUT deadline across both HMM -EBUSY retries and post-fault mmu_interval_read_retry() retries. Use hmm_range_fault_unlocked_timeout() instead. The HMM helper now owns the mmap lock and refreshes range->notifier_seq for its internal retries. Nouveau keeps its existing absolute deadline in the outer loop and passes the remaining jiffies to the helper for each fault attempt, so retries caused by mmu_interval_read_retry() do not reset the overall retry budget. Nouveau still validates the interval notifier sequence while holding svmm->mutex before programming the GPU mapping. Reviewed-by: Jason Gunthorpe Signed-off-by: Stanislav Kinsburskii --- drivers/gpu/drm/nouveau/nouveau_svm.c | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/drivers/gpu/drm/nouveau/nouveau_svm.c b/drivers/gpu/drm/nouvea= u/nouveau_svm.c index dcc92131488e..58735446d783 100644 --- a/drivers/gpu/drm/nouveau/nouveau_svm.c +++ b/drivers/gpu/drm/nouveau/nouveau_svm.c @@ -678,20 +678,22 @@ static int nouveau_range_fault(struct nouveau_svmm *s= vmm, range.end =3D notifier->notifier.interval_tree.last + 1; =20 while (true) { - if (time_after(jiffies, timeout)) { + long remaining =3D timeout - jiffies; + + /* + * The HMM timeout only bounds retries while HMM is walking and + * faulting the range. This fault is handled by a kernel worker, + * so fatal signals from the faulting process cannot stop an + * endless stream of invalidations here. + */ + if (time_after_eq(jiffies, timeout)) { ret =3D -EBUSY; goto out; } =20 - range.notifier_seq =3D mmu_interval_read_begin(range.notifier); - mmap_read_lock(mm); - ret =3D hmm_range_fault(&range); - mmap_read_unlock(mm); - if (ret) { - if (ret =3D=3D -EBUSY) - continue; + ret =3D hmm_range_fault_unlocked_timeout(&range, remaining); + if (ret) goto out; - } =20 mutex_lock(&svmm->mutex); if (mmu_interval_read_retry(range.notifier, --=20 2.43.0 From nobody Fri Jul 24 04:53:54 2026 Received: from mail-pf1-f176.google.com (mail-pf1-f176.google.com [209.85.210.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 776D33D3D16 for ; Thu, 23 Jul 2026 17:36:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784828223; cv=none; b=oUd4xqF11Jtlmu0v+6BykcCHW0mxgqLz2j6IdLZKm1aINIVlA8dMoUUZ/IpJsxhtgbL/0tq0USaIMsZW+tXP+AYuTSd9MTZMZjPyly83T8vVi/EXZbUzdgmAh198OUaaI/E56WyYihjwI3kHxhothjfM616dPdvztYZaUhF0PRg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784828223; c=relaxed/simple; bh=wCDSwsZ/5jgwWzEqFja4q7w3SxWPNbitK21qCohy6H8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=X/1224FuP5XrMnHo6lR6ctDQUU2D5J4bWExkNh691BZzSITQib94d6mJeXKNjlo7Co4DDMlOwRNfs/Jbq7cqwycbhLKi5ks6Q2kq80lzcevhulUqB23pNY1VrfmZOzfj318GqYCFPsvKcy6CqAI95pTXWcCQd3h6p8qMjKSwKLw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=jzoH7ryx; arc=none smtp.client-ip=209.85.210.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="jzoH7ryx" Received: by mail-pf1-f176.google.com with SMTP id d2e1a72fcca58-8453427d3f4so944310b3a.3 for ; Thu, 23 Jul 2026 10:36:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1784828214; x=1785433014; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=Hwr2PA6k/yzOHBkhyRcV60oQVyvCsFMHOEQ+QEbH+Ag=; b=jzoH7ryxgSw+y1bE/RYZqr6Afgy4TpkGNtQTGolNvAGj73SbggEjTkbqeXKAAC0bwa 4IUqcI6QLz7xVuCtPTYe7yXdTMbbRghVyZ411PH+ypOPC65riMLpSiIiZ5UpvsFuYs4U YqN2rgS7wKE6eoocuuu+CF65r3Oe3mUQFrFOMEdjlGQB4sLstr7/0ow7JfsVKNKCZ1C3 puIE293rl6nIrOUlUN4gKti7+6ays49hwz+eWaISQi5Wj+3NtPmeJ5Js8XTNU3L0wLbJ 4RN4DeSZpqlSsEfGOx/6IAeIN9yYchuCXgsDbteNoKuPtFrx0TCxK9qwiccpRkxSvdxs dm0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784828214; x=1785433014; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=Hwr2PA6k/yzOHBkhyRcV60oQVyvCsFMHOEQ+QEbH+Ag=; b=ZoTzdSP6orvp4A0RKj7TdwkMPebx82f/BP1Q//jTZyQmagZN+BLRiSIaddgK6Q6YL7 yzMyJe20opTKUcZShF+qUhfypEj0JRowi13/mQ6M73Zv8n++laukYpf+GCd0BDrEI7I8 esI57tu8TV3JZmhRdGzJ+bAn+RGm/vvVZmAAFOLafkK6C0a/q/ia0MbtfRigtNn2WbWb EoCsQCL+mD36iVY9JUFNtA8pP/wqdG1HZCCV9rINdqune1mn0vISVQbpEyaLRfd/+PBL PbXF44e/QeqqByi1cxaLAVoToo6UZIBlGyU4YNAxRTkBR5XqeUCSc5GN61lSKxYuFFxa v3XA== X-Forwarded-Encrypted: i=1; AHgh+RoG+hxdd2IiGtvFc1Z5eGjnW/ZoB8RQpOn+fl7366xKRSuJKeOjIGORRkSwT1FFfyOEdkWjJ0JrqO9n/Ws=@vger.kernel.org X-Gm-Message-State: AOJu0YxNk8ama5u0EM9Hum+T6QTzEd+bpYqYGOAIcOf3Mty/Tn9jTffv Om2/twGmr27TjLqmZflneRGKU7U5jUtOT9ByNABOZfHPscsYbh3YXouI X-Gm-Gg: AR+sD11toZOErm1HpeJ6P7+eQKCO82WQ9Ya8fY1ufBO0f0a2lssUaTGVkqz2fk2D87V FFD6z3dTH3pYfH6gUG+y6YVcserr2OCoVZtrVmvU5h5JB593YlN+1RNkqh6svFHzltl45KROxwG NtbQKvJWXkPQ+Euz9y86cg0Nux23i1c5lepchFo56HCoSTYNMb6oqRZP/bQDDK2A0Pm4TYToZXc 3NGqqIU+To9QgA0WIAwFwFkup908JpMQ3s64jnQ3VuDqDdtFX4aPjfuY0rJslEkqxFpfJ0tB+Oe dTy6Wouo9H3apaxnXeZq95hgqsOWLSObgHUz68DGVmeP2H/n3COuqvB1+K7sQo/v+A24DQZ3ydW A5maiqPUq4WPR7j8s0RIjDu4H1KyzafeVDCRVUZVQ/Nr+ngDUFgOCSv1ejzt+8454lY/TK/a7z9 coKSLbp+bjDveIAjDVWkLziSwYsXfC45pmvVzzPme1tdMPEk3x X-Received: by 2002:a05:6a00:e88:b0:847:b16b:46d9 with SMTP id d2e1a72fcca58-84e2b8aa77bmr4617793b3a.34.1784828214639; Thu, 23 Jul 2026 10:36:54 -0700 (PDT) Received: from [192.168.0.160] (c-98-225-44-182.hsd1.wa.comcast.net. [98.225.44.182]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-84e20622dedsm2691612b3a.11.2026.07.23.10.36.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Jul 2026 10:36:54 -0700 (PDT) From: Stanislav Kinsburskii Date: Thu, 23 Jul 2026 10:36:38 -0700 Subject: [PATCH v11 6/8] RDMA/umem: Use hmm_range_fault_unlocked_timeout() for ODP faults Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260723-hmm-v10-v11-6-c55b003a4b61@gmail.com> References: <20260723-hmm-v10-v11-0-c55b003a4b61@gmail.com> In-Reply-To: <20260723-hmm-v10-v11-0-c55b003a4b61@gmail.com> To: Jason Gunthorpe , Leon Romanovsky , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jonathan Corbet , Shuah Khan , Shuah Khan , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Long Li , Lyude Paul , Danilo Krummrich , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Simona Vetter , Min Ma , Lizhi Hou , Oded Gabbay , skinsburskii@gmail.com Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-hyperv@vger.kernel.org, dri-devel@lists.freedesktop.org, nouveau@lists.freedesktop.org, linux-rdma@vger.kernel.org, Jason Gunthorpe X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1784828202; l=2457; i=skinsburskii@gmail.com; s=20260722; h=from:subject:message-id; bh=wCDSwsZ/5jgwWzEqFja4q7w3SxWPNbitK21qCohy6H8=; b=gezzOLTGQ+6T3Wy5w3nFtel93rrT2Kd9eeZ0n6f5FEQw8Beb0IrNif+mDyjnRhH39tdmJbOsG yXEZFknqYwdCOt8awUKVwHYE2B6xTW+3MpwtLFeYQDVzLwrWkfuTvq6 X-Developer-Key: i=skinsburskii@gmail.com; a=ed25519; pk=bDpriHBYgeTdkIDweZDCemxsU93neJBOCn3YLIuJpnE= ib_umem_odp_map_dma_and_lock() takes mmap_read_lock() only around hmm_range_fault(), then retries -EBUSY until HMM_RANGE_DEFAULT_TIMEOUT expires. Use hmm_range_fault_unlocked_timeout() instead. The HMM helper now owns the mmap lock and refreshes range->notifier_seq for its internal retries. ODP keeps using HMM_RANGE_DEFAULT_TIMEOUT for each HMM fault attempt, while interval invalidation retries continue to be handled by the existing outer loop. ODP still validates the interval notifier sequence while holding umem_mutex before DMA mapping pages. Reviewed-by: Jason Gunthorpe Signed-off-by: Stanislav Kinsburskii --- drivers/infiniband/core/umem_odp.c | 18 +++++------------- 1 file changed, 5 insertions(+), 13 deletions(-) diff --git a/drivers/infiniband/core/umem_odp.c b/drivers/infiniband/core/u= mem_odp.c index 404fa1cc3254..9cc21cd762d9 100644 --- a/drivers/infiniband/core/umem_odp.c +++ b/drivers/infiniband/core/umem_odp.c @@ -329,7 +329,7 @@ int ib_umem_odp_map_dma_and_lock(struct ib_umem_odp *um= em_odp, u64 user_virt, struct mm_struct *owning_mm =3D umem_odp->umem.owning_mm; int pfn_index, dma_index, ret =3D 0, start_idx; unsigned int page_shift, hmm_order, pfn_start_idx; - unsigned long num_pfns, current_seq; + unsigned long num_pfns; struct hmm_range range =3D {}; unsigned long timeout; =20 @@ -363,26 +363,18 @@ int ib_umem_odp_map_dma_and_lock(struct ib_umem_odp *= umem_odp, u64 user_virt, } =20 range.hmm_pfns =3D &(umem_odp->map.pfn_list[pfn_start_idx]); - timeout =3D jiffies + msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT); + timeout =3D msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT); =20 retry: - current_seq =3D range.notifier_seq =3D - mmu_interval_read_begin(&umem_odp->notifier); - - mmap_read_lock(owning_mm); - ret =3D hmm_range_fault(&range); - mmap_read_unlock(owning_mm); - if (unlikely(ret)) { - if (ret =3D=3D -EBUSY && !time_after(jiffies, timeout)) - goto retry; + ret =3D hmm_range_fault_unlocked_timeout(&range, timeout); + if (unlikely(ret)) goto out_put_mm; - } =20 start_idx =3D (range.start - ib_umem_start(umem_odp)) >> page_shift; dma_index =3D start_idx; =20 mutex_lock(&umem_odp->umem_mutex); - if (mmu_interval_read_retry(&umem_odp->notifier, current_seq)) { + if (mmu_interval_read_retry(&umem_odp->notifier, range.notifier_seq)) { mutex_unlock(&umem_odp->umem_mutex); goto retry; } --=20 2.43.0 From nobody Fri Jul 24 04:53:54 2026 Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DCD0B485CC2 for ; Thu, 23 Jul 2026 17:37:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784828229; cv=none; b=DijMS8/SvrI9WiQBU3uReMsHZ+Tl6jwAHpjgjG7FORiDJx3O1F89xXlOl7s18ig6zxXVy5a+XZ1ZxzkQHKiynWXS6CXh9zH1lx5wR1JOOwpH72bkYZhzYA8ibP2dLs/r+CB9hSczyWI5Qlvn4iJsGzHNgqtFV7mhzSv0VC5/LP8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784828229; c=relaxed/simple; bh=Js9Z5fScNTLs9yZDEPq5MMQDWBFdccrrYdkvKMfBR0c=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=AkwG5dk7RzgfujN/g//kq0l9L2bBe2X/FAU6o7CJfodXLj+glpmCCr9+hF5Lv4MB96DJLN5FepE/5j98iGZuU7se5A05glt+cXue+LVpMKeEYcFhsUk7hbNZ6rPa3XZ1JUmPRl90jWAyYTuUxnKEClfkdFSKevTV7o/tCBL2wPk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=BKL6hJTS; arc=none smtp.client-ip=209.85.210.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="BKL6hJTS" Received: by mail-pf1-f181.google.com with SMTP id d2e1a72fcca58-84a4d8fd6ecso981446b3a.1 for ; Thu, 23 Jul 2026 10:37:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1784828216; x=1785433016; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=/3EHZQlaXq9sLmhVgtuP5ODA2wxc36yvs54yemcncOc=; b=BKL6hJTSZk19OoDFcA63/ZRoV0prsj8yCa4ogkU90ZAlAakTfXuIGGS/iowNK20Shm 48LLdV1VLcRWB+urPSsdyN0SvDg3d5TRjBBZ91VHO9dGcLM7NP5U5NMulOAal5hJ07Q4 J1ds7S3tCtGMQ0nJS4JKg2CXAmTMXTAXD90ceLfgLW81TiwyrV1T0V6qmP7F3+23sGKK ystiPsEBmqOhU53eERSPQWhWZYpis1kHPTqaTAM2/8tKMKtDgkNi4GGJGu6EhArOMTt9 X2otbFigngyTor75rMG2t8eHnCm9TQHuyZEGyW2NqF+eQpSl9GlrB927etwZYxmjpij1 XyUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784828216; x=1785433016; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=/3EHZQlaXq9sLmhVgtuP5ODA2wxc36yvs54yemcncOc=; b=EEaXyHENyZLUCLzhQVc/tyTXe+v4S+iKiCGAKscTCApolvifwB5RK7kzChBhoirfEa IqPvjwQwIklDjYIIAGiMlXlBriwDbd2eEN+ShztXMxb84Xx+vem50XmP9Rrz8SnZkIBD 61mXjqC/Sszgg1mvxQYk7h/CXanQprlKgM3MhbINkScB/zZuysFnSKd/wghDyTKXNbgT ls4QsBfbC6T7FxEI+/t/OtTCsyZ5eAdYBCgw2ikcB8DXobnMI/or4jexquA74+ZAalYW NM19ZBusUrBQeWf5i7RReMXHnqE9GbmeKty1OiGRuMzMDzYNuqCPXPhNIjI79VvbhzIo wWoA== X-Forwarded-Encrypted: i=1; AHgh+RqSGW8v6Og6spf5LYfr1jVCRAK9YrMWGhzsVu3xg2LxZXy4drSxtb2F7CwCdqoFsxxcYu7zK+lu/de5whQ=@vger.kernel.org X-Gm-Message-State: AOJu0Ywb7mwMwcevwZZvAuIXjyXWfA5AbJxIOTugnh+3BnodLXmLlYhA R8EUdzKrvxnWNLsrY4xfF6C7Fr29Mk/564QX46N2HvmQNvjS1UicgoEt X-Gm-Gg: AR+sD13kMr7dwxFiSdbupxilfJbtckljbKldlcKodbyEqo8StAJiJgAct2GYvbCdCJl 8CVt7vbZLS7qld6Igt1D0GEmkYz10XrOUa5Rwx7wxYrC9axXEk5PW006HO5SqFZ6goYJrBemwf4 dKXRRmEbuDRGL9WWmhmBOhTdBzuG3R68W+zcJLLphn0ZJKUs5U35q71wAFeshmG1ip2QE4luXpw R0ffmRfp3/UMomeEihg//lsH4DUWzbLZSHjF89GLYjumSWA9YMZmoriL1kIvQCPDddcHkwhspfs yWZ/S83AW87hCIMO7EjsOwWerp1azR9W1T8IYKRGGEgAF2/tCWMVR6OFwP6XRoXta7MQneAznX3 +3eR8V8DDFV1uH7otHTuBYdZ9BdbHa20J9AYch4d6wzbo6cC5Bjd/QykSfh80wXEc6BUOSw9uZC PUSxCFr+qPL7Az/3q+zVSBLbK2R+0c+u9Zy4+bxuLJ3t6zaN98 X-Received: by 2002:a05:6a00:4482:b0:848:47d3:47ec with SMTP id d2e1a72fcca58-84e2b8ab6d2mr4797080b3a.31.1784828216068; Thu, 23 Jul 2026 10:36:56 -0700 (PDT) Received: from [192.168.0.160] (c-98-225-44-182.hsd1.wa.comcast.net. [98.225.44.182]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-84e20622dedsm2691612b3a.11.2026.07.23.10.36.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Jul 2026 10:36:55 -0700 (PDT) From: Stanislav Kinsburskii Date: Thu, 23 Jul 2026 10:36:39 -0700 Subject: [PATCH v11 7/8] accel/amdxdna: Use hmm_range_fault_unlocked_timeout() for range population Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260723-hmm-v10-v11-7-c55b003a4b61@gmail.com> References: <20260723-hmm-v10-v11-0-c55b003a4b61@gmail.com> In-Reply-To: <20260723-hmm-v10-v11-0-c55b003a4b61@gmail.com> To: Jason Gunthorpe , Leon Romanovsky , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jonathan Corbet , Shuah Khan , Shuah Khan , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Long Li , Lyude Paul , Danilo Krummrich , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Simona Vetter , Min Ma , Lizhi Hou , Oded Gabbay , skinsburskii@gmail.com Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-hyperv@vger.kernel.org, dri-devel@lists.freedesktop.org, nouveau@lists.freedesktop.org, linux-rdma@vger.kernel.org, Jason Gunthorpe X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1784828202; l=2650; i=skinsburskii@gmail.com; s=20260722; h=from:subject:message-id; bh=Js9Z5fScNTLs9yZDEPq5MMQDWBFdccrrYdkvKMfBR0c=; b=UoKX2QEi16TQG/XSPiXepu/KONklG+8l1kn86LgWhqxQQe32BnNrpIu04JJYOeap8yhmhRNnI Xgchah8NIfnDwgLdxf/plAUECr/N2GDtTgg00z0Czhp7vw9hyXoBoqQ X-Developer-Key: i=skinsburskii@gmail.com; a=ed25519; pk=bDpriHBYgeTdkIDweZDCemxsU93neJBOCn3YLIuJpnE= aie2_populate_range() takes mmap_read_lock() only around hmm_range_fault(). It also open-codes the mmu interval sequence setup before each HMM walk and retries -EBUSY until HMM_RANGE_DEFAULT_TIMEOUT expires. Use hmm_range_fault_unlocked_timeout() instead. The HMM helper now owns the mmap lock and refreshes mapp->range.notifier_seq for its internal retries, so the driver only needs to call the helper and then validate the sequence before marking the mapping populated. Pass HMM_RANGE_DEFAULT_TIMEOUT as the helper retry budget for each HMM population attempt. This scopes the timeout to repeated HMM notifier retries while preserving the existing outer loop that moves between invalid mappings and restarts when the interval is invalidated before the driver updates its mapping state. Keep returning -ETIME when the HMM retry budget expires, matching the driver's existing timeout error convention. Reviewed-by: Jason Gunthorpe Signed-off-by: Stanislav Kinsburskii --- drivers/accel/amdxdna/aie2_ctx.c | 23 ++++------------------- 1 file changed, 4 insertions(+), 19 deletions(-) diff --git a/drivers/accel/amdxdna/aie2_ctx.c b/drivers/accel/amdxdna/aie2_= ctx.c index 54486960cbf5..b5b4ca263002 100644 --- a/drivers/accel/amdxdna/aie2_ctx.c +++ b/drivers/accel/amdxdna/aie2_ctx.c @@ -1034,7 +1034,7 @@ static int aie2_populate_range(struct amdxdna_gem_obj= *abo) bool found; int ret; =20 - timeout =3D jiffies + msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT); + timeout =3D msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT); again: found =3D false; down_write(&xdna->notifier_lock); @@ -1061,24 +1061,9 @@ static int aie2_populate_range(struct amdxdna_gem_ob= j *abo) return -EFAULT; } =20 - mapp->range.notifier_seq =3D mmu_interval_read_begin(&mapp->notifier); - mmap_read_lock(mm); - ret =3D hmm_range_fault(&mapp->range); - mmap_read_unlock(mm); - if (ret) { - if (time_after(jiffies, timeout)) { - ret =3D -ETIME; - goto put_mm; - } - - if (ret =3D=3D -EBUSY) { - amdxdna_umap_put(mapp); - mmput(mm); - goto again; - } - + ret =3D hmm_range_fault_unlocked_timeout(&mapp->range, timeout); + if (ret) goto put_mm; - } =20 down_write(&xdna->notifier_lock); if (mmu_interval_read_retry(&mapp->notifier, mapp->range.notifier_seq)) { @@ -1096,7 +1081,7 @@ static int aie2_populate_range(struct amdxdna_gem_obj= *abo) put_mm: amdxdna_umap_put(mapp); mmput(mm); - return ret; + return ret =3D=3D -EBUSY ? -ETIME : ret; } =20 int aie2_cmd_submit(struct amdxdna_hwctx *hwctx, struct amdxdna_sched_job = *job, u64 *seq) --=20 2.43.0 From nobody Fri Jul 24 04:53:54 2026 Received: from mail-pf1-f169.google.com (mail-pf1-f169.google.com [209.85.210.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6E8604766B4 for ; Thu, 23 Jul 2026 17:37:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784828226; cv=none; b=K2ioVEzdwFvjfgSOAuhF7GMaSoO0NHjqWQRX8QE3eOXNlPgPhS7g6mmN1HPEl2G7XKyMoUJUXlGTfcQ1xFo/lmWZVmAOzX1EuQIW+Kgsesdr5SUNuNdmlAjztGxQtDATTOhd52XfpjSjRJFdIgInTlb6fCHo6j4aaPoRB/rXL6g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784828226; c=relaxed/simple; bh=OHRNaT5JjbQYbg0HfVsh/goi/Wp2JeVv837tKjXheiU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=FrETS7H8dmF31lo8yej3qHZ2HlFo7F+n1W3jyo5eZmf96QW/zf2TvsOLARoXnpq/oOtKaO8F9UIGrqbFpSGLjfGhmYXsvZbanB+eY7+woTxxKClGgMyTs2MnDXfumwn3bBQiiaEYKJD8S6hbXMZi9t4SPIamc6Ar24v5cLMjnFI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=WSXNC98m; arc=none smtp.client-ip=209.85.210.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="WSXNC98m" Received: by mail-pf1-f169.google.com with SMTP id d2e1a72fcca58-84a4d8fd6ecso981481b3a.1 for ; Thu, 23 Jul 2026 10:36:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1784828218; x=1785433018; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=i8Qd33SMLkRtmqANiDW8AEhl0t9s2Dut7/75bk+4IdQ=; b=WSXNC98mn9E0uZv6c2hSBxaJ5THbotrgF8Z9RkvDYuq4Q4evsHpW4g3mqJYQWggIpf zt0Gp0677ef4lE+UEuWL6cq09YFhfDkld1spJvjM/x2tEbA5n+/fYqZSj6/fh9IXcCTu ixmXyltpDNi/NOEf9arwP00oJMN8zMDDDnQRimdOkX2p4kTQDi8PjYqwaCtc9DgMkCxx c64FSyVvbcVHAJtVXLCEExjtElC0ViZjOER1MjCb/lF7xwMYJ8wnYTOeuoqaDZ/Lyq/E BIdjNu3rl7Vd3ly213tz3cfTC31gvKlDq9z/0YF6dMEzjdJYXBn35fUWfRiuLEwR4v92 Xr2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784828218; x=1785433018; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=i8Qd33SMLkRtmqANiDW8AEhl0t9s2Dut7/75bk+4IdQ=; b=kT2+WNRfHLVA3RFccGNVh3oOdueGXRifYtqQLQqBpzdWgJOK4ZCOK6wkDBC0jcdkUB DW40A2bjMWQF7merYCVSkTyKZxgJpsbtE47+fJhO95IXI8gF0b+qKekdZEOa1uhGsxGG tEesOCCwi09cIWQREtZpQLtBZvnBeO/HznyV9jaKbVo3trUvOEK6joAIJU2P8KzVwUE+ /4/dCIrRnaFvyIDm0sjTZwhiPDRWNVmXWeZs8ELuHHirFlTys6zlyKJv45oVaM0iCxd5 ZRbBuEQsG4ZYMtFapSDxyveVntGfIs2xrOB6/m0ag6jN2Leia2jO0d04khj/IaEurpzt H7pQ== X-Forwarded-Encrypted: i=1; AHgh+Rpn4vv+srpWx46qx/spIA/0Dq3FMvSZzCa7SE/2xkzSR1aNbuprcscTWhSsxjMUuBQc+rt0NZhWHh3ppyE=@vger.kernel.org X-Gm-Message-State: AOJu0YyDP8pub2VXMuz1Xx6yGj32BOozXNvTyeaDVBm5CvoU4nROP8s4 rZ5vEXrG7z1ps48LF7Vj2z78sy0W7K+XhNXSQoNWSukfFsjaXY+oDZvt X-Gm-Gg: AR+sD124O3/xUk93x/+KBpWYmiMSoBXgfeARkjJXOXUb78gOo5xYtm2junlkaAJ20rk l+2DJ4d8UtFRfeC4VenciDnSMZ74+0c4/V8La+v52qg20YLj7zhchOKhFhvKIGOA/m4kSUc+N8o BKenwLFDFQ0pv5JO+Umcj7DylfVh1+02tjslPhMfPoChepLKNKBbY+vGUfqg0iIiSBAYFnvp8Ax AsQ9WEfUEW2zI2kQAq8wQj1yhnz6tC9QWpS4CPKq0oImYkciP6ak5a93Jje++pYJu4Cno6X59od rP3Jj0UODNKNe1JbEfSnGKJuqNNSLIwpYcDpDeCMf1Vn2BmZp9gT7qqCyocCezhTjXSKyVdDBET YSJIf5ySND1V3j2lkdunbKPvpFS2LNrRcGgbfVfAfe4+ZFTh8k3+wY+tUEGAphhoG5GWocJazXM ot+8lpbNjQvOt1fxyA3zhStDoRVJQWFU1c/HbR4jOeD+tnZxmU X-Received: by 2002:a05:6a00:cd3:b0:848:88e6:95d3 with SMTP id d2e1a72fcca58-84e2bc4d259mr4728321b3a.72.1784828217972; Thu, 23 Jul 2026 10:36:57 -0700 (PDT) Received: from [192.168.0.160] (c-98-225-44-182.hsd1.wa.comcast.net. [98.225.44.182]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-84e20622dedsm2691612b3a.11.2026.07.23.10.36.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Jul 2026 10:36:57 -0700 (PDT) From: Stanislav Kinsburskii Date: Thu, 23 Jul 2026 10:36:40 -0700 Subject: [PATCH v11 8/8] drm/gpusvm: Use hmm_range_fault_unlocked_timeout() for range faults Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260723-hmm-v10-v11-8-c55b003a4b61@gmail.com> References: <20260723-hmm-v10-v11-0-c55b003a4b61@gmail.com> In-Reply-To: <20260723-hmm-v10-v11-0-c55b003a4b61@gmail.com> To: Jason Gunthorpe , Leon Romanovsky , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jonathan Corbet , Shuah Khan , Shuah Khan , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Long Li , Lyude Paul , Danilo Krummrich , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Simona Vetter , Min Ma , Lizhi Hou , Oded Gabbay , skinsburskii@gmail.com Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-hyperv@vger.kernel.org, dri-devel@lists.freedesktop.org, nouveau@lists.freedesktop.org, linux-rdma@vger.kernel.org, Jason Gunthorpe X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1784828202; l=4921; i=skinsburskii@gmail.com; s=20260722; h=from:subject:message-id; bh=OHRNaT5JjbQYbg0HfVsh/goi/Wp2JeVv837tKjXheiU=; b=3mS7hpEm1cC4EjlfjSHzs08itt/haPD5UmH5YZzhkX5wXqdTOzSMnls0rW++T0vSS8hxNEDOH H/0vN9YPqS8C3NJp8Qfk40j/53Cv5CNaQqrwQS1NX9qK2UYGBofyGE6 X-Developer-Key: i=skinsburskii@gmail.com; a=ed25519; pk=bDpriHBYgeTdkIDweZDCemxsU93neJBOCn3YLIuJpnE= Several GPU SVM paths take mmap_read_lock() only to call hmm_range_fault() and open-code mmu interval sequence setup before each HMM walk. They also retry -EBUSY until HMM_RANGE_DEFAULT_TIMEOUT expires. Use hmm_range_fault_unlocked_timeout() for those faults. The HMM helper now owns mmap_lock acquisition and refreshes range->notifier_seq for its internal retries, while GPU SVM keeps its existing driver-lock validation with mmu_interval_read_retry() after a successful fault. drm_gpusvm_scan_mm() and drm_gpusvm_range_evict() pass HMM_RANGE_DEFAULT_TIMEOUT as the helper retry budget for each HMM fault attempt. drm_gpusvm_get_pages() keeps its existing absolute outer deadline because it can be reached from GPU page-fault workers, where fatal signals from the faulting process cannot stop an endless invalidation retry loop. It passes the remaining time from that deadline to HMM for each fault attempt. Leave drm_gpusvm_check_pages() on hmm_range_fault() because that path is called with the mmap lock already held by its caller. Reviewed-by: Jason Gunthorpe Signed-off-by: Stanislav Kinsburskii --- drivers/gpu/drm/drm_gpusvm.c | 60 ++++++++--------------------------------= ---- 1 file changed, 10 insertions(+), 50 deletions(-) diff --git a/drivers/gpu/drm/drm_gpusvm.c b/drivers/gpu/drm/drm_gpusvm.c index 958cb605aedd..e912e994beb2 100644 --- a/drivers/gpu/drm/drm_gpusvm.c +++ b/drivers/gpu/drm/drm_gpusvm.c @@ -773,8 +773,7 @@ enum drm_gpusvm_scan_result drm_gpusvm_scan_mm(struct d= rm_gpusvm_range *range, .end =3D end, .dev_private_owner =3D dev_private_owner, }; - unsigned long timeout =3D - jiffies + msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT); + unsigned long timeout =3D msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT); enum drm_gpusvm_scan_result state =3D DRM_GPUSVM_SCAN_UNPOPULATED, new_st= ate; unsigned long *pfns; unsigned long npages =3D npages_in_range(start, end); @@ -788,22 +787,7 @@ enum drm_gpusvm_scan_result drm_gpusvm_scan_mm(struct = drm_gpusvm_range *range, hmm_range.hmm_pfns =3D pfns; =20 retry: - hmm_range.notifier_seq =3D mmu_interval_read_begin(notifier); - mmap_read_lock(range->gpusvm->mm); - - while (true) { - err =3D hmm_range_fault(&hmm_range); - if (err =3D=3D -EBUSY) { - if (time_after(jiffies, timeout)) - break; - - hmm_range.notifier_seq =3D - mmu_interval_read_begin(notifier); - continue; - } - break; - } - mmap_read_unlock(range->gpusvm->mm); + err =3D hmm_range_fault_unlocked_timeout(&hmm_range, timeout); if (err) goto err_free; =20 @@ -1408,6 +1392,7 @@ int drm_gpusvm_get_pages(struct drm_gpusvm *gpusvm, void *zdd; unsigned long timeout =3D jiffies + msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT); + unsigned long remaining; unsigned long i, j; unsigned long npages =3D npages_in_range(pages_start, pages_end); unsigned long num_dma_mapped; @@ -1422,7 +1407,9 @@ int drm_gpusvm_get_pages(struct drm_gpusvm *gpusvm, struct dma_iova_state *state =3D &svm_pages->state; =20 retry: - if (time_after(jiffies, timeout)) + remaining =3D timeout - jiffies; + + if (time_after_eq(jiffies, timeout)) return -EBUSY; =20 hmm_range.notifier_seq =3D mmu_interval_read_begin(notifier); @@ -1439,21 +1426,7 @@ int drm_gpusvm_get_pages(struct drm_gpusvm *gpusvm, } =20 hmm_range.hmm_pfns =3D pfns; - while (true) { - mmap_read_lock(mm); - err =3D hmm_range_fault(&hmm_range); - mmap_read_unlock(mm); - - if (err =3D=3D -EBUSY) { - if (time_after(jiffies, timeout)) - break; - - hmm_range.notifier_seq =3D - mmu_interval_read_begin(notifier); - continue; - } - break; - } + err =3D hmm_range_fault_unlocked_timeout(&hmm_range, remaining); mmput(mm); if (err) goto err_free; @@ -1720,8 +1693,7 @@ int drm_gpusvm_range_evict(struct drm_gpusvm *gpusvm, .end =3D drm_gpusvm_range_end(range), .dev_private_owner =3D NULL, }; - unsigned long timeout =3D - jiffies + msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT); + unsigned long timeout =3D msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT); unsigned long *pfns; unsigned long npages =3D npages_in_range(drm_gpusvm_range_start(range), drm_gpusvm_range_end(range)); @@ -1736,24 +1708,12 @@ int drm_gpusvm_range_evict(struct drm_gpusvm *gpusv= m, return -ENOMEM; =20 hmm_range.hmm_pfns =3D pfns; - while (!time_after(jiffies, timeout)) { - hmm_range.notifier_seq =3D mmu_interval_read_begin(notifier); - if (time_after(jiffies, timeout)) { - err =3D -ETIME; - break; - } - - mmap_read_lock(mm); - err =3D hmm_range_fault(&hmm_range); - mmap_read_unlock(mm); - if (err !=3D -EBUSY) - break; - } + err =3D hmm_range_fault_unlocked_timeout(&hmm_range, timeout); =20 kvfree(pfns); mmput(mm); =20 - return err; + return err =3D=3D -EBUSY ? -ETIME : err; } EXPORT_SYMBOL_GPL(drm_gpusvm_range_evict); =20 --=20 2.43.0