From nobody Mon Apr 27 07:24:01 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C4071C43334 for ; Wed, 15 Jun 2022 15:29:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1347382AbiFOP3v (ORCPT ); Wed, 15 Jun 2022 11:29:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50610 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1355337AbiFOP2i (ORCPT ); Wed, 15 Jun 2022 11:28:38 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 17CA917A8E; Wed, 15 Jun 2022 08:27:47 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id A6F90616B1; Wed, 15 Jun 2022 15:27:46 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id F3672C385A2; Wed, 15 Jun 2022 15:27:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1655306866; bh=nX1w5nyKD3K3Gw/ZF5PMRaXHVZxvYBa3oXkY2JHQUN0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=iyea28kx9k5ypeyLzp7cl1Ekca/aVGlkg75Bt3ZbxKPGTXSxU2vKgZ8XIjqXitRPW 5H/QWhoNVlftRm8xtMxFKzTFvb3+5FAulFoYwyuq6PTJtUbITk4V2Bq82d1H8kTmB6 QNp00KVxH49Z2PIgW1iWNFlcNAKcgfHNvWHG0GqwEtti8EUo5zTPzIHK/ksGGLL23S mohEEiyyP0AAxyro4qjdSFiw2litXaNf8u5fGvHIyVdW8Ka8CWwKiRNS10bIBNA658 ksgmxr6fGW3QfqzSjjT+JNh4fzUt3CaZX+qIvOZ1MfF0+4Jlvnpe45SC9fT+dgXX5t IO2UvEwRD8+aQ== Received: from mchehab by mail.kernel.org with local (Exim 4.95) (envelope-from ) id 1o1Uvm-00A4Jd-Cq; Wed, 15 Jun 2022 16:27:42 +0100 From: Mauro Carvalho Chehab Cc: Chris Wilson , "Fei Yang" , =?UTF-8?q?Micha=C5=82=20Winiarski?= , "Thomas Hellstrom" , =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= , Andi Shyti , Daniel Vetter , Daniele Ceraolo Spurio , Dave Airlie , David Airlie , Jani Nikula , Jason Ekstrand , John Harrison , Joonas Lahtinen , Lucas De Marchi , Maarten Lankhorst , Matt Roper , Matthew Auld , Matthew Brost , Mauro Carvalho Chehab , Ramalingam C , Rodrigo Vivi , Tvrtko Ursulin , dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, linux-kernel@vger.kernel.org, mauro.chehab@linux.intel.com, stable@vger.kernel.org Subject: [PATCH 1/6] drm/i915/gt: Ignore TLB invalidations on idle engines Date: Wed, 15 Jun 2022 16:27:35 +0100 Message-Id: X-Mailer: git-send-email 2.36.1 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Chris Wilson As an extension of the current skip TLB invalidations, check if the device is powered down prior to any engine activity, as, on such cases, all the TLBs were already invalidated, so an explicit TLB invalidation is not needed. This becomes more significant with GuC, as it can only do so when the connection to the GuC is awake. Fixes: 7938d61591d3 ("drm/i915: Flush TLBs before releasing backing store") Signed-off-by: Chris Wilson Cc: Fei Yang Cc: Andi Shyti Cc: stable@vger.kernel.org Acked-by: Thomas Hellstr=C3=B6m Signed-off-by: Mauro Carvalho Chehab Reviewed-by: Andi Shyti --- See [PATCH 0/6] at: https://lore.kernel.org/all/cover.1655306128.git.mcheha= b@kernel.org/ drivers/gpu/drm/i915/gem/i915_gem_pages.c | 10 +++++---- drivers/gpu/drm/i915/gt/intel_gt.c | 26 +++++++++++++++++------ drivers/gpu/drm/i915/gt/intel_gt_pm.h | 3 +++ 3 files changed, 28 insertions(+), 11 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_pages.c b/drivers/gpu/drm/i9= 15/gem/i915_gem_pages.c index 97c820eee115..6835279943df 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_pages.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_pages.c @@ -6,14 +6,15 @@ =20 #include =20 +#include "gt/intel_gt.h" +#include "gt/intel_gt_pm.h" + #include "i915_drv.h" #include "i915_gem_object.h" #include "i915_scatterlist.h" #include "i915_gem_lmem.h" #include "i915_gem_mman.h" =20 -#include "gt/intel_gt.h" - void __i915_gem_object_set_pages(struct drm_i915_gem_object *obj, struct sg_table *pages, unsigned int sg_page_sizes) @@ -217,10 +218,11 @@ __i915_gem_object_unset_pages(struct drm_i915_gem_obj= ect *obj) =20 if (test_and_clear_bit(I915_BO_WAS_BOUND_BIT, &obj->flags)) { struct drm_i915_private *i915 =3D to_i915(obj->base.dev); + struct intel_gt *gt =3D to_gt(i915); intel_wakeref_t wakeref; =20 - with_intel_runtime_pm_if_active(&i915->runtime_pm, wakeref) - intel_gt_invalidate_tlbs(to_gt(i915)); + with_intel_gt_pm_if_awake(gt, wakeref) + intel_gt_invalidate_tlbs(gt); } =20 return pages; diff --git a/drivers/gpu/drm/i915/gt/intel_gt.c b/drivers/gpu/drm/i915/gt/i= ntel_gt.c index f33290358c51..d5ed6a6ac67c 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt.c +++ b/drivers/gpu/drm/i915/gt/intel_gt.c @@ -11,6 +11,7 @@ =20 #include "i915_drv.h" #include "intel_context.h" +#include "intel_engine_pm.h" #include "intel_engine_regs.h" #include "intel_gt.h" #include "intel_gt_buffer_pool.h" @@ -1216,6 +1217,7 @@ void intel_gt_invalidate_tlbs(struct intel_gt *gt) struct drm_i915_private *i915 =3D gt->i915; struct intel_uncore *uncore =3D gt->uncore; struct intel_engine_cs *engine; + intel_engine_mask_t awake, tmp; enum intel_engine_id id; const i915_reg_t *regs; unsigned int num =3D 0; @@ -1239,12 +1241,27 @@ void intel_gt_invalidate_tlbs(struct intel_gt *gt) =20 GEM_TRACE("\n"); =20 - assert_rpm_wakelock_held(&i915->runtime_pm); - mutex_lock(>->tlb_invalidate_lock); intel_uncore_forcewake_get(uncore, FORCEWAKE_ALL); =20 + awake =3D 0; for_each_engine(engine, gt, id) { + struct reg_and_bit rb; + + if (!intel_engine_pm_is_awake(engine)) + continue; + + rb =3D get_reg_and_bit(engine, regs =3D=3D gen8_regs, regs, num); + if (!i915_mmio_reg_offset(rb.reg)) + continue; + + intel_uncore_write_fw(uncore, rb.reg, rb.bit); + awake |=3D engine->mask; + } + + for_each_engine_masked(engine, gt, awake, tmp) { + struct reg_and_bit rb; + /* * HW architecture suggest typical invalidation time at 40us, * with pessimistic cases up to 100us and a recommendation to @@ -1252,13 +1269,8 @@ void intel_gt_invalidate_tlbs(struct intel_gt *gt) */ const unsigned int timeout_us =3D 100; const unsigned int timeout_ms =3D 4; - struct reg_and_bit rb; =20 rb =3D get_reg_and_bit(engine, regs =3D=3D gen8_regs, regs, num); - if (!i915_mmio_reg_offset(rb.reg)) - continue; - - intel_uncore_write_fw(uncore, rb.reg, rb.bit); if (__intel_wait_for_register_fw(uncore, rb.reg, rb.bit, 0, timeout_us, timeout_ms, diff --git a/drivers/gpu/drm/i915/gt/intel_gt_pm.h b/drivers/gpu/drm/i915/g= t/intel_gt_pm.h index bc898df7a48c..a334787a4939 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt_pm.h +++ b/drivers/gpu/drm/i915/gt/intel_gt_pm.h @@ -55,6 +55,9 @@ static inline void intel_gt_pm_might_put(struct intel_gt = *gt) for (tmp =3D 1, intel_gt_pm_get(gt); tmp; \ intel_gt_pm_put(gt), tmp =3D 0) =20 +#define with_intel_gt_pm_if_awake(gt, wf) \ + for (wf =3D intel_gt_pm_get_if_awake(gt); wf; intel_gt_pm_put_async(gt), = wf =3D 0) + static inline int intel_gt_pm_wait_for_idle(struct intel_gt *gt) { return intel_wakeref_wait_for_idle(>->wakeref); --=20 2.36.1 From nobody Mon Apr 27 07:24:01 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ADE43C43334 for ; Wed, 15 Jun 2022 15:30:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1354900AbiFOPaH (ORCPT ); Wed, 15 Jun 2022 11:30:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48224 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1355309AbiFOP2i (ORCPT ); Wed, 15 Jun 2022 11:28:38 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 238874F1F3; Wed, 15 Jun 2022 08:27:47 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id AB29B61720; Wed, 15 Jun 2022 15:27:46 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 049ADC341C0; Wed, 15 Jun 2022 15:27:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1655306866; bh=8ReZPcBGEj7z1AKZG+D3Lo4912WJuXG6ESeM/dKTzl4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UaIVU0Tss08B9KniFRBUMT/rBzskVD1GrxtrCV+Mt0QMmLPB8FiAva4kFBjBzAtdL LPvF1LiI/+W7CGgihPWqX+zSwgiacXT6sqK8UoZqNPfhGb4/lXgzgqLVAY5NXavsrB b0Uppr8tCzYTyyms5Zqs7z+i0yOJy1wnCmJEriZJfhQF1C2ynOXI52UeOgF4vvdlxv Zj5bI59MvUSn4JYMcTAwDqnS1TCZNKSCZz5LO75zRUQ0nX9/E00WmS6enpGeSRNHY3 2Hq0VGGSS2WESzax7kt87Mb+5BnCos1jPnTF5vdjTglFNPpNfpdHtDge0SUJIoLTSA 9MATc4KNiJxWA== Received: from mchehab by mail.kernel.org with local (Exim 4.95) (envelope-from ) id 1o1Uvm-00A4Jg-Df; Wed, 15 Jun 2022 16:27:42 +0100 From: Mauro Carvalho Chehab Cc: Chris Wilson , "Fei Yang" , =?UTF-8?q?Micha=C5=82=20Winiarski?= , "Thomas Hellstrom" , Andi Shyti , Daniel Vetter , Daniele Ceraolo Spurio , Dave Airlie , David Airlie , Jani Nikula , John Harrison , Joonas Lahtinen , Lucas De Marchi , Matt Roper , Matthew Auld , Rodrigo Vivi , Tvrtko Ursulin , dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, linux-kernel@vger.kernel.org, mauro.chehab@linux.intel.com, stable@vger.kernel.org, =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= , Mauro Carvalho Chehab Subject: [PATCH 2/6] drm/i915/gt: Invalidate TLB of the OA unit at TLB invalidations Date: Wed, 15 Jun 2022 16:27:36 +0100 Message-Id: <653bf9815d562f02c7247c6b66b85b243f3172e7.1655306128.git.mchehab@kernel.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Chris Wilson On gen12 HW, ensure that the TLB of the OA unit is also invalidated as just invalidating the TLB of an engine is not enough. Fixes: 7938d61591d3 ("drm/i915: Flush TLBs before releasing backing store") Signed-off-by: Chris Wilson Cc: Fei Yang Cc: Andi Shyti Cc: stable@vger.kernel.org Acked-by: Thomas Hellstr=C3=B6m Signed-off-by: Mauro Carvalho Chehab Reviewed-by: Andi Shyti --- See [PATCH 0/6] at: https://lore.kernel.org/all/cover.1655306128.git.mcheha= b@kernel.org/ drivers/gpu/drm/i915/gt/intel_gt.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/drivers/gpu/drm/i915/gt/intel_gt.c b/drivers/gpu/drm/i915/gt/i= ntel_gt.c index d5ed6a6ac67c..61b7ec5118f9 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt.c +++ b/drivers/gpu/drm/i915/gt/intel_gt.c @@ -10,6 +10,7 @@ #include "pxp/intel_pxp.h" =20 #include "i915_drv.h" +#include "i915_perf_oa_regs.h" #include "intel_context.h" #include "intel_engine_pm.h" #include "intel_engine_regs.h" @@ -1259,6 +1260,15 @@ void intel_gt_invalidate_tlbs(struct intel_gt *gt) awake |=3D engine->mask; } =20 + /* Wa_2207587034:tgl,dg1,rkl,adl-s,adl-p */ + if (awake && + (IS_TIGERLAKE(i915) || + IS_DG1(i915) || + IS_ROCKETLAKE(i915) || + IS_ALDERLAKE_S(i915) || + IS_ALDERLAKE_P(i915))) + intel_uncore_write_fw(uncore, GEN12_OA_TLB_INV_CR, 1); + for_each_engine_masked(engine, gt, awake, tmp) { struct reg_and_bit rb; =20 --=20 2.36.1 From nobody Mon Apr 27 07:24:01 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 880F6C433EF for ; Wed, 15 Jun 2022 15:30:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1353580AbiFOPaQ (ORCPT ); Wed, 15 Jun 2022 11:30:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50524 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1355472AbiFOP2i (ORCPT ); Wed, 15 Jun 2022 11:28:38 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AFE94427F1; Wed, 15 Jun 2022 08:27:48 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 55EF2B81F0B; Wed, 15 Jun 2022 15:27:47 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id ECDEDC34115; Wed, 15 Jun 2022 15:27:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1655306866; bh=d5dFQ9NHsAkSm+gLQzVN8kPNqSNsxZsVoyo0jFhjNL0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LXQf8xhEcd/2SP9lwnz0ukumi2OpBNyaaWzqoWEweJNDGmokgwGwt3y1sppwNHyQ7 Rzxzuq4qly/kJRh9TpXSJs/vZ4SSwYySPo6oCtR3lSb9i2ZGukCYt4ukqKgYmFIxCi 4RYwiBl2BDFOyJEqVGqfPdTjg/wDJ2XcvxfPYlQ/2GwAx3ebFWcSy330pPk76hRvDv m1eLCgWxxKsmve7jSKKvt972JvLb/DQ1dXV3QlwxS2UNwbNxjFozXlJyyu8o8WPTFV KO/BfNu64ZOcJb9+Gdm3qMvPUeiwovYMC/uqNJ7oIVn1Y+/K4J48asEamxxyleAP/i 6fvv0mYBklH9w== Received: from mchehab by mail.kernel.org with local (Exim 4.95) (envelope-from ) id 1o1Uvm-00A4Jj-EX; Wed, 15 Jun 2022 16:27:42 +0100 From: Mauro Carvalho Chehab Cc: Chris Wilson , "Fei Yang" , =?UTF-8?q?Micha=C5=82=20Winiarski?= , "Thomas Hellstrom" , Andi Shyti , Daniel Vetter , Daniele Ceraolo Spurio , Dave Airlie , David Airlie , Jani Nikula , John Harrison , Joonas Lahtinen , Lucas De Marchi , Matt Roper , Matthew Auld , Rodrigo Vivi , Tvrtko Ursulin , dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, linux-kernel@vger.kernel.org, mauro.chehab@linux.intel.com, stable@vger.kernel.org, =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= , Mauro Carvalho Chehab Subject: [PATCH 3/6] drm/i915/gt: Skip TLB invalidations once wedged Date: Wed, 15 Jun 2022 16:27:37 +0100 Message-Id: <9d9e663ca8e97becf04e1d4c8cb8a9a1f397a5f1.1655306128.git.mchehab@kernel.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Chris Wilson Skip all further TLB invalidations once the device is wedged and had been reset, as, on such cases, it can no longer process instructions on the GPU and the user no longer has access to the TLB's in each engine. Fixes: 7938d61591d3 ("drm/i915: Flush TLBs before releasing backing store") Signed-off-by: Chris Wilson Cc: Fei Yang Cc: Andi Shyti Cc: stable@vger.kernel.org Acked-by: Thomas Hellstr=C3=B6m Signed-off-by: Mauro Carvalho Chehab Reviewed-by: Andi Shyti --- See [PATCH 0/6] at: https://lore.kernel.org/all/cover.1655306128.git.mcheha= b@kernel.org/ drivers/gpu/drm/i915/gt/intel_gt.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/gpu/drm/i915/gt/intel_gt.c b/drivers/gpu/drm/i915/gt/i= ntel_gt.c index 61b7ec5118f9..fb4fd5273ca4 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt.c +++ b/drivers/gpu/drm/i915/gt/intel_gt.c @@ -1226,6 +1226,9 @@ void intel_gt_invalidate_tlbs(struct intel_gt *gt) if (I915_SELFTEST_ONLY(gt->awake =3D=3D -ENODEV)) return; =20 + if (intel_gt_is_wedged(gt)) + return; + if (GRAPHICS_VER(i915) =3D=3D 12) { regs =3D gen12_regs; num =3D ARRAY_SIZE(gen12_regs); --=20 2.36.1 From nobody Mon Apr 27 07:24:01 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D8AF1CCA473 for ; Wed, 15 Jun 2022 15:29:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234903AbiFOP3r (ORCPT ); Wed, 15 Jun 2022 11:29:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48226 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1355365AbiFOP2i (ORCPT ); Wed, 15 Jun 2022 11:28:38 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C6CBC4F456; Wed, 15 Jun 2022 08:27:47 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 3DE6461730; Wed, 15 Jun 2022 15:27:47 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 075C2C341C4; Wed, 15 Jun 2022 15:27:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1655306866; bh=ArrWWi9aGs9TPTpy73ldENPFG9RCYJiCftXoVPyfiXA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=g+Z9a8tBUvrdOq9En5CSui7C1egHykyAomklROUSNevyZlUzrxdBtW+8Jg/Sbs3dx B/frSxNCMCV1HpkQb3FUy9yPEHvE/C4Fgcscb6nwZk2wud73DWJQmGG+ZnZeaMqoqj S4tTuphrd2FjPf2ytW5WMFAYIzf4OoLS6qhqVzgqnTBXWADmBbQnxh9XrDdAC5v4o8 DKszVAokucAtHdAzZu2vvm0TGcdwwfuXbJEyhSwPMcZQ73CL7YMWu1oShtibuLPhCx +43Tz0HKhqsADIEkozMT7BH7OSAxi6O3sxqaB4RdVWT2S1xvBhG2DqLK/O3jJNGX25 2jkOkUinAk7vQ== Received: from mchehab by mail.kernel.org with local (Exim 4.95) (envelope-from ) id 1o1Uvm-00A4Jm-FD; Wed, 15 Jun 2022 16:27:42 +0100 From: Mauro Carvalho Chehab Cc: Chris Wilson , "Fei Yang" , "Thomas Hellstrom" , Daniel Vetter , Dave Airlie , David Airlie , Jani Nikula , Joonas Lahtinen , Rodrigo Vivi , Tvrtko Ursulin , dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, linux-kernel@vger.kernel.org, mauro.chehab@linux.intel.com, Andi Shyti , stable@vger.kernel.org, =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= , Mauro Carvalho Chehab Subject: [PATCH 4/6] drm/i915/gt: Only invalidate TLBs exposed to user manipulation Date: Wed, 15 Jun 2022 16:27:38 +0100 Message-Id: <387b9a8d3e719ad2db4fce56c0bfc0f909fd6df6.1655306128.git.mchehab@kernel.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Chris Wilson Don't flush TLBs when the buffer is only used in the GGTT under full control of the kernel, as there's no risk of of concurrent access and stale access from prefetch. We only need to invalidate the TLB if they are accessible by the user. Fixes: 7938d61591d3 ("drm/i915: Flush TLBs before releasing backing store") Signed-off-by: Chris Wilson Cc: Fei Yang Cc: Andi Shyti Cc: stable@vger.kernel.org Acked-by: Thomas Hellstr=C3=B6m Signed-off-by: Mauro Carvalho Chehab Reviewed-by: Andi Shyti --- See [PATCH 0/6] at: https://lore.kernel.org/all/cover.1655306128.git.mcheha= b@kernel.org/ drivers/gpu/drm/i915/i915_vma.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/i915/i915_vma.c b/drivers/gpu/drm/i915/i915_vm= a.c index 0bffb70b3c5f..7989986161e8 100644 --- a/drivers/gpu/drm/i915/i915_vma.c +++ b/drivers/gpu/drm/i915/i915_vma.c @@ -537,7 +537,8 @@ int i915_vma_bind(struct i915_vma *vma, bind_flags); } =20 - set_bit(I915_BO_WAS_BOUND_BIT, &vma->obj->flags); + if (bind_flags & I915_VMA_LOCAL_BIND) + set_bit(I915_BO_WAS_BOUND_BIT, &vma->obj->flags); =20 atomic_or(bind_flags, &vma->flags); return 0; --=20 2.36.1 From nobody Mon Apr 27 07:24:01 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8E307C43334 for ; Wed, 15 Jun 2022 15:30:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352365AbiFOPaD (ORCPT ); Wed, 15 Jun 2022 11:30:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48010 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1355504AbiFOP2j (ORCPT ); Wed, 15 Jun 2022 11:28:39 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CE39042A0D; Wed, 15 Jun 2022 08:27:48 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 72012B81F0F; Wed, 15 Jun 2022 15:27:47 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id F1203C3411C; Wed, 15 Jun 2022 15:27:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1655306866; bh=BlDIEcKJ14OpD6+ESND5UCNOIyMjbS2bd2Y8eqJ8uxk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=F9MjNUhSfIW8LIsi3WBectn1THEMp6emsr48hbbUBq2P3TCS3ncW61UNYtleA9Knt IxxtwpahQvzmOqagK7VYFtPf3ANq/H27tb65repDfjjs3Sf+RMG4fI0RH5bWBk0sDb Ev4qrU/xSHXtHUr8+dGV980TkCrkOBq206yN7HUvhTWOwhIATeZikMwpUHhFWKakbW obpNFsin/mwnj2cyGt4iBEf/unFNA0BLc8H5PxONTr2RgUnMlTlekmnPHeKHHJUs+F bPobjAueges7heOGy0Xt7zGXXx9NFJCCfQAk0TGNRq6k0WSOf9jvnGn92WhMk6WyOR MniWUE7lV7lbA== Received: from mchehab by mail.kernel.org with local (Exim 4.95) (envelope-from ) id 1o1Uvm-00A4Jp-Fz; Wed, 15 Jun 2022 16:27:42 +0100 From: Mauro Carvalho Chehab Cc: Chris Wilson , "Fei Yang" , "Thomas Hellstrom" , Bruce Chang , Daniel Vetter , Dave Airlie , David Airlie , Jani Nikula , John Harrison , Joonas Lahtinen , Matt Roper , Matthew Brost , Rodrigo Vivi , Tejas Upadhyay , Tvrtko Ursulin , Umesh Nerlige Ramappa , dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, linux-kernel@vger.kernel.org, mauro.chehab@linux.intel.com, Mika Kuoppala , Chris Wilson , Andi Shyti , stable@vger.kernel.org, =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= , Mauro Carvalho Chehab Subject: [PATCH 5/6] drm/i915/gt: Serialize GRDOM access between multiple engine resets Date: Wed, 15 Jun 2022 16:27:39 +0100 Message-Id: <5ee647f243a774927ec328bfca8212abc4957909.1655306128.git.mchehab@kernel.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Chris Wilson Don't allow two engines to be reset in parallel, as they would both try to select a reset bit (and send requests to common registers) and wait on that register, at the same time. Serialize control of the reset requests/acks using the uncore->lock, which will also ensure that no other GT state changes at the same time as the actual reset. Fixes: 7938d61591d3 ("drm/i915: Flush TLBs before releasing backing store") Reported-by: Mika Kuoppala Signed-off-by: Chris Wilson Cc: Mika Kuoppala Cc: Andi Shyti Cc: stable@vger.kernel.org Acked-by: Thomas Hellstr=C3=B6m Signed-off-by: Mauro Carvalho Chehab Reviewed-by: Andi Shyti Reviewed-by: Tvrtko Ursulin --- See [PATCH 0/6] at: https://lore.kernel.org/all/cover.1655306128.git.mcheha= b@kernel.org/ drivers/gpu/drm/i915/gt/intel_reset.c | 37 ++++++++++++++++++++------- 1 file changed, 28 insertions(+), 9 deletions(-) diff --git a/drivers/gpu/drm/i915/gt/intel_reset.c b/drivers/gpu/drm/i915/g= t/intel_reset.c index a5338c3fde7a..c68d36fb5bbd 100644 --- a/drivers/gpu/drm/i915/gt/intel_reset.c +++ b/drivers/gpu/drm/i915/gt/intel_reset.c @@ -300,9 +300,9 @@ static int gen6_hw_domain_reset(struct intel_gt *gt, u3= 2 hw_domain_mask) return err; } =20 -static int gen6_reset_engines(struct intel_gt *gt, - intel_engine_mask_t engine_mask, - unsigned int retry) +static int __gen6_reset_engines(struct intel_gt *gt, + intel_engine_mask_t engine_mask, + unsigned int retry) { struct intel_engine_cs *engine; u32 hw_mask; @@ -321,6 +321,20 @@ static int gen6_reset_engines(struct intel_gt *gt, return gen6_hw_domain_reset(gt, hw_mask); } =20 +static int gen6_reset_engines(struct intel_gt *gt, + intel_engine_mask_t engine_mask, + unsigned int retry) +{ + unsigned long flags; + int ret; + + spin_lock_irqsave(>->uncore->lock, flags); + ret =3D __gen6_reset_engines(gt, engine_mask, retry); + spin_unlock_irqrestore(>->uncore->lock, flags); + + return ret; +} + static struct intel_engine_cs *find_sfc_paired_vecs_engine(struct intel_en= gine_cs *engine) { int vecs_id; @@ -487,9 +501,9 @@ static void gen11_unlock_sfc(struct intel_engine_cs *en= gine) rmw_clear_fw(uncore, sfc_lock.lock_reg, sfc_lock.lock_bit); } =20 -static int gen11_reset_engines(struct intel_gt *gt, - intel_engine_mask_t engine_mask, - unsigned int retry) +static int __gen11_reset_engines(struct intel_gt *gt, + intel_engine_mask_t engine_mask, + unsigned int retry) { struct intel_engine_cs *engine; intel_engine_mask_t tmp; @@ -583,8 +597,11 @@ static int gen8_reset_engines(struct intel_gt *gt, struct intel_engine_cs *engine; const bool reset_non_ready =3D retry >=3D 1; intel_engine_mask_t tmp; + unsigned long flags; int ret; =20 + spin_lock_irqsave(>->uncore->lock, flags); + for_each_engine_masked(engine, gt, engine_mask, tmp) { ret =3D gen8_engine_reset_prepare(engine); if (ret && !reset_non_ready) @@ -612,17 +629,19 @@ static int gen8_reset_engines(struct intel_gt *gt, * This is best effort, so ignore any error from the initial reset. */ if (IS_DG2(gt->i915) && engine_mask =3D=3D ALL_ENGINES) - gen11_reset_engines(gt, gt->info.engine_mask, 0); + __gen11_reset_engines(gt, gt->info.engine_mask, 0); =20 if (GRAPHICS_VER(gt->i915) >=3D 11) - ret =3D gen11_reset_engines(gt, engine_mask, retry); + ret =3D __gen11_reset_engines(gt, engine_mask, retry); else - ret =3D gen6_reset_engines(gt, engine_mask, retry); + ret =3D __gen6_reset_engines(gt, engine_mask, retry); =20 skip_reset: for_each_engine_masked(engine, gt, engine_mask, tmp) gen8_engine_reset_cancel(engine); =20 + spin_unlock_irqrestore(>->uncore->lock, flags); + return ret; } =20 --=20 2.36.1 From nobody Mon Apr 27 07:24:01 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 60A69C43334 for ; Wed, 15 Jun 2022 15:29:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1353958AbiFOP34 (ORCPT ); Wed, 15 Jun 2022 11:29:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50624 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1355496AbiFOP2j (ORCPT ); Wed, 15 Jun 2022 11:28:39 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B01A342A06; Wed, 15 Jun 2022 08:27:48 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 6061BB81F0E; Wed, 15 Jun 2022 15:27:47 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 02093C385A5; Wed, 15 Jun 2022 15:27:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1655306866; bh=7pc/xKe4ZdwnskrQCRUD+TTNRE6A+ErRUxrsSCmYE1E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Qaq1+tBx1+bt4SshGdEjpnAntGkki8cqcHlFENHDJujz41uB5g3hWZyTnPexsguLs IH68RjS9AS+BlNt1b5VYGzSm7Q4RPHbtjNFW2heHi2/37cayJje7Vrmd8/Zhr1fxGZ XOvEXtxq81DyZaok2AIhQUEXMGlUv0DJ6t3oYpM6Psf/PocxKX47EctWpGmx4xEFnb Oy6nHDBgLSzWgDNbHOMsIIlK3ho5hrnNtl/Yaz3TwniC4u4VIA7qBxqDZ/YM4gGRYS 6btexI/QV2/Knts39rsqhcwp4L4SI1D8XFHaEVlk2foWx+I8XlNcYIxp3STyq98n95 Hv5kWhYH6N68w== Received: from mchehab by mail.kernel.org with local (Exim 4.95) (envelope-from ) id 1o1Uvm-00A4Js-Gn; Wed, 15 Jun 2022 16:27:42 +0100 From: Mauro Carvalho Chehab Cc: Chris Wilson , "Fei Yang" , =?UTF-8?q?Micha=C5=82=20Winiarski?= , "Thomas Hellstrom" , =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= , Andi Shyti , Daniel Vetter , Daniele Ceraolo Spurio , Dave Airlie , David Airlie , Jani Nikula , Joonas Lahtinen , Lucas De Marchi , Matt Roper , Matthew Auld , Rodrigo Vivi , Tvrtko Ursulin , dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, linux-kernel@vger.kernel.org, mauro.chehab@linux.intel.com, Mauro Carvalho Chehab , stable@vger.kernel.org Subject: [PATCH 6/6] drm/i915/gt: Serialize TLB invalidates with GT resets Date: Wed, 15 Jun 2022 16:27:40 +0100 Message-Id: X-Mailer: git-send-email 2.36.1 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Chris Wilson Avoid trying to invalidate the TLB in the middle of performing an engine reset, as this may result in the reset timing out. Currently, the TLB invalidate is only serialised by its own mutex, forgoing the uncore lock, but we can take the uncore->lock as well to serialise the mmio access, thereby serialising with the GDRST. Tested on a NUC5i7RYB, BIOS RYBDWi35.86A.0380.2019.0517.1530 with i915 selftest/hangcheck. Fixes: 7938d61591d3 ("drm/i915: Flush TLBs before releasing backing store") Reported-by: Mauro Carvalho Chehab Tested-by: Mauro Carvalho Chehab Reviewed-by: Mauro Carvalho Chehab Signed-off-by: Chris Wilson Cc: Tvrtko Ursulin Cc: stable@vger.kernel.org Acked-by: Thomas Hellstr=C3=B6m Signed-off-by: Mauro Carvalho Chehab Reviewed-by: Andi Shyti --- See [PATCH 0/6] at: https://lore.kernel.org/all/cover.1655306128.git.mcheha= b@kernel.org/ drivers/gpu/drm/i915/gt/intel_gt.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/gpu/drm/i915/gt/intel_gt.c b/drivers/gpu/drm/i915/gt/i= ntel_gt.c index fb4fd5273ca4..33eb93586858 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt.c +++ b/drivers/gpu/drm/i915/gt/intel_gt.c @@ -1248,6 +1248,8 @@ void intel_gt_invalidate_tlbs(struct intel_gt *gt) mutex_lock(>->tlb_invalidate_lock); intel_uncore_forcewake_get(uncore, FORCEWAKE_ALL); =20 + spin_lock_irq(&uncore->lock); /* seralise invalidate with GT reset */ + awake =3D 0; for_each_engine(engine, gt, id) { struct reg_and_bit rb; @@ -1272,6 +1274,8 @@ void intel_gt_invalidate_tlbs(struct intel_gt *gt) IS_ALDERLAKE_P(i915))) intel_uncore_write_fw(uncore, GEN12_OA_TLB_INV_CR, 1); =20 + spin_unlock_irq(&uncore->lock); + for_each_engine_masked(engine, gt, awake, tmp) { struct reg_and_bit rb; =20 --=20 2.36.1