From nobody Sat Apr 18 14:11:47 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9962EC433EF for ; Wed, 13 Jul 2022 09:31:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236148AbiGMJbT (ORCPT ); Wed, 13 Jul 2022 05:31:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43984 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235528AbiGMJa3 (ORCPT ); Wed, 13 Jul 2022 05:30:29 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 85750F32E0; Wed, 13 Jul 2022 02:30:25 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id E2B1CB81D69; Wed, 13 Jul 2022 09:30:23 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 04FEEC341CD; Wed, 13 Jul 2022 09:30:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1657704622; bh=RUqK2XBsYMsaByMESrmgiO1BREUfe12Q9IJMe+D1jIc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OyAbLBwe/llXAtYI2X1GUJnLeHKM507rJ95TKh5vxj+oRbJqrrJI9crVxoRJ9q2lK yWaPsUg/CpUad1/VkG1b6MjC3wb1hIVOKpv8Uu3mDfHQy2AMtyvJsdYQbMjFLrfJYl IkHbFSOPsdJtJyWnjtU0oiJdHi51fp+gLR87sjwjhIobLMNCh1NjYS4uCciPX2cOQc jCOEObXlwF4wKUKrIcOCs0nMWYa77M5JJolKsMedMGL6pcIHPkCBig/jqaC+7P4k35 KTSh+5EzoCVqJKTK0U6xPfLYq27Xd7XhLzx1IlN0C21u74Gmz+5rr7joUN5YpXqqG0 zjQAq66cdR+cA== Received: from mchehab by mail.kernel.org with local (Exim 4.95) (envelope-from ) id 1oBYhH-0050LC-HV; Wed, 13 Jul 2022 10:30:19 +0100 From: Mauro Carvalho Chehab Cc: Chris Wilson , =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= , Andi Shyti , Daniel Vetter , Daniele Ceraolo Spurio , Dave Airlie , David Airlie , Jani Nikula , Jason Ekstrand , John Harrison , Joonas Lahtinen , Lucas De Marchi , Maarten Lankhorst , Matt Roper , Matthew Auld , Matthew Brost , Mauro Carvalho Chehab , Rodrigo Vivi , Tvrtko Ursulin , dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, linux-kernel@vger.kernel.org, stable@vger.kernel.org, Fei Yang Subject: [PATCH 01/21] drm/i915/gt: Ignore TLB invalidations on idle engines Date: Wed, 13 Jul 2022 10:29:58 +0100 Message-Id: <63074d74d0d3df9f3ebce2c9e78868205c9ae61d.1657703926.git.mchehab@kernel.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Chris Wilson Check if the device is powered down prior to any engine activity, as, on such cases, all the TLBs were already invalidated, so an explicit TLB invalidation is not needed, thus reducing the performance regression impact due to it. This becomes more significant with GuC, as it can only do so when the connection to the GuC is awake. Cc: stable@vger.kernel.org Fixes: 7938d61591d3 ("drm/i915: Flush TLBs before releasing backing store") Signed-off-by: Chris Wilson Cc: Fei Yang Cc: Andi Shyti Cc: Thomas Hellstr=C3=B6m Signed-off-by: Mauro Carvalho Chehab --- See [PATCH 00/21] at: https://lore.kernel.org/all/cover.1657703926.git.mche= hab@kernel.org/ drivers/gpu/drm/i915/gem/i915_gem_pages.c | 10 ++++++---- drivers/gpu/drm/i915/gt/intel_gt.c | 17 ++++++++++------- drivers/gpu/drm/i915/gt/intel_gt_pm.h | 3 +++ 3 files changed, 19 insertions(+), 11 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_pages.c b/drivers/gpu/drm/i9= 15/gem/i915_gem_pages.c index 97c820eee115..6835279943df 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_pages.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_pages.c @@ -6,14 +6,15 @@ =20 #include =20 +#include "gt/intel_gt.h" +#include "gt/intel_gt_pm.h" + #include "i915_drv.h" #include "i915_gem_object.h" #include "i915_scatterlist.h" #include "i915_gem_lmem.h" #include "i915_gem_mman.h" =20 -#include "gt/intel_gt.h" - void __i915_gem_object_set_pages(struct drm_i915_gem_object *obj, struct sg_table *pages, unsigned int sg_page_sizes) @@ -217,10 +218,11 @@ __i915_gem_object_unset_pages(struct drm_i915_gem_obj= ect *obj) =20 if (test_and_clear_bit(I915_BO_WAS_BOUND_BIT, &obj->flags)) { struct drm_i915_private *i915 =3D to_i915(obj->base.dev); + struct intel_gt *gt =3D to_gt(i915); intel_wakeref_t wakeref; =20 - with_intel_runtime_pm_if_active(&i915->runtime_pm, wakeref) - intel_gt_invalidate_tlbs(to_gt(i915)); + with_intel_gt_pm_if_awake(gt, wakeref) + intel_gt_invalidate_tlbs(gt); } =20 return pages; diff --git a/drivers/gpu/drm/i915/gt/intel_gt.c b/drivers/gpu/drm/i915/gt/i= ntel_gt.c index 68c2b0d8f187..c4d43da84d8e 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt.c +++ b/drivers/gpu/drm/i915/gt/intel_gt.c @@ -12,6 +12,7 @@ =20 #include "i915_drv.h" #include "intel_context.h" +#include "intel_engine_pm.h" #include "intel_engine_regs.h" #include "intel_ggtt_gmch.h" #include "intel_gt.h" @@ -924,6 +925,7 @@ void intel_gt_invalidate_tlbs(struct intel_gt *gt) struct drm_i915_private *i915 =3D gt->i915; struct intel_uncore *uncore =3D gt->uncore; struct intel_engine_cs *engine; + intel_engine_mask_t awake, tmp; enum intel_engine_id id; const i915_reg_t *regs; unsigned int num =3D 0; @@ -947,26 +949,31 @@ void intel_gt_invalidate_tlbs(struct intel_gt *gt) =20 GEM_TRACE("\n"); =20 - assert_rpm_wakelock_held(&i915->runtime_pm); - mutex_lock(>->tlb_invalidate_lock); intel_uncore_forcewake_get(uncore, FORCEWAKE_ALL); =20 spin_lock_irq(&uncore->lock); /* serialise invalidate with GT reset */ =20 + awake =3D 0; for_each_engine(engine, gt, id) { struct reg_and_bit rb; =20 + if (!intel_engine_pm_is_awake(engine)) + continue; + rb =3D get_reg_and_bit(engine, regs =3D=3D gen8_regs, regs, num); if (!i915_mmio_reg_offset(rb.reg)) continue; =20 intel_uncore_write_fw(uncore, rb.reg, rb.bit); + awake |=3D engine->mask; } =20 spin_unlock_irq(&uncore->lock); =20 - for_each_engine(engine, gt, id) { + for_each_engine_masked(engine, gt, awake, tmp) { + struct reg_and_bit rb; + /* * HW architecture suggest typical invalidation time at 40us, * with pessimistic cases up to 100us and a recommendation to @@ -974,12 +981,8 @@ void intel_gt_invalidate_tlbs(struct intel_gt *gt) */ const unsigned int timeout_us =3D 100; const unsigned int timeout_ms =3D 4; - struct reg_and_bit rb; =20 rb =3D get_reg_and_bit(engine, regs =3D=3D gen8_regs, regs, num); - if (!i915_mmio_reg_offset(rb.reg)) - continue; - if (__intel_wait_for_register_fw(uncore, rb.reg, rb.bit, 0, timeout_us, timeout_ms, diff --git a/drivers/gpu/drm/i915/gt/intel_gt_pm.h b/drivers/gpu/drm/i915/g= t/intel_gt_pm.h index bc898df7a48c..a334787a4939 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt_pm.h +++ b/drivers/gpu/drm/i915/gt/intel_gt_pm.h @@ -55,6 +55,9 @@ static inline void intel_gt_pm_might_put(struct intel_gt = *gt) for (tmp =3D 1, intel_gt_pm_get(gt); tmp; \ intel_gt_pm_put(gt), tmp =3D 0) =20 +#define with_intel_gt_pm_if_awake(gt, wf) \ + for (wf =3D intel_gt_pm_get_if_awake(gt); wf; intel_gt_pm_put_async(gt), = wf =3D 0) + static inline int intel_gt_pm_wait_for_idle(struct intel_gt *gt) { return intel_wakeref_wait_for_idle(>->wakeref); --=20 2.36.1 From nobody Sat Apr 18 14:11:47 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6B593C43334 for ; Wed, 13 Jul 2022 09:30:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235489AbiGMJa1 (ORCPT ); Wed, 13 Jul 2022 05:30:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43722 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235096AbiGMJaX (ORCPT ); Wed, 13 Jul 2022 05:30:23 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 21A67F2E0D for ; Wed, 13 Jul 2022 02:30:23 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id A5C1961CA1 for ; Wed, 13 Jul 2022 09:30:22 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E7D05C341C0; Wed, 13 Jul 2022 09:30:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1657704621; bh=oMqR086Sh4/11DEbRL1QYHjCa4npxyd592GKCoYy7dA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OgEhUNPjQ6hv/NsRUJVgEJP4P8zctXjwJ7IsMcn8R6KW67CXZpEX0jKlUTwKqp0JS tciXxSP/G1wxho/bwQ7rMbRvSPmpUMFYYDEDg0fdE6AVvxl9Va7CMkP/muPH7cDfpt HwKQW2kntXRpUYZYG8EZAS0lIbPzI7cc5Xb/+uB5YdtRkn+8Y1ju7riT8VTAv21rQZ 3VxqsY700jJAwuLy/AonV1jz+DhCgmmxX05KBhnC5DcT74zI6ZCu5UfnDqbPeeqVaB oJl7PZFDWBhrXF462nvFRujzYXeEp5bb9CllQp5YQWqAwmE3kLZiG8tYIsHH9pITGZ M+qgLV+iXacdg== Received: from mchehab by mail.kernel.org with local (Exim 4.95) (envelope-from ) id 1oBYhH-0050LF-IC; Wed, 13 Jul 2022 10:30:19 +0100 From: Mauro Carvalho Chehab Cc: Mauro Carvalho Chehab , Chris Wilson , Daniel Vetter , David Airlie , Jani Nikula , John Harrison , Joonas Lahtinen , Matthew Brost , Rodrigo Vivi , Tvrtko Ursulin , dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, linux-kernel@vger.kernel.org Subject: [PATCH 02/21] drm/i915/gt: document with_intel_gt_pm_if_awake() Date: Wed, 13 Jul 2022 10:29:59 +0100 Message-Id: X-Mailer: git-send-email 2.36.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add a kernel-doc markup to document this new macro. Signed-off-by: Mauro Carvalho Chehab --- See [PATCH 00/21] at: https://lore.kernel.org/all/cover.1657703926.git.mche= hab@kernel.org/ drivers/gpu/drm/i915/gt/intel_gt_pm.h | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/drivers/gpu/drm/i915/gt/intel_gt_pm.h b/drivers/gpu/drm/i915/g= t/intel_gt_pm.h index a334787a4939..4d4caf612fdc 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt_pm.h +++ b/drivers/gpu/drm/i915/gt/intel_gt_pm.h @@ -55,6 +55,13 @@ static inline void intel_gt_pm_might_put(struct intel_gt= *gt) for (tmp =3D 1, intel_gt_pm_get(gt); tmp; \ intel_gt_pm_put(gt), tmp =3D 0) =20 +/** + * with_intel_gt_pm_if_awake - if GT is PM awake, get a reference to preve= nt + * it to sleep, run some code and then put the reference away. + * + * @gt: pointer to the gt + * @wf: pointer to a temporary wakeref. + */ #define with_intel_gt_pm_if_awake(gt, wf) \ for (wf =3D intel_gt_pm_get_if_awake(gt); wf; intel_gt_pm_put_async(gt), = wf =3D 0) =20 --=20 2.36.1 From nobody Sat Apr 18 14:11:47 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7295CC43334 for ; Wed, 13 Jul 2022 09:30:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234186AbiGMJaw (ORCPT ); Wed, 13 Jul 2022 05:30:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43800 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235377AbiGMJaZ (ORCPT ); Wed, 13 Jul 2022 05:30:25 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9131EEE1F8; Wed, 13 Jul 2022 02:30:24 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 2D406B81CD2; Wed, 13 Jul 2022 09:30:23 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id DB669C34114; Wed, 13 Jul 2022 09:30:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1657704621; bh=TwXaLs6Qq1k/ViDsjq1vtPJ62qR+0nfRAZMZIFfXwvY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lM43VL+56kTLfitdH7ZNh3bl3OKPYGv0CN/4i6cU/58LXkg0Z/TYeYJJm1XxLiPJK RzXf0EjLOIMAtQ/xVBzpVliGJQxoRj56TNc1jDCUshX9hD1Y5AWHzd4fPj/hlJHD6u 09s6sIo7fTDmu3Kl0lC6d49Tun9JOqaAmKlME7v+JBF6be2lZMcXyXl1qIa2Ccp/0K FkRfrCExEsG9zk07B9SR4HuHH8A3xDOMcpa4VzU+qOx+PUUFzY2/5EWZm7rdbtBExO YIClcqCUboOMsfJJ6yA1QnYdAgMBz9tU5Id+ERDyOHMyT6FdbnJLpnoRP2Yo1uOdYt pITCazZ/bKRaw== Received: from mchehab by mail.kernel.org with local (Exim 4.95) (envelope-from ) id 1oBYhH-0050LI-Is; Wed, 13 Jul 2022 10:30:19 +0100 From: Mauro Carvalho Chehab Cc: Chris Wilson , Andi Shyti , Daniel Vetter , Daniele Ceraolo Spurio , Dave Airlie , David Airlie , Jani Nikula , Joonas Lahtinen , Lucas De Marchi , Matt Roper , Mauro Carvalho Chehab , Rodrigo Vivi , Tvrtko Ursulin , dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, linux-kernel@vger.kernel.org, stable@vger.kernel.org, Fei Yang , =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= Subject: [PATCH 03/21] drm/i915/gt: Invalidate TLB of the OA unit at TLB invalidations Date: Wed, 13 Jul 2022 10:30:00 +0100 Message-Id: <0a56680efc82f69d1651b513e0de077b441e866e.1657703926.git.mchehab@kernel.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Chris Wilson Ensure that the TLB of the OA unit is also invalidated on gen12 HW, as just invalidating the TLB of an engine is not enough. Cc: stable@vger.kernel.org Fixes: 7938d61591d3 ("drm/i915: Flush TLBs before releasing backing store") Signed-off-by: Chris Wilson Cc: Fei Yang Cc: Andi Shyti Acked-by: Thomas Hellstr=C3=B6m Signed-off-by: Mauro Carvalho Chehab --- See [PATCH 00/21] at: https://lore.kernel.org/all/cover.1657703926.git.mche= hab@kernel.org/ drivers/gpu/drm/i915/gt/intel_gt.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/drivers/gpu/drm/i915/gt/intel_gt.c b/drivers/gpu/drm/i915/gt/i= ntel_gt.c index c4d43da84d8e..1d84418e8676 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt.c +++ b/drivers/gpu/drm/i915/gt/intel_gt.c @@ -11,6 +11,7 @@ #include "pxp/intel_pxp.h" =20 #include "i915_drv.h" +#include "i915_perf_oa_regs.h" #include "intel_context.h" #include "intel_engine_pm.h" #include "intel_engine_regs.h" @@ -969,6 +970,15 @@ void intel_gt_invalidate_tlbs(struct intel_gt *gt) awake |=3D engine->mask; } =20 + /* Wa_2207587034:tgl,dg1,rkl,adl-s,adl-p */ + if (awake && + (IS_TIGERLAKE(i915) || + IS_DG1(i915) || + IS_ROCKETLAKE(i915) || + IS_ALDERLAKE_S(i915) || + IS_ALDERLAKE_P(i915))) + intel_uncore_write_fw(uncore, GEN12_OA_TLB_INV_CR, 1); + spin_unlock_irq(&uncore->lock); =20 for_each_engine_masked(engine, gt, awake, tmp) { --=20 2.36.1 From nobody Sat Apr 18 14:11:47 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8F0B9CCA482 for ; Wed, 13 Jul 2022 09:30:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235960AbiGMJa5 (ORCPT ); Wed, 13 Jul 2022 05:30:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43808 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235383AbiGMJaZ (ORCPT ); Wed, 13 Jul 2022 05:30:25 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A1472F2E2E; Wed, 13 Jul 2022 02:30:24 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 4088FB81D62; Wed, 13 Jul 2022 09:30:23 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id DFFDCC3411E; Wed, 13 Jul 2022 09:30:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1657704621; bh=8B8ALfcdC8EwjTlOgjtqiOwcizUYe+bdB6pww0lUJSw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qiuOt/04hD+49hrrU8PsBpLBHnWDBB8euq2/LV+O1tPFgamgy3XhpewKG7QjLzh7q 5g4fJT6rG+llX1qDOZ2xGnZXVA6wFmlYFs1/ISmx4Ha+5W8vc4AZHSwy2C/nYmoDTw kl3mmXGzu4RIlx4yfcwjt2uWKnBHUP+kC3AWhEqQmMs4YTP96We4/VId6Wmysf8T3D /z5PCKLLaeXA8wPLyLVBOcAvEkU/WvrPAnp+qMKvbJgYQsYOQKy7dE/5z/IFPVRexW Q1DNXK9nALdUZuRuxHDN3cTpNVdm31Kg+OAfrypieflPE8kmeiuOfhtk1O1ezN3WBg UO3H7jIBONAeA== Received: from mchehab by mail.kernel.org with local (Exim 4.95) (envelope-from ) id 1oBYhH-0050LL-JV; Wed, 13 Jul 2022 10:30:19 +0100 From: Mauro Carvalho Chehab Cc: Chris Wilson , Daniel Vetter , Dave Airlie , David Airlie , Jani Nikula , Joonas Lahtinen , Rodrigo Vivi , Tvrtko Ursulin , dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, linux-kernel@vger.kernel.org, stable@vger.kernel.org, Fei Yang , Andi Shyti , =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= , Mauro Carvalho Chehab Subject: [PATCH 04/21] drm/i915/gt: Only invalidate TLBs exposed to user manipulation Date: Wed, 13 Jul 2022 10:30:01 +0100 Message-Id: X-Mailer: git-send-email 2.36.1 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Chris Wilson Don't flush TLBs when the buffer is only used in the GGTT under full control of the kernel, as there's no risk of concurrent access and stale access from prefetch. We only need to invalidate the TLB if they are accessible by the user. That helps to reduce the performance regression introduced by TLB invalidate logic. Cc: stable@vger.kernel.org Fixes: 7938d61591d3 ("drm/i915: Flush TLBs before releasing backing store") Signed-off-by: Chris Wilson Cc: Fei Yang Cc: Andi Shyti Acked-by: Thomas Hellstr=C3=B6m Signed-off-by: Mauro Carvalho Chehab --- See [PATCH 00/21] at: https://lore.kernel.org/all/cover.1657703926.git.mche= hab@kernel.org/ drivers/gpu/drm/i915/i915_vma.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/i915/i915_vma.c b/drivers/gpu/drm/i915/i915_vm= a.c index ef3b04c7e153..646f419b2035 100644 --- a/drivers/gpu/drm/i915/i915_vma.c +++ b/drivers/gpu/drm/i915/i915_vma.c @@ -538,7 +538,8 @@ int i915_vma_bind(struct i915_vma *vma, bind_flags); } =20 - set_bit(I915_BO_WAS_BOUND_BIT, &vma->obj->flags); + if (bind_flags & I915_VMA_LOCAL_BIND) + set_bit(I915_BO_WAS_BOUND_BIT, &vma->obj->flags); =20 atomic_or(bind_flags, &vma->flags); return 0; --=20 2.36.1 From nobody Sat Apr 18 14:11:47 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 858D1C433EF for ; Wed, 13 Jul 2022 09:31:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236137AbiGMJbN (ORCPT ); Wed, 13 Jul 2022 05:31:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43858 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235456AbiGMJa0 (ORCPT ); Wed, 13 Jul 2022 05:30:26 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E9924EA14B; Wed, 13 Jul 2022 02:30:24 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 5BA04B81D63; Wed, 13 Jul 2022 09:30:23 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E2CF2C341C6; Wed, 13 Jul 2022 09:30:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1657704621; bh=YJNkVNAUvK5StDP52pIbkShfQ+2J49Rn1f/TmIabGZI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WuyDvdbdi94Eg3JERcGkhWtGT6aN6uWBEKdOWsE4QjEKyrsCaP/qcdcfjogVtBDGU bI1YrxRVQGJkv8JugiM7yrW/E3ZbkTZbEpf9LvOU3Anqeh4Lwj1xva7ENM5mVyvnB7 2plsUVI42lUlXvTgCt4kFhWzmJsYbicyZ5rUKfMMvi0AZY8BTUtjki3VvHliZ0tr1O Nm7RXnewrhbtYopFhntoNRlKYE8GvDAdl7i/YdyeLrOkFd58PzYXKVmpK9qIPvSGUS 2QNUbaKF36wbj6c+81rtg+op9NSPFoAMgy0YvVEdkedT7f5TtZTLDCJB7RNbzHYYHB mN8dWUTGrSYFg== Received: from mchehab by mail.kernel.org with local (Exim 4.95) (envelope-from ) id 1oBYhH-0050LO-KC; Wed, 13 Jul 2022 10:30:19 +0100 From: Mauro Carvalho Chehab Cc: Chris Wilson , Andi Shyti , Daniel Vetter , Daniele Ceraolo Spurio , Dave Airlie , David Airlie , Jani Nikula , Joonas Lahtinen , Lucas De Marchi , Matt Roper , Mauro Carvalho Chehab , Rodrigo Vivi , Tvrtko Ursulin , dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, linux-kernel@vger.kernel.org, stable@vger.kernel.org, Fei Yang , =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= Subject: [PATCH 05/21] drm/i915/gt: Skip TLB invalidations once wedged Date: Wed, 13 Jul 2022 10:30:02 +0100 Message-Id: <1af2335658aacc73cdbbe26e041d065e198e6870.1657703926.git.mchehab@kernel.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Chris Wilson Skip all further TLB invalidations once the device is wedged and had been reset, as, on such cases, it can no longer process instructions on the GPU and the user no longer has access to the TLB's in each engine. That helps to reduce the performance regression introduced by TLB invalidate logic. Cc: stable@vger.kernel.org Fixes: 7938d61591d3 ("drm/i915: Flush TLBs before releasing backing store") Signed-off-by: Chris Wilson Cc: Fei Yang Cc: Andi Shyti Acked-by: Thomas Hellstr=C3=B6m Signed-off-by: Mauro Carvalho Chehab --- See [PATCH 00/21] at: https://lore.kernel.org/all/cover.1657703926.git.mche= hab@kernel.org/ drivers/gpu/drm/i915/gt/intel_gt.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/gpu/drm/i915/gt/intel_gt.c b/drivers/gpu/drm/i915/gt/i= ntel_gt.c index 1d84418e8676..5c55a90672f4 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt.c +++ b/drivers/gpu/drm/i915/gt/intel_gt.c @@ -934,6 +934,9 @@ void intel_gt_invalidate_tlbs(struct intel_gt *gt) if (I915_SELFTEST_ONLY(gt->awake =3D=3D -ENODEV)) return; =20 + if (intel_gt_is_wedged(gt)) + return; + if (GRAPHICS_VER(i915) =3D=3D 12) { regs =3D gen12_regs; num =3D ARRAY_SIZE(gen12_regs); --=20 2.36.1 From nobody Sat Apr 18 14:11:47 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 91C4FC43334 for ; Wed, 13 Jul 2022 09:31:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236208AbiGMJbn (ORCPT ); Wed, 13 Jul 2022 05:31:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43860 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235710AbiGMJag (ORCPT ); Wed, 13 Jul 2022 05:30:36 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EF024F4222; Wed, 13 Jul 2022 02:30:26 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id B5212B81D77; Wed, 13 Jul 2022 09:30:24 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 06BE8C341CE; Wed, 13 Jul 2022 09:30:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1657704622; bh=4kra6JYeTpbloCB6kM7O8tizc0ckPfouvDEAsME0I7s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NemMJha7tWvDKcj4DC+bd1y8XTLHj9TvZ1LtAyIdVc2dYXFHJmeKaK1iyBjQCRxy3 rzAVThpXORItJO06UWTeBO2a8iDjKWrQu1+fFkJ6vVutOepGgqaTFx/MsfztuNf2+N V1CELjToakFHpe+JRkFCCcjdjk8dW0Rk2R0WqJpy4+JNRtpTetKJu4WpLWtehune/8 u/bNjpmGKEWv6selcxgSwrI5NW8ZbBbdnvriSS0fY42OQVD6e2C0QFd1oaDFH8CWLI IkMKIwXVn5Qt/rAQ2KIEWws7KbJLeGqP4N2E2zrLGiCHjTA5/1nqSgN1TKDcaO0bmS lgv3n92bNscbw== Received: from mchehab by mail.kernel.org with local (Exim 4.95) (envelope-from ) id 1oBYhH-0050LR-L7; Wed, 13 Jul 2022 10:30:19 +0100 From: Mauro Carvalho Chehab Cc: Chris Wilson , =?UTF-8?q?Christian=20K=C3=B6nig?= , =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= , Abdiel Janulgue , Andi Shyti , Andrzej Hajda , Ayaz A Siddiqui , Casey Bowman , Daniel Vetter , Daniele Ceraolo Spurio , Dave Airlie , David Airlie , Jani Nikula , Jason Ekstrand , John Harrison , Joonas Lahtinen , Lucas De Marchi , Maarten Lankhorst , Matt Roper , Matthew Auld , Mauro Carvalho Chehab , Prathap Kumar Valsan , Rodrigo Vivi , Sumit Semwal , Tomas Winkler , Tvrtko Ursulin , dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, stable@vger.kernel.org, Tvrtko Ursulin , Fei Yang Subject: [PATCH 06/21] drm/i915/gt: Batch TLB invalidations Date: Wed, 13 Jul 2022 10:30:03 +0100 Message-Id: X-Mailer: git-send-email 2.36.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Chris Wilson Invalidate TLB in patch, in order to reduce performance regressions. Currently, every caller performs a full barrier around a TLB invalidation, ignoring all other invalidations that may have already removed their PTEs from the cache. As this is a synchronous operation and can be quite slow, we cause multiple threads to contend on the TLB invalidate mutex blocking userspace. We only need to invalidate the TLB once after replacing our PTE to ensure that there is no possible continued access to the physical address before releasing our pages. By tracking a seqno for each full TLB invalidate we can quickly determine if one has been performed since rewriting the PTE, and only if necessary trigger one for ourselves. That helps to reduce the performance regression introduced by TLB invalidate logic. [mchehab: rebased to not require moving the code to a separate file] Cc: stable@vger.kernel.org Fixes: 7938d61591d3 ("drm/i915: Flush TLBs before releasing backing store") Suggested-by: Tvrtko Ursulin Signed-off-by: Chris Wilson Cc: Fei Yang Signed-off-by: Mauro Carvalho Chehab --- See [PATCH 00/21] at: https://lore.kernel.org/all/cover.1657703926.git.mche= hab@kernel.org/ .../gpu/drm/i915/gem/i915_gem_object_types.h | 3 +- drivers/gpu/drm/i915/gem/i915_gem_pages.c | 21 +++++--- drivers/gpu/drm/i915/gt/intel_gt.c | 53 ++++++++++++++----- drivers/gpu/drm/i915/gt/intel_gt.h | 12 ++++- drivers/gpu/drm/i915/gt/intel_gt_types.h | 18 ++++++- drivers/gpu/drm/i915/gt/intel_ppgtt.c | 8 ++- drivers/gpu/drm/i915/i915_vma.c | 34 +++++++++--- drivers/gpu/drm/i915/i915_vma.h | 1 + drivers/gpu/drm/i915/i915_vma_resource.c | 5 +- drivers/gpu/drm/i915/i915_vma_resource.h | 6 ++- 10 files changed, 125 insertions(+), 36 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object_types.h b/drivers/gpu= /drm/i915/gem/i915_gem_object_types.h index 5cf36a130061..9f6b14ec189a 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_object_types.h +++ b/drivers/gpu/drm/i915/gem/i915_gem_object_types.h @@ -335,7 +335,6 @@ struct drm_i915_gem_object { #define I915_BO_READONLY BIT(7) #define I915_TILING_QUIRK_BIT 8 /* unknown swizzling; do not release! = */ #define I915_BO_PROTECTED BIT(9) -#define I915_BO_WAS_BOUND_BIT 10 /** * @mem_flags - Mutable placement-related flags * @@ -616,6 +615,8 @@ struct drm_i915_gem_object { * pages were last acquired. */ bool dirty:1; + + u32 tlb; } mm; =20 struct { diff --git a/drivers/gpu/drm/i915/gem/i915_gem_pages.c b/drivers/gpu/drm/i9= 15/gem/i915_gem_pages.c index 6835279943df..8357dbdcab5c 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_pages.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_pages.c @@ -191,6 +191,18 @@ static void unmap_object(struct drm_i915_gem_object *o= bj, void *ptr) vunmap(ptr); } =20 +static void flush_tlb_invalidate(struct drm_i915_gem_object *obj) +{ + struct drm_i915_private *i915 =3D to_i915(obj->base.dev); + struct intel_gt *gt =3D to_gt(i915); + + if (!obj->mm.tlb) + return; + + intel_gt_invalidate_tlb(gt, obj->mm.tlb); + obj->mm.tlb =3D 0; +} + struct sg_table * __i915_gem_object_unset_pages(struct drm_i915_gem_object *obj) { @@ -216,14 +228,7 @@ __i915_gem_object_unset_pages(struct drm_i915_gem_obje= ct *obj) __i915_gem_object_reset_page_iter(obj); obj->mm.page_sizes.phys =3D obj->mm.page_sizes.sg =3D 0; =20 - if (test_and_clear_bit(I915_BO_WAS_BOUND_BIT, &obj->flags)) { - struct drm_i915_private *i915 =3D to_i915(obj->base.dev); - struct intel_gt *gt =3D to_gt(i915); - intel_wakeref_t wakeref; - - with_intel_gt_pm_if_awake(gt, wakeref) - intel_gt_invalidate_tlbs(gt); - } + flush_tlb_invalidate(obj); =20 return pages; } diff --git a/drivers/gpu/drm/i915/gt/intel_gt.c b/drivers/gpu/drm/i915/gt/i= ntel_gt.c index 5c55a90672f4..f435e06125aa 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt.c +++ b/drivers/gpu/drm/i915/gt/intel_gt.c @@ -38,8 +38,6 @@ static void __intel_gt_init_early(struct intel_gt *gt) { spin_lock_init(>->irq_lock); =20 - mutex_init(>->tlb_invalidate_lock); - INIT_LIST_HEAD(>->closed_vma); spin_lock_init(>->closed_lock); =20 @@ -50,6 +48,8 @@ static void __intel_gt_init_early(struct intel_gt *gt) intel_gt_init_reset(gt); intel_gt_init_requests(gt); intel_gt_init_timelines(gt); + mutex_init(>->tlb.invalidate_lock); + seqcount_mutex_init(>->tlb.seqno, >->tlb.invalidate_lock); intel_gt_pm_init_early(gt); =20 intel_uc_init_early(>->uc); @@ -770,6 +770,7 @@ void intel_gt_driver_late_release_all(struct drm_i915_p= rivate *i915) intel_gt_fini_requests(gt); intel_gt_fini_reset(gt); intel_gt_fini_timelines(gt); + mutex_destroy(>->tlb.invalidate_lock); intel_engines_free(gt); } } @@ -908,7 +909,7 @@ get_reg_and_bit(const struct intel_engine_cs *engine, c= onst bool gen8, return rb; } =20 -void intel_gt_invalidate_tlbs(struct intel_gt *gt) +static void mmio_invalidate_full(struct intel_gt *gt) { static const i915_reg_t gen8_regs[] =3D { [RENDER_CLASS] =3D GEN8_RTCR, @@ -931,12 +932,6 @@ void intel_gt_invalidate_tlbs(struct intel_gt *gt) const i915_reg_t *regs; unsigned int num =3D 0; =20 - if (I915_SELFTEST_ONLY(gt->awake =3D=3D -ENODEV)) - return; - - if (intel_gt_is_wedged(gt)) - return; - if (GRAPHICS_VER(i915) =3D=3D 12) { regs =3D gen12_regs; num =3D ARRAY_SIZE(gen12_regs); @@ -951,9 +946,6 @@ void intel_gt_invalidate_tlbs(struct intel_gt *gt) "Platform does not implement TLB invalidation!")) return; =20 - GEM_TRACE("\n"); - - mutex_lock(>->tlb_invalidate_lock); intel_uncore_forcewake_get(uncore, FORCEWAKE_ALL); =20 spin_lock_irq(&uncore->lock); /* serialise invalidate with GT reset */ @@ -973,6 +965,8 @@ void intel_gt_invalidate_tlbs(struct intel_gt *gt) awake |=3D engine->mask; } =20 + GT_TRACE(gt, "invalidated engines %08x\n", awake); + /* Wa_2207587034:tgl,dg1,rkl,adl-s,adl-p */ if (awake && (IS_TIGERLAKE(i915) || @@ -1012,5 +1006,38 @@ void intel_gt_invalidate_tlbs(struct intel_gt *gt) * transitions. */ intel_uncore_forcewake_put_delayed(uncore, FORCEWAKE_ALL); - mutex_unlock(>->tlb_invalidate_lock); +} + +static bool tlb_seqno_passed(const struct intel_gt *gt, u32 seqno) +{ + u32 cur =3D intel_gt_tlb_seqno(gt); + + /* Only skip if a *full* TLB invalidate barrier has passed */ + return (s32)(cur - ALIGN(seqno, 2)) > 0; +} + +void intel_gt_invalidate_tlb(struct intel_gt *gt, u32 seqno) +{ + intel_wakeref_t wakeref; + + if (I915_SELFTEST_ONLY(gt->awake =3D=3D -ENODEV)) + return; + + if (intel_gt_is_wedged(gt)) + return; + + if (tlb_seqno_passed(gt, seqno)) + return; + + with_intel_gt_pm_if_awake(gt, wakeref) { + mutex_lock(>->tlb.invalidate_lock); + if (tlb_seqno_passed(gt, seqno)) + goto unlock; + + mmio_invalidate_full(gt); + + write_seqcount_invalidate(>->tlb.seqno); +unlock: + mutex_unlock(>->tlb.invalidate_lock); + } } diff --git a/drivers/gpu/drm/i915/gt/intel_gt.h b/drivers/gpu/drm/i915/gt/i= ntel_gt.h index 82d6f248d876..40b06adf509a 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt.h +++ b/drivers/gpu/drm/i915/gt/intel_gt.h @@ -101,6 +101,16 @@ void intel_gt_info_print(const struct intel_gt_info *i= nfo, =20 void intel_gt_watchdog_work(struct work_struct *work); =20 -void intel_gt_invalidate_tlbs(struct intel_gt *gt); +static inline u32 intel_gt_tlb_seqno(const struct intel_gt *gt) +{ + return seqprop_sequence(>->tlb.seqno); +} + +static inline u32 intel_gt_next_invalidate_tlb_full(const struct intel_gt = *gt) +{ + return intel_gt_tlb_seqno(gt) | 1; +} + +void intel_gt_invalidate_tlb(struct intel_gt *gt, u32 seqno); =20 #endif /* __INTEL_GT_H__ */ diff --git a/drivers/gpu/drm/i915/gt/intel_gt_types.h b/drivers/gpu/drm/i91= 5/gt/intel_gt_types.h index df708802889d..3804a583382b 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt_types.h +++ b/drivers/gpu/drm/i915/gt/intel_gt_types.h @@ -11,6 +11,7 @@ #include #include #include +#include #include #include #include @@ -83,7 +84,22 @@ struct intel_gt { struct intel_uc uc; struct intel_gsc gsc; =20 - struct mutex tlb_invalidate_lock; + struct { + /* Serialize global tlb invalidations */ + struct mutex invalidate_lock; + + /* + * Batch TLB invalidations + * + * After unbinding the PTE, we need to ensure the TLB + * are invalidated prior to releasing the physical pages. + * But we only need one such invalidation for all unbinds, + * so we track how many TLB invalidations have been + * performed since unbind the PTE and only emit an extra + * invalidate if no full barrier has been passed. + */ + seqcount_mutex_t seqno; + } tlb; =20 struct i915_wa_list wa_list; =20 diff --git a/drivers/gpu/drm/i915/gt/intel_ppgtt.c b/drivers/gpu/drm/i915/g= t/intel_ppgtt.c index d8b94d638559..2da6c82a8bd2 100644 --- a/drivers/gpu/drm/i915/gt/intel_ppgtt.c +++ b/drivers/gpu/drm/i915/gt/intel_ppgtt.c @@ -206,8 +206,12 @@ void ppgtt_bind_vma(struct i915_address_space *vm, void ppgtt_unbind_vma(struct i915_address_space *vm, struct i915_vma_resource *vma_res) { - if (vma_res->allocated) - vm->clear_range(vm, vma_res->start, vma_res->vma_size); + if (!vma_res->allocated) + return; + + vm->clear_range(vm, vma_res->start, vma_res->vma_size); + if (vma_res->tlb) + vma_invalidate_tlb(vm, *vma_res->tlb); } =20 static unsigned long pd_count(u64 size, int shift) diff --git a/drivers/gpu/drm/i915/i915_vma.c b/drivers/gpu/drm/i915/i915_vm= a.c index 646f419b2035..84a9ccbc5fc5 100644 --- a/drivers/gpu/drm/i915/i915_vma.c +++ b/drivers/gpu/drm/i915/i915_vma.c @@ -538,9 +538,6 @@ int i915_vma_bind(struct i915_vma *vma, bind_flags); } =20 - if (bind_flags & I915_VMA_LOCAL_BIND) - set_bit(I915_BO_WAS_BOUND_BIT, &vma->obj->flags); - atomic_or(bind_flags, &vma->flags); return 0; } @@ -1311,6 +1308,19 @@ I915_SELFTEST_EXPORT int i915_vma_get_pages(struct i= 915_vma *vma) return err; } =20 +void vma_invalidate_tlb(struct i915_address_space *vm, u32 tlb) +{ + /* + * Before we release the pages that were bound by this vma, we + * must invalidate all the TLBs that may still have a reference + * back to our physical address. It only needs to be done once, + * so after updating the PTE to point away from the pages, record + * the most recent TLB invalidation seqno, and if we have not yet + * flushed the TLBs upon release, perform a full invalidation. + */ + WRITE_ONCE(tlb, intel_gt_next_invalidate_tlb_full(vm->gt)); +} + static void __vma_put_pages(struct i915_vma *vma, unsigned int count) { /* We allocate under vma_get_pages, so beware the shrinker */ @@ -1942,7 +1952,12 @@ struct dma_fence *__i915_vma_evict(struct i915_vma *= vma, bool async) vma->vm->skip_pte_rewrite; trace_i915_vma_unbind(vma); =20 - unbind_fence =3D i915_vma_resource_unbind(vma_res); + if (async) + unbind_fence =3D i915_vma_resource_unbind(vma_res, + &vma->obj->mm.tlb); + else + unbind_fence =3D i915_vma_resource_unbind(vma_res, NULL); + vma->resource =3D NULL; =20 atomic_and(~(I915_VMA_BIND_MASK | I915_VMA_ERROR | I915_VMA_GGTT_WRITE), @@ -1950,10 +1965,13 @@ struct dma_fence *__i915_vma_evict(struct i915_vma = *vma, bool async) =20 i915_vma_detach(vma); =20 - if (!async && unbind_fence) { - dma_fence_wait(unbind_fence, false); - dma_fence_put(unbind_fence); - unbind_fence =3D NULL; + if (!async) { + if (unbind_fence) { + dma_fence_wait(unbind_fence, false); + dma_fence_put(unbind_fence); + unbind_fence =3D NULL; + } + vma_invalidate_tlb(vma->vm, vma->obj->mm.tlb); } =20 /* diff --git a/drivers/gpu/drm/i915/i915_vma.h b/drivers/gpu/drm/i915/i915_vm= a.h index 88ca0bd9c900..5048eed536da 100644 --- a/drivers/gpu/drm/i915/i915_vma.h +++ b/drivers/gpu/drm/i915/i915_vma.h @@ -213,6 +213,7 @@ bool i915_vma_misplaced(const struct i915_vma *vma, u64 size, u64 alignment, u64 flags); void __i915_vma_set_map_and_fenceable(struct i915_vma *vma); void i915_vma_revoke_mmap(struct i915_vma *vma); +void vma_invalidate_tlb(struct i915_address_space *vm, u32 tlb); struct dma_fence *__i915_vma_evict(struct i915_vma *vma, bool async); int __i915_vma_unbind(struct i915_vma *vma); int __must_check i915_vma_unbind(struct i915_vma *vma); diff --git a/drivers/gpu/drm/i915/i915_vma_resource.c b/drivers/gpu/drm/i91= 5/i915_vma_resource.c index 27c55027387a..5a67995ea5fe 100644 --- a/drivers/gpu/drm/i915/i915_vma_resource.c +++ b/drivers/gpu/drm/i915/i915_vma_resource.c @@ -223,10 +223,13 @@ i915_vma_resource_fence_notify(struct i915_sw_fence *= fence, * Return: A refcounted pointer to a dma-fence that signals when unbinding= is * complete. */ -struct dma_fence *i915_vma_resource_unbind(struct i915_vma_resource *vma_r= es) +struct dma_fence *i915_vma_resource_unbind(struct i915_vma_resource *vma_r= es, + u32 *tlb) { struct i915_address_space *vm =3D vma_res->vm; =20 + vma_res->tlb =3D tlb; + /* Reference for the sw fence */ i915_vma_resource_get(vma_res); =20 diff --git a/drivers/gpu/drm/i915/i915_vma_resource.h b/drivers/gpu/drm/i91= 5/i915_vma_resource.h index 5d8427caa2ba..06923d1816e7 100644 --- a/drivers/gpu/drm/i915/i915_vma_resource.h +++ b/drivers/gpu/drm/i915/i915_vma_resource.h @@ -67,6 +67,7 @@ struct i915_page_sizes { * taken when the unbind is scheduled. * @skip_pte_rewrite: During ggtt suspend and vm takedown pte rewriting * needs to be skipped for unbind. + * @tlb: pointer for obj->mm.tlb, if async unbind. Otherwise, NULL * * The lifetime of a struct i915_vma_resource is from a binding request to * the actual possible asynchronous unbind has completed. @@ -119,6 +120,8 @@ struct i915_vma_resource { bool immediate_unbind:1; bool needs_wakeref:1; bool skip_pte_rewrite:1; + + u32 *tlb; }; =20 bool i915_vma_resource_hold(struct i915_vma_resource *vma_res, @@ -131,7 +134,8 @@ struct i915_vma_resource *i915_vma_resource_alloc(void); =20 void i915_vma_resource_free(struct i915_vma_resource *vma_res); =20 -struct dma_fence *i915_vma_resource_unbind(struct i915_vma_resource *vma_r= es); +struct dma_fence *i915_vma_resource_unbind(struct i915_vma_resource *vma_r= es, + u32 *tlb); =20 void __i915_vma_resource_init(struct i915_vma_resource *vma_res); =20 --=20 2.36.1 From nobody Sat Apr 18 14:11:47 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 10993C43334 for ; Wed, 13 Jul 2022 09:30:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235210AbiGMJa3 (ORCPT ); Wed, 13 Jul 2022 05:30:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43720 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235030AbiGMJaX (ORCPT ); Wed, 13 Jul 2022 05:30:23 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 02D79EEA9C for ; Wed, 13 Jul 2022 02:30:23 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 9081661C63 for ; Wed, 13 Jul 2022 09:30:22 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E505FC341C8; Wed, 13 Jul 2022 09:30:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1657704621; bh=smG9F2RXMNXCU8+/rdRMph25oHjT7Q2WcPhpkWuHDgo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=P7K0y2Z9Vs9n9sU15b+DrTXWHH58JYmo2jAunLW/rw5GihE1pJ/RFTReKKJi/f9yG uBpNNO7K7BzKqbIrHuLvMYV6mKtFa0zzNpsjk7VtUWiXwn6NFvKzNKds6GdPY81ODj S/bfgZ4h861narp4RQeYoO7IxDDVMotcyJl3B5ICckeg32oP7jPAyUtSxHO3gjWqsw PfUIfpQpr/EYE2Je0P5IG5lXOcSBwgjOt7J/Ah49rLc+4c1Fmv+FkBdaVH3XI5UCFO d7sT0aUK09aIRAA8huPew1Fb+xwP7ZT4h0Jd+qiEqSgYZ3DyN8XX1zWN6e5s4SWuRw CTqj0DXT8WOSA== Received: from mchehab by mail.kernel.org with local (Exim 4.95) (envelope-from ) id 1oBYhH-0050LU-Lr; Wed, 13 Jul 2022 10:30:19 +0100 From: Mauro Carvalho Chehab Cc: Mauro Carvalho Chehab , Daniel Vetter , David Airlie , Jani Nikula , Joonas Lahtinen , Rodrigo Vivi , Tvrtko Ursulin , dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, linux-kernel@vger.kernel.org Subject: [PATCH 07/21] drm/i915/gt: describe the new tlb parameter at i915_vma_resource Date: Wed, 13 Jul 2022 10:30:04 +0100 Message-Id: <29c9a422328bc7c644dbc1e83a28f09e502d34d6.1657703926.git.mchehab@kernel.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" TLB cache invalidation can happen on two different situations: 1. synchronously, at __vma_put_pages(); 2. asynchronously. On the first case, TLB cache invalidation happens inside __vma_put_pages(). So, no need to do it later on. However, on the second case, the pages will keep in memory until __i915_vma_evict() is called. So, we need to store the TLB data at struct i915_vma_resource, in order to do a TLB cache invalidation before allowing userspace to re-use the same memory. So, i915_vma_resource_unbind() has gained a new parameter in order to store the TLB data at the second case. Document it. Signed-off-by: Mauro Carvalho Chehab --- See [PATCH 00/21] at: https://lore.kernel.org/all/cover.1657703926.git.mche= hab@kernel.org/ drivers/gpu/drm/i915/i915_vma_resource.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/gpu/drm/i915/i915_vma_resource.c b/drivers/gpu/drm/i91= 5/i915_vma_resource.c index 5a67995ea5fe..4fe09ea0a825 100644 --- a/drivers/gpu/drm/i915/i915_vma_resource.c +++ b/drivers/gpu/drm/i915/i915_vma_resource.c @@ -216,6 +216,10 @@ i915_vma_resource_fence_notify(struct i915_sw_fence *f= ence, /** * i915_vma_resource_unbind - Unbind a vma resource * @vma_res: The vma resource to unbind. + * @tlb: pointer to vma->obj->mm.tlb associated with the resource + * to be stored at vma_res->tlb. When not-NULL, it will be used + * to do TLB cache invalidation before freeing a VMA resource. + * used only for async unbind. * * At this point this function does little more than publish a fence that * signals immediately unless signaling is held back. --=20 2.36.1 From nobody Sat Apr 18 14:11:47 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E9FAC433EF for ; Wed, 13 Jul 2022 09:31:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236106AbiGMJbh (ORCPT ); Wed, 13 Jul 2022 05:31:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44218 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235715AbiGMJag (ORCPT ); Wed, 13 Jul 2022 05:30:36 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EEF09F421B for ; Wed, 13 Jul 2022 02:30:26 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id EF9B9B81D79 for ; Wed, 13 Jul 2022 09:30:24 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 144D5C341CF; Wed, 13 Jul 2022 09:30:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1657704622; bh=4SyRnJAg34ibNLvFHYlDtmr/HWuPBiGZKsyFILogPEo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PVlDy0yocMPcpGSKVou4jQyLUEbqYN3P5mded5lyz/DNnqVUuUs49EBDlfrSK1/27 bempnHCev0vlGgXlieSxuVtQUGCwQc4TY+ap31ESK6Ug64U0okxvj9uiDYrQgWQH2H u01ysvV+RBEncFemMZ1tshxtxFr1//9G1MTRvofxhpbrvAVYjZmV7NQivLJxbTJpB7 W3oAHkkOB2kxDM09K48SO+TuhzHSuKOmo2FyVgT3Hdkcym05EjQe+KgWv9RPQxho5I F9gGrzKwCRot3gsIiB4GqYEKpp0K8Lfp99prbKEVPoO7PrJf42J/JPYdU3QhAwxTYE L/XdsnL48niSA== Received: from mchehab by mail.kernel.org with local (Exim 4.95) (envelope-from ) id 1oBYhH-0050LX-Me; Wed, 13 Jul 2022 10:30:19 +0100 From: Mauro Carvalho Chehab Cc: Chris Wilson , =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= , Andi Shyti , Casey Bowman , Daniel Vetter , Daniele Ceraolo Spurio , David Airlie , Jani Nikula , Jason Ekstrand , Joonas Lahtinen , Lucas De Marchi , Maarten Lankhorst , Matt Roper , Matthew Auld , Matthew Brost , Mauro Carvalho Chehab , Prathap Kumar Valsan , Rodrigo Vivi , Tomas Winkler , Tvrtko Ursulin , dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, linux-kernel@vger.kernel.org, Fei Yang Subject: [PATCH 08/21] drm/i915/gt: Move TLB invalidation to its own file Date: Wed, 13 Jul 2022 10:30:05 +0100 Message-Id: <77b408e130cf8a0461a7d929ce5b5380305cad9c.1657703926.git.mchehab@kernel.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Chris Wilson Prepare for supporting more TLB invalidation scenarios by moving the current MMIO invalidation to its own file. Signed-off-by: Chris Wilson Cc: Fei Yang Signed-off-by: Mauro Carvalho Chehab --- See [PATCH 00/21] at: https://lore.kernel.org/all/cover.1657703926.git.mche= hab@kernel.org/ drivers/gpu/drm/i915/Makefile | 1 + drivers/gpu/drm/i915/gem/i915_gem_pages.c | 4 +- drivers/gpu/drm/i915/gt/intel_gt.c | 168 +------------------- drivers/gpu/drm/i915/gt/intel_gt.h | 12 -- drivers/gpu/drm/i915/gt/intel_tlb.c | 183 ++++++++++++++++++++++ drivers/gpu/drm/i915/gt/intel_tlb.h | 29 ++++ drivers/gpu/drm/i915/i915_vma.c | 1 + 7 files changed, 219 insertions(+), 179 deletions(-) create mode 100644 drivers/gpu/drm/i915/gt/intel_tlb.c create mode 100644 drivers/gpu/drm/i915/gt/intel_tlb.h diff --git a/drivers/gpu/drm/i915/Makefile b/drivers/gpu/drm/i915/Makefile index 522ef9b4aff3..d3df9832d1f7 100644 --- a/drivers/gpu/drm/i915/Makefile +++ b/drivers/gpu/drm/i915/Makefile @@ -126,6 +126,7 @@ gt-y +=3D \ gt/intel_sseu.o \ gt/intel_sseu_debugfs.o \ gt/intel_timeline.o \ + gt/intel_tlb.o \ gt/intel_workarounds.o \ gt/shmem_utils.o \ gt/sysfs_engines.o diff --git a/drivers/gpu/drm/i915/gem/i915_gem_pages.c b/drivers/gpu/drm/i9= 15/gem/i915_gem_pages.c index 8357dbdcab5c..1cd76cc5d9f3 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_pages.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_pages.c @@ -7,7 +7,7 @@ #include =20 #include "gt/intel_gt.h" -#include "gt/intel_gt_pm.h" +#include "gt/intel_tlb.h" =20 #include "i915_drv.h" #include "i915_gem_object.h" @@ -199,7 +199,7 @@ static void flush_tlb_invalidate(struct drm_i915_gem_ob= ject *obj) if (!obj->mm.tlb) return; =20 - intel_gt_invalidate_tlb(gt, obj->mm.tlb); + intel_gt_invalidate_tlb_full(gt, obj->mm.tlb); obj->mm.tlb =3D 0; } =20 diff --git a/drivers/gpu/drm/i915/gt/intel_gt.c b/drivers/gpu/drm/i915/gt/i= ntel_gt.c index f435e06125aa..18d82cd620bd 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt.c +++ b/drivers/gpu/drm/i915/gt/intel_gt.c @@ -11,9 +11,7 @@ #include "pxp/intel_pxp.h" =20 #include "i915_drv.h" -#include "i915_perf_oa_regs.h" #include "intel_context.h" -#include "intel_engine_pm.h" #include "intel_engine_regs.h" #include "intel_ggtt_gmch.h" #include "intel_gt.h" @@ -31,6 +29,7 @@ #include "intel_renderstate.h" #include "intel_rps.h" #include "intel_gt_sysfs.h" +#include "intel_tlb.h" #include "intel_uncore.h" #include "shmem_utils.h" =20 @@ -48,8 +47,7 @@ static void __intel_gt_init_early(struct intel_gt *gt) intel_gt_init_reset(gt); intel_gt_init_requests(gt); intel_gt_init_timelines(gt); - mutex_init(>->tlb.invalidate_lock); - seqcount_mutex_init(>->tlb.seqno, >->tlb.invalidate_lock); + intel_gt_init_tlb(gt); intel_gt_pm_init_early(gt); =20 intel_uc_init_early(>->uc); @@ -770,7 +768,7 @@ void intel_gt_driver_late_release_all(struct drm_i915_p= rivate *i915) intel_gt_fini_requests(gt); intel_gt_fini_reset(gt); intel_gt_fini_timelines(gt); - mutex_destroy(>->tlb.invalidate_lock); + intel_gt_fini_tlb(gt); intel_engines_free(gt); } } @@ -881,163 +879,3 @@ void intel_gt_info_print(const struct intel_gt_info *= info, =20 intel_sseu_dump(&info->sseu, p); } - -struct reg_and_bit { - i915_reg_t reg; - u32 bit; -}; - -static struct reg_and_bit -get_reg_and_bit(const struct intel_engine_cs *engine, const bool gen8, - const i915_reg_t *regs, const unsigned int num) -{ - const unsigned int class =3D engine->class; - struct reg_and_bit rb =3D { }; - - if (drm_WARN_ON_ONCE(&engine->i915->drm, - class >=3D num || !regs[class].reg)) - return rb; - - rb.reg =3D regs[class]; - if (gen8 && class =3D=3D VIDEO_DECODE_CLASS) - rb.reg.reg +=3D 4 * engine->instance; /* GEN8_M2TCR */ - else - rb.bit =3D engine->instance; - - rb.bit =3D BIT(rb.bit); - - return rb; -} - -static void mmio_invalidate_full(struct intel_gt *gt) -{ - static const i915_reg_t gen8_regs[] =3D { - [RENDER_CLASS] =3D GEN8_RTCR, - [VIDEO_DECODE_CLASS] =3D GEN8_M1TCR, /* , GEN8_M2TCR */ - [VIDEO_ENHANCEMENT_CLASS] =3D GEN8_VTCR, - [COPY_ENGINE_CLASS] =3D GEN8_BTCR, - }; - static const i915_reg_t gen12_regs[] =3D { - [RENDER_CLASS] =3D GEN12_GFX_TLB_INV_CR, - [VIDEO_DECODE_CLASS] =3D GEN12_VD_TLB_INV_CR, - [VIDEO_ENHANCEMENT_CLASS] =3D GEN12_VE_TLB_INV_CR, - [COPY_ENGINE_CLASS] =3D GEN12_BLT_TLB_INV_CR, - [COMPUTE_CLASS] =3D GEN12_COMPCTX_TLB_INV_CR, - }; - struct drm_i915_private *i915 =3D gt->i915; - struct intel_uncore *uncore =3D gt->uncore; - struct intel_engine_cs *engine; - intel_engine_mask_t awake, tmp; - enum intel_engine_id id; - const i915_reg_t *regs; - unsigned int num =3D 0; - - if (GRAPHICS_VER(i915) =3D=3D 12) { - regs =3D gen12_regs; - num =3D ARRAY_SIZE(gen12_regs); - } else if (GRAPHICS_VER(i915) >=3D 8 && GRAPHICS_VER(i915) <=3D 11) { - regs =3D gen8_regs; - num =3D ARRAY_SIZE(gen8_regs); - } else if (GRAPHICS_VER(i915) < 8) { - return; - } - - if (drm_WARN_ONCE(&i915->drm, !num, - "Platform does not implement TLB invalidation!")) - return; - - intel_uncore_forcewake_get(uncore, FORCEWAKE_ALL); - - spin_lock_irq(&uncore->lock); /* serialise invalidate with GT reset */ - - awake =3D 0; - for_each_engine(engine, gt, id) { - struct reg_and_bit rb; - - if (!intel_engine_pm_is_awake(engine)) - continue; - - rb =3D get_reg_and_bit(engine, regs =3D=3D gen8_regs, regs, num); - if (!i915_mmio_reg_offset(rb.reg)) - continue; - - intel_uncore_write_fw(uncore, rb.reg, rb.bit); - awake |=3D engine->mask; - } - - GT_TRACE(gt, "invalidated engines %08x\n", awake); - - /* Wa_2207587034:tgl,dg1,rkl,adl-s,adl-p */ - if (awake && - (IS_TIGERLAKE(i915) || - IS_DG1(i915) || - IS_ROCKETLAKE(i915) || - IS_ALDERLAKE_S(i915) || - IS_ALDERLAKE_P(i915))) - intel_uncore_write_fw(uncore, GEN12_OA_TLB_INV_CR, 1); - - spin_unlock_irq(&uncore->lock); - - for_each_engine_masked(engine, gt, awake, tmp) { - struct reg_and_bit rb; - - /* - * HW architecture suggest typical invalidation time at 40us, - * with pessimistic cases up to 100us and a recommendation to - * cap at 1ms. We go a bit higher just in case. - */ - const unsigned int timeout_us =3D 100; - const unsigned int timeout_ms =3D 4; - - rb =3D get_reg_and_bit(engine, regs =3D=3D gen8_regs, regs, num); - if (__intel_wait_for_register_fw(uncore, - rb.reg, rb.bit, 0, - timeout_us, timeout_ms, - NULL)) - drm_err_ratelimited(>->i915->drm, - "%s TLB invalidation did not complete in %ums!\n", - engine->name, timeout_ms); - } - - /* - * Use delayed put since a) we mostly expect a flurry of TLB - * invalidations so it is good to avoid paying the forcewake cost and - * b) it works around a bug in Icelake which cannot cope with too rapid - * transitions. - */ - intel_uncore_forcewake_put_delayed(uncore, FORCEWAKE_ALL); -} - -static bool tlb_seqno_passed(const struct intel_gt *gt, u32 seqno) -{ - u32 cur =3D intel_gt_tlb_seqno(gt); - - /* Only skip if a *full* TLB invalidate barrier has passed */ - return (s32)(cur - ALIGN(seqno, 2)) > 0; -} - -void intel_gt_invalidate_tlb(struct intel_gt *gt, u32 seqno) -{ - intel_wakeref_t wakeref; - - if (I915_SELFTEST_ONLY(gt->awake =3D=3D -ENODEV)) - return; - - if (intel_gt_is_wedged(gt)) - return; - - if (tlb_seqno_passed(gt, seqno)) - return; - - with_intel_gt_pm_if_awake(gt, wakeref) { - mutex_lock(>->tlb.invalidate_lock); - if (tlb_seqno_passed(gt, seqno)) - goto unlock; - - mmio_invalidate_full(gt); - - write_seqcount_invalidate(>->tlb.seqno); -unlock: - mutex_unlock(>->tlb.invalidate_lock); - } -} diff --git a/drivers/gpu/drm/i915/gt/intel_gt.h b/drivers/gpu/drm/i915/gt/i= ntel_gt.h index 40b06adf509a..b4bba16cdb53 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt.h +++ b/drivers/gpu/drm/i915/gt/intel_gt.h @@ -101,16 +101,4 @@ void intel_gt_info_print(const struct intel_gt_info *i= nfo, =20 void intel_gt_watchdog_work(struct work_struct *work); =20 -static inline u32 intel_gt_tlb_seqno(const struct intel_gt *gt) -{ - return seqprop_sequence(>->tlb.seqno); -} - -static inline u32 intel_gt_next_invalidate_tlb_full(const struct intel_gt = *gt) -{ - return intel_gt_tlb_seqno(gt) | 1; -} - -void intel_gt_invalidate_tlb(struct intel_gt *gt, u32 seqno); - #endif /* __INTEL_GT_H__ */ diff --git a/drivers/gpu/drm/i915/gt/intel_tlb.c b/drivers/gpu/drm/i915/gt/= intel_tlb.c new file mode 100644 index 000000000000..af8cae979489 --- /dev/null +++ b/drivers/gpu/drm/i915/gt/intel_tlb.c @@ -0,0 +1,183 @@ +// SPDX-License-Identifier: MIT +/* + * Copyright =C2=A9 2022 Intel Corporation + */ + +#include "i915_drv.h" +#include "i915_perf_oa_regs.h" +#include "intel_engine_pm.h" +#include "intel_gt.h" +#include "intel_gt_pm.h" +#include "intel_gt_regs.h" +#include "intel_tlb.h" + +struct reg_and_bit { + i915_reg_t reg; + u32 bit; +}; + +static struct reg_and_bit +get_reg_and_bit(const struct intel_engine_cs *engine, const bool gen8, + const i915_reg_t *regs, const unsigned int num) +{ + const unsigned int class =3D engine->class; + struct reg_and_bit rb =3D { }; + + if (drm_WARN_ON_ONCE(&engine->i915->drm, + class >=3D num || !regs[class].reg)) + return rb; + + rb.reg =3D regs[class]; + if (gen8 && class =3D=3D VIDEO_DECODE_CLASS) + rb.reg.reg +=3D 4 * engine->instance; /* GEN8_M2TCR */ + else + rb.bit =3D engine->instance; + + rb.bit =3D BIT(rb.bit); + + return rb; +} + +static bool tlb_seqno_passed(const struct intel_gt *gt, u32 seqno) +{ + u32 cur =3D intel_gt_tlb_seqno(gt); + + /* Only skip if a *full* TLB invalidate barrier has passed */ + return (s32)(cur - ALIGN(seqno, 2)) > 0; +} + +static void mmio_invalidate_full(struct intel_gt *gt) +{ + static const i915_reg_t gen8_regs[] =3D { + [RENDER_CLASS] =3D GEN8_RTCR, + [VIDEO_DECODE_CLASS] =3D GEN8_M1TCR, /* , GEN8_M2TCR */ + [VIDEO_ENHANCEMENT_CLASS] =3D GEN8_VTCR, + [COPY_ENGINE_CLASS] =3D GEN8_BTCR, + }; + static const i915_reg_t gen12_regs[] =3D { + [RENDER_CLASS] =3D GEN12_GFX_TLB_INV_CR, + [VIDEO_DECODE_CLASS] =3D GEN12_VD_TLB_INV_CR, + [VIDEO_ENHANCEMENT_CLASS] =3D GEN12_VE_TLB_INV_CR, + [COPY_ENGINE_CLASS] =3D GEN12_BLT_TLB_INV_CR, + [COMPUTE_CLASS] =3D GEN12_COMPCTX_TLB_INV_CR, + }; + struct drm_i915_private *i915 =3D gt->i915; + struct intel_uncore *uncore =3D gt->uncore; + struct intel_engine_cs *engine; + intel_engine_mask_t awake, tmp; + enum intel_engine_id id; + const i915_reg_t *regs; + unsigned int num =3D 0; + + if (GRAPHICS_VER(i915) =3D=3D 12) { + regs =3D gen12_regs; + num =3D ARRAY_SIZE(gen12_regs); + } else if (GRAPHICS_VER(i915) >=3D 8 && GRAPHICS_VER(i915) <=3D 11) { + regs =3D gen8_regs; + num =3D ARRAY_SIZE(gen8_regs); + } else if (GRAPHICS_VER(i915) < 8) { + return; + } + + if (drm_WARN_ONCE(&i915->drm, !num, + "Platform does not implement TLB invalidation!")) + return; + + intel_uncore_forcewake_get(uncore, FORCEWAKE_ALL); + + spin_lock_irq(&uncore->lock); /* serialise invalidate with GT reset */ + + awake =3D 0; + for_each_engine(engine, gt, id) { + struct reg_and_bit rb; + + if (!intel_engine_pm_is_awake(engine)) + continue; + + rb =3D get_reg_and_bit(engine, regs =3D=3D gen8_regs, regs, num); + if (!i915_mmio_reg_offset(rb.reg)) + continue; + + intel_uncore_write_fw(uncore, rb.reg, rb.bit); + awake |=3D engine->mask; + } + + GT_TRACE(gt, "invalidated engines %08x\n", awake); + + /* Wa_2207587034:tgl,dg1,rkl,adl-s,adl-p */ + if (awake && + (IS_TIGERLAKE(i915) || + IS_DG1(i915) || + IS_ROCKETLAKE(i915) || + IS_ALDERLAKE_S(i915) || + IS_ALDERLAKE_P(i915))) + intel_uncore_write_fw(uncore, GEN12_OA_TLB_INV_CR, 1); + + spin_unlock_irq(&uncore->lock); + + for_each_engine_masked(engine, gt, awake, tmp) { + struct reg_and_bit rb; + + /* + * HW architecture suggest typical invalidation time at 40us, + * with pessimistic cases up to 100us and a recommendation to + * cap at 1ms. We go a bit higher just in case. + */ + const unsigned int timeout_us =3D 100; + const unsigned int timeout_ms =3D 4; + + rb =3D get_reg_and_bit(engine, regs =3D=3D gen8_regs, regs, num); + if (__intel_wait_for_register_fw(uncore, + rb.reg, rb.bit, 0, + timeout_us, timeout_ms, + NULL)) + drm_err_ratelimited(>->i915->drm, + "%s TLB invalidation did not complete in %ums!\n", + engine->name, timeout_ms); + } + + /* + * Use delayed put since a) we mostly expect a flurry of TLB + * invalidations so it is good to avoid paying the forcewake cost and + * b) it works around a bug in Icelake which cannot cope with too rapid + * transitions. + */ + intel_uncore_forcewake_put_delayed(uncore, FORCEWAKE_ALL); +} + +void intel_gt_invalidate_tlb_full(struct intel_gt *gt, u32 seqno) +{ + intel_wakeref_t wakeref; + + if (I915_SELFTEST_ONLY(gt->awake =3D=3D -ENODEV)) + return; + + if (intel_gt_is_wedged(gt)) + return; + + if (tlb_seqno_passed(gt, seqno)) + return; + + with_intel_gt_pm_if_awake(gt, wakeref) { + mutex_lock(>->tlb.invalidate_lock); + if (tlb_seqno_passed(gt, seqno)) + goto unlock; + + mmio_invalidate_full(gt); + + write_seqcount_invalidate(>->tlb.seqno); +unlock: + mutex_unlock(>->tlb.invalidate_lock); + } +} + +void intel_gt_init_tlb(struct intel_gt *gt) +{ + mutex_init(>->tlb.invalidate_lock); + seqcount_mutex_init(>->tlb.seqno, >->tlb.invalidate_lock); +} + +void intel_gt_fini_tlb(struct intel_gt *gt) +{ + mutex_destroy(>->tlb.invalidate_lock); +} diff --git a/drivers/gpu/drm/i915/gt/intel_tlb.h b/drivers/gpu/drm/i915/gt/= intel_tlb.h new file mode 100644 index 000000000000..46ce25bf5afe --- /dev/null +++ b/drivers/gpu/drm/i915/gt/intel_tlb.h @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: MIT */ +/* + * Copyright =C2=A9 2022 Intel Corporation + */ + +#ifndef INTEL_TLB_H +#define INTEL_TLB_H + +#include +#include + +#include "intel_gt_types.h" + +void intel_gt_invalidate_tlb_full(struct intel_gt *gt, u32 seqno); + +void intel_gt_init_tlb(struct intel_gt *gt); +void intel_gt_fini_tlb(struct intel_gt *gt); + +static inline u32 intel_gt_tlb_seqno(const struct intel_gt *gt) +{ + return seqprop_sequence(>->tlb.seqno); +} + +static inline u32 intel_gt_next_invalidate_tlb_full(const struct intel_gt = *gt) +{ + return intel_gt_tlb_seqno(gt) | 1; +} + +#endif /* INTEL_TLB_H */ diff --git a/drivers/gpu/drm/i915/i915_vma.c b/drivers/gpu/drm/i915/i915_vm= a.c index 84a9ccbc5fc5..fe947d1456d5 100644 --- a/drivers/gpu/drm/i915/i915_vma.c +++ b/drivers/gpu/drm/i915/i915_vma.c @@ -33,6 +33,7 @@ #include "gt/intel_engine_heartbeat.h" #include "gt/intel_gt.h" #include "gt/intel_gt_requests.h" +#include "gt/intel_tlb.h" =20 #include "i915_drv.h" #include "i915_gem_evict.h" --=20 2.36.1 From nobody Sat Apr 18 14:11:47 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 27AD8C433EF for ; Wed, 13 Jul 2022 09:31:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236165AbiGMJb2 (ORCPT ); Wed, 13 Jul 2022 05:31:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44010 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235577AbiGMJaa (ORCPT ); Wed, 13 Jul 2022 05:30:30 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 85A17F32E4 for ; Wed, 13 Jul 2022 02:30:25 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 1D77061CCF for ; Wed, 13 Jul 2022 09:30:24 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 07134C385A2; Wed, 13 Jul 2022 09:30:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1657704622; bh=+GgweU1Qvz3U0myu7Eg5zYjBW9WmB7ZwmFBD8Cp4cUM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Pp6vjho4VjWsQPlyPa2b6aseR9AtRNLEmExELYnroH25HjSoYFv7MB6Pq6ICIUoqR fMShmj+jlj1Cmm5Hje1dZFiZAZF+SoPkI3NwD3EQxO35uQtn6/OPvkobnr+vbk1hN/ xpEnq9TlvUA+Tu98JyRO0CF39mX7fwP6Oh6BZl7FQGr6lj4fWgW17BDcaVV0xwBWaG yGuLX8qDZ4rJmXyHNheE04KpBGxqVyW9R7R3lJlDYSYg+AhBYTqwijPbsoMsN6gsCL eoMuH1odwnwp+WuzwQqQTHYPbEtOsILEyD1XVD2LxNcJWC87dQO7HhsLOB8lpY7M86 QDPVB5eFYGouw== Received: from mchehab by mail.kernel.org with local (Exim 4.95) (envelope-from ) id 1oBYhH-0050La-Nj; Wed, 13 Jul 2022 10:30:19 +0100 From: Mauro Carvalho Chehab Cc: Prathap Kumar Valsan , Alan Previn , Borislav Petkov , Daniel Vetter , Daniele Ceraolo Spurio , David Airlie , Jani Nikula , John Harrison , Joonas Lahtinen , Lucas De Marchi , Matt Roper , Matthew Brost , Mauro Carvalho Chehab , Michal Wajdeczko , Rodrigo Vivi , Tvrtko Ursulin , Umesh Nerlige Ramappa , Vinay Belgaumkar , dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, linux-kernel@vger.kernel.org, Bruce Chang , Chris Wilson Subject: [PATCH 09/21] drm/i915/guc: Define CTB based TLB invalidation routines Date: Wed, 13 Jul 2022 10:30:06 +0100 Message-Id: <4855438b4aea605421211d00bd281489a4a45503.1657703926.git.mchehab@kernel.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Prathap Kumar Valsan Add routines to interface with GuC firmware for TLB invalidation. Signed-off-by: Prathap Kumar Valsan Cc: Bruce Chang Cc: Michal Wajdeczko Cc: Matthew Brost Cc: Chris Wilson Signed-off-by: Mauro Carvalho Chehab --- See [PATCH 00/21] at: https://lore.kernel.org/all/cover.1657703926.git.mche= hab@kernel.org/ .../gpu/drm/i915/gt/uc/abi/guc_actions_abi.h | 35 +++++++ drivers/gpu/drm/i915/gt/uc/intel_guc.c | 90 ++++++++++++++++++ drivers/gpu/drm/i915/gt/uc/intel_guc.h | 13 +++ drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c | 24 ++++- drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h | 6 ++ .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 91 ++++++++++++++++++- 6 files changed, 253 insertions(+), 6 deletions(-) diff --git a/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h b/drivers/gpu= /drm/i915/gt/uc/abi/guc_actions_abi.h index 4ef9990ed7f8..2e39d8df4c82 100644 --- a/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h +++ b/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h @@ -134,6 +134,10 @@ enum intel_guc_action { INTEL_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC =3D 0x4601, INTEL_GUC_ACTION_CLIENT_SOFT_RESET =3D 0x5507, INTEL_GUC_ACTION_SET_ENG_UTIL_BUFF =3D 0x550A, + INTEL_GUC_ACTION_NOTIFY_MEMORY_CAT_ERROR =3D 0x6000, + INTEL_GUC_ACTION_PAGE_FAULT_NOTIFICATION =3D 0x6001, + INTEL_GUC_ACTION_TLB_INVALIDATION =3D 0x7000, + INTEL_GUC_ACTION_TLB_INVALIDATION_DONE =3D 0x7001, INTEL_GUC_ACTION_STATE_CAPTURE_NOTIFICATION =3D 0x8002, INTEL_GUC_ACTION_NOTIFY_FLUSH_LOG_BUFFER_TO_FILE =3D 0x8003, INTEL_GUC_ACTION_NOTIFY_CRASH_DUMP_POSTED =3D 0x8004, @@ -177,4 +181,35 @@ enum intel_guc_state_capture_event_status { =20 #define INTEL_GUC_STATE_CAPTURE_EVENT_STATUS_MASK 0x000000FF =20 +#define INTEL_GUC_TLB_INVAL_TYPE_SHIFT 0 +#define INTEL_GUC_TLB_INVAL_MODE_SHIFT 8 +/* Flush PPC or SMRO caches along with TLB invalidation request */ +#define INTEL_GUC_TLB_INVAL_FLUSH_CACHE (1 << 31) + +enum intel_guc_tlb_invalidation_type { + INTEL_GUC_TLB_INVAL_GUC =3D 0x3, +}; + +/* + * 0: Heavy mode of Invalidation: + * The pipeline of the engine(s) for which the invalidation is targeted to= is + * blocked, and all the in-flight transactions are guaranteed to be Global= ly + * Observed before completing the TLB invalidation + * 1: Lite mode of Invalidation: + * TLBs of the targeted engine(s) are immediately invalidated. + * In-flight transactions are NOT guaranteed to be Globally Observed before + * completing TLB invalidation. + * Light Invalidation Mode is to be used only when + * it can be guaranteed (by SW) that the address translations remain invar= iant + * for the in-flight transactions across the TLB invalidation. In other wo= rds, + * this mode can be used when the TLB invalidation is intended to clear ou= t the + * stale cached translations that are no longer in use. Light Invalidation= Mode + * is much faster than the Heavy Invalidation Mode, as it does not wait fo= r the + * in-flight transactions to be GOd. + */ +enum intel_guc_tlb_inval_mode { + INTEL_GUC_TLB_INVAL_MODE_HEAVY =3D 0x0, + INTEL_GUC_TLB_INVAL_MODE_LITE =3D 0x1, +}; + #endif /* _ABI_GUC_ACTIONS_ABI_H */ diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc.c b/drivers/gpu/drm/i915/= gt/uc/intel_guc.c index 2706a8c65090..5c59f9b144a3 100644 --- a/drivers/gpu/drm/i915/gt/uc/intel_guc.c +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc.c @@ -855,6 +855,96 @@ int intel_guc_self_cfg64(struct intel_guc *guc, u16 ke= y, u64 value) return __guc_self_cfg(guc, key, 2, value); } =20 +static int guc_send_invalidate_tlb(struct intel_guc *guc, u32 *action, u32= size) +{ + struct intel_guc_tlb_wait _wq, *wq =3D &_wq; + DEFINE_WAIT_FUNC(wait, woken_wake_function); + int err =3D 0; + u32 seqno; + + init_waitqueue_head(&_wq.wq); + + if (xa_alloc_cyclic_irq(&guc->tlb_lookup, &seqno, wq, + xa_limit_32b, &guc->next_seqno, + GFP_ATOMIC | __GFP_NOWARN) < 0) { + /* Under severe memory pressure? Serialise TLB allocations */ + xa_lock_irq(&guc->tlb_lookup); + wq =3D xa_load(&guc->tlb_lookup, guc->serial_slot); + wait_event_lock_irq(wq->wq, + !READ_ONCE(wq->status), + guc->tlb_lookup.xa_lock); + /* + * Update wq->status under lock to ensure only one waiter can + * issue the tlb invalidation command using the serial slot at a + * time. The condition is set to false before releasing the lock + * so that other caller continue to wait until woken up again. + */ + wq->status =3D 1; + xa_unlock_irq(&guc->tlb_lookup); + + seqno =3D guc->serial_slot; + } + + action[1] =3D seqno; + + add_wait_queue(&wq->wq, &wait); + + err =3D intel_guc_send_busy_loop(guc, action, size, G2H_LEN_DW_INVALIDATE= _TLB, true); + if (err) { + /* + * XXX: Failure of tlb invalidation is critical and would + * warrant a gt reset. + */ + goto out; + } +/* + * GuC has a timeout of 1ms for a tlb invalidation response from GAM. On a + * timeout GuC drops the request and has no mechanism to notify the host a= bout + * the timeout. So keep a larger timeout that accounts for this individual + * timeout and max number of outstanding invalidation requests that can be + * queued in CT buffer. + */ +#define OUTSTANDING_GUC_TIMEOUT_PERIOD (HZ) + if (!wait_woken(&wait, TASK_UNINTERRUPTIBLE, + OUTSTANDING_GUC_TIMEOUT_PERIOD)) { + /* + * XXX: Failure of tlb invalidation is critical and would + * warrant a gt reset. + */ + drm_err(&guc_to_gt(guc)->i915->drm, + "tlb invalidation response timed out for seqno %u\n", seqno); + err =3D -ETIME; + } +out: + remove_wait_queue(&wq->wq, &wait); + if (seqno !=3D guc->serial_slot) + xa_erase_irq(&guc->tlb_lookup, seqno); + + return err; +} + +/* + * Guc TLB Invalidation: Invalidate the TLB's of GuC itself. + */ +int intel_guc_invalidate_tlb_guc(struct intel_guc *guc, + enum intel_guc_tlb_inval_mode mode) +{ + u32 action[] =3D { + INTEL_GUC_ACTION_TLB_INVALIDATION, + 0, + INTEL_GUC_TLB_INVAL_GUC << INTEL_GUC_TLB_INVAL_TYPE_SHIFT | + mode << INTEL_GUC_TLB_INVAL_MODE_SHIFT | + INTEL_GUC_TLB_INVAL_FLUSH_CACHE, + }; + + if (!INTEL_GUC_SUPPORTS_TLB_INVALIDATION(guc)) { + DRM_ERROR("Tlb invalidation: Operation not supported in this platform!\n= "); + return 0; + } + + return guc_send_invalidate_tlb(guc, action, ARRAY_SIZE(action)); +} + /** * intel_guc_load_status - dump information about GuC load status * @guc: the GuC diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc.h b/drivers/gpu/drm/i915/= gt/uc/intel_guc.h index d0d99f178f2d..f82a121b0838 100644 --- a/drivers/gpu/drm/i915/gt/uc/intel_guc.h +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc.h @@ -77,6 +77,10 @@ struct intel_guc { atomic_t outstanding_submission_g2h; =20 /** @interrupts: pointers to GuC interrupt-managing functions. */ + struct xarray tlb_lookup; + u32 serial_slot; + u32 next_seqno; + struct { void (*reset)(struct intel_guc *guc); void (*enable)(struct intel_guc *guc); @@ -248,6 +252,11 @@ struct intel_guc { #endif }; =20 +struct intel_guc_tlb_wait { + struct wait_queue_head wq; + u8 status; +} __aligned(4); + static inline struct intel_guc *log_to_guc(struct intel_guc_log *log) { return container_of(log, struct intel_guc, log); @@ -363,6 +372,9 @@ int intel_guc_allocate_and_map_vma(struct intel_guc *gu= c, u32 size, int intel_guc_self_cfg32(struct intel_guc *guc, u16 key, u32 value); int intel_guc_self_cfg64(struct intel_guc *guc, u16 key, u64 value); =20 +int intel_guc_invalidate_tlb_guc(struct intel_guc *guc, + enum intel_guc_tlb_inval_mode mode); + static inline bool intel_guc_is_supported(struct intel_guc *guc) { return intel_uc_fw_is_supported(&guc->fw); @@ -440,6 +452,7 @@ int intel_guc_engine_failure_process_msg(struct intel_g= uc *guc, const u32 *msg, u32 len); int intel_guc_error_capture_process_msg(struct intel_guc *guc, const u32 *msg, u32 len); +void intel_guc_tlb_invalidation_done(struct intel_guc *guc, u32 seqno); =20 struct intel_engine_cs * intel_guc_lookup_engine(struct intel_guc *guc, u8 guc_class, u8 instance); diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c b/drivers/gpu/drm/i9= 15/gt/uc/intel_guc_ct.c index f01325cd1b62..c1ce542b7855 100644 --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c @@ -1023,7 +1023,7 @@ static int ct_process_request(struct intel_guc_ct *ct= , struct ct_incoming_msg *r return 0; } =20 -static bool ct_process_incoming_requests(struct intel_guc_ct *ct) +static bool ct_process_incoming_requests(struct intel_guc_ct *ct, struct l= ist_head *incoming) { unsigned long flags; struct ct_incoming_msg *request; @@ -1031,11 +1031,11 @@ static bool ct_process_incoming_requests(struct int= el_guc_ct *ct) int err; =20 spin_lock_irqsave(&ct->requests.lock, flags); - request =3D list_first_entry_or_null(&ct->requests.incoming, + request =3D list_first_entry_or_null(incoming, struct ct_incoming_msg, link); if (request) list_del(&request->link); - done =3D !!list_empty(&ct->requests.incoming); + done =3D !!list_empty(incoming); spin_unlock_irqrestore(&ct->requests.lock, flags); =20 if (!request) @@ -1058,7 +1058,7 @@ static void ct_incoming_request_worker_func(struct wo= rk_struct *w) bool done; =20 do { - done =3D ct_process_incoming_requests(ct); + done =3D ct_process_incoming_requests(ct, &ct->requests.incoming); } while (!done); } =20 @@ -1078,14 +1078,30 @@ static int ct_handle_event(struct intel_guc_ct *ct,= struct ct_incoming_msg *requ switch (action) { case INTEL_GUC_ACTION_SCHED_CONTEXT_MODE_DONE: case INTEL_GUC_ACTION_DEREGISTER_CONTEXT_DONE: + case INTEL_GUC_ACTION_TLB_INVALIDATION_DONE: g2h_release_space(ct, request->size); } + /* Handle tlb invalidation response in interrupt context */ + if (action =3D=3D INTEL_GUC_ACTION_TLB_INVALIDATION_DONE) { + const u32 *payload; + u32 hxg_len, len; + + hxg_len =3D request->size - GUC_CTB_MSG_MIN_LEN; + len =3D hxg_len - GUC_HXG_MSG_MIN_LEN; + if (unlikely(len < 1)) + return -EPROTO; + payload =3D &hxg[GUC_HXG_MSG_MIN_LEN]; + intel_guc_tlb_invalidation_done(ct_to_guc(ct), payload[0]); + ct_free_msg(request); + return 0; + } =20 spin_lock_irqsave(&ct->requests.lock, flags); list_add_tail(&request->link, &ct->requests.incoming); spin_unlock_irqrestore(&ct->requests.lock, flags); =20 queue_work(system_unbound_wq, &ct->requests.worker); + return 0; } =20 diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h b/drivers/gpu/drm/= i915/gt/uc/intel_guc_fwif.h index b3c9a9327f76..3edf567b3f65 100644 --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h @@ -22,6 +22,7 @@ /* Payload length only i.e. don't include G2H header length */ #define G2H_LEN_DW_SCHED_CONTEXT_MODE_SET 2 #define G2H_LEN_DW_DEREGISTER_CONTEXT 1 +#define G2H_LEN_DW_INVALIDATE_TLB 1 =20 #define GUC_CONTEXT_DISABLE 0 #define GUC_CONTEXT_ENABLE 1 @@ -431,4 +432,9 @@ enum intel_guc_recv_message { INTEL_GUC_RECV_MSG_EXCEPTION =3D BIT(30), }; =20 +#define INTEL_GUC_SUPPORTS_TLB_INVALIDATION(guc) \ + ((intel_guc_ct_enabled(&(guc)->ct)) && \ + (intel_guc_submission_is_used(guc)) && \ + (GRAPHICS_VER(guc_to_gt((guc))->i915) >=3D 12)) + #endif diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gp= u/drm/i915/gt/uc/intel_guc_submission.c index 40f726c61e95..6888ea1bc7c1 100644 --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c @@ -1653,11 +1653,20 @@ static void __guc_reset_context(struct intel_contex= t *ce, intel_engine_mask_t st intel_context_put(parent); } =20 +static void wake_up_tlb_invalidate(struct intel_guc_tlb_wait *wait) +{ + /* Barrier to ensure the store is observed by the woken thread */ + smp_store_mb(wait->status, 0); + wake_up(&wait->wq); +} + void intel_guc_submission_reset(struct intel_guc *guc, intel_engine_mask_t= stalled) { + struct intel_guc_tlb_wait *wait; struct intel_context *ce; unsigned long index; unsigned long flags; + unsigned long i; =20 if (unlikely(!guc_submission_initialized(guc))) { /* Reset called during driver load? GuC not yet initialised! */ @@ -1683,6 +1692,13 @@ void intel_guc_submission_reset(struct intel_guc *gu= c, intel_engine_mask_t stall =20 /* GuC is blown away, drop all references to contexts */ xa_destroy(&guc->context_lookup); + + /* + * The full GT reset will have cleared the TLB caches and flushed the + * G2H message queue; we can release all the blocked waiters. + */ + xa_for_each(&guc->tlb_lookup, i, wait) + wake_up_tlb_invalidate(wait); } =20 static void guc_cancel_context_requests(struct intel_context *ce) @@ -1805,6 +1821,41 @@ void intel_guc_submission_reset_finish(struct intel_= guc *guc) static void destroyed_worker_func(struct work_struct *w); static void reset_fail_worker_func(struct work_struct *w); =20 +static int init_tlb_lookup(struct intel_guc *guc) +{ + struct intel_guc_tlb_wait *wait; + int err; + + xa_init_flags(&guc->tlb_lookup, XA_FLAGS_ALLOC); + + wait =3D kzalloc(sizeof(*wait), GFP_KERNEL); + if (!wait) + return -ENOMEM; + + init_waitqueue_head(&wait->wq); + err =3D xa_alloc_cyclic_irq(&guc->tlb_lookup, &guc->serial_slot, wait, + xa_limit_32b, &guc->next_seqno, GFP_KERNEL); + if (err =3D=3D -ENOMEM) { + kfree(wait); + return err; + } + + return 0; +} + +static void fini_tlb_lookup(struct intel_guc *guc) +{ + struct intel_guc_tlb_wait *wait; + + wait =3D xa_load(&guc->tlb_lookup, guc->serial_slot); + if (wait) { + GEM_BUG_ON(wait->status); + kfree(wait); + } + + xa_destroy(&guc->tlb_lookup); +} + /* * Set up the memory resources to be shared with the GuC (via the GGTT) * at firmware loading time. @@ -1812,20 +1863,31 @@ static void reset_fail_worker_func(struct work_stru= ct *w); int intel_guc_submission_init(struct intel_guc *guc) { struct intel_gt *gt =3D guc_to_gt(guc); + int ret; =20 if (guc->submission_initialized) return 0; =20 + ret =3D init_tlb_lookup(guc); + if (ret) + return ret; + guc->submission_state.guc_ids_bitmap =3D bitmap_zalloc(NUMBER_MULTI_LRC_GUC_ID(guc), GFP_KERNEL); - if (!guc->submission_state.guc_ids_bitmap) - return -ENOMEM; + if (!guc->submission_state.guc_ids_bitmap) { + ret =3D -ENOMEM; + goto err; + } =20 guc->timestamp.ping_delay =3D (POLL_TIME_CLKS / gt->clock_frequency + 1) = * HZ; guc->timestamp.shift =3D gpm_timestamp_shift(gt); guc->submission_initialized =3D true; =20 return 0; + +err: + fini_tlb_lookup(guc); + return ret; } =20 void intel_guc_submission_fini(struct intel_guc *guc) @@ -1836,6 +1898,7 @@ void intel_guc_submission_fini(struct intel_guc *guc) guc_flush_destroyed_contexts(guc); i915_sched_engine_put(guc->sched_engine); bitmap_free(guc->submission_state.guc_ids_bitmap); + fini_tlb_lookup(guc); guc->submission_initialized =3D false; } =20 @@ -4027,6 +4090,30 @@ g2h_context_lookup(struct intel_guc *guc, u32 ctx_id) return ce; } =20 +static void wait_wake_outstanding_tlb_g2h(struct intel_guc *guc, u32 seqno) +{ + struct intel_guc_tlb_wait *wait; + unsigned long flags; + + xa_lock_irqsave(&guc->tlb_lookup, flags); + wait =3D xa_load(&guc->tlb_lookup, seqno); + + /* We received a response after the waiting task did exit with a timeout = */ + if (unlikely(!wait)) + drm_dbg(&guc_to_gt(guc)->i915->drm, + "Stale tlb invalidation response with seqno %d\n", seqno); + + if (wait) + wake_up_tlb_invalidate(wait); + + xa_unlock_irqrestore(&guc->tlb_lookup, flags); +} + +void intel_guc_tlb_invalidation_done(struct intel_guc *guc, u32 seqno) +{ + wait_wake_outstanding_tlb_g2h(guc, seqno); +} + int intel_guc_deregister_done_process_msg(struct intel_guc *guc, const u32 *msg, u32 len) --=20 2.36.1 From nobody Sat Apr 18 14:11:47 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 83743C43334 for ; Wed, 13 Jul 2022 09:31:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236142AbiGMJbQ (ORCPT ); Wed, 13 Jul 2022 05:31:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43982 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234961AbiGMJa3 (ORCPT ); Wed, 13 Jul 2022 05:30:29 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6F911F2E0F for ; Wed, 13 Jul 2022 02:30:25 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id E73B8B81D6A for ; Wed, 13 Jul 2022 09:30:23 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0E4F2C341D1; Wed, 13 Jul 2022 09:30:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1657704622; bh=wMNkL9BlFfZJJ7e12qDIh/CAGTODMlA3hMOonBBSBIA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kKsBel7mj3ls/9+FJ+M6yCLp7kQw/5gZ8Oo5Lx6pRk+sPzn9/OMRvmuqYB2JC3/RP 1Eeff3/MjVgHS4owXL61rqiH6ltPLCn59woKSEy2LqneGPjJ7IgwDdfhchFoPm/7Xi lMaJquH1VYmkll9JXP3L/hDMAF2ef2zqQZsdIj6HXRo/jZX8Y4zSYIlQKpkFrXoEbA Njxr+TLK4PWhC+2O4rIJSnzrQYflxHAC3oHTw2ALP9xQ+z2migkW+KrqFpShl94vJr 4XT0uidmQk0DHk2q9jrnluZlXlZk4fq0OQTeuRIy/COc2vWXalcxGrqCY1lzizPVfy loL0/rQO+F48w== Received: from mchehab by mail.kernel.org with local (Exim 4.95) (envelope-from ) id 1oBYhH-0050Ld-Od; Wed, 13 Jul 2022 10:30:19 +0100 From: Mauro Carvalho Chehab Cc: Mauro Carvalho Chehab , Alan Previn , Borislav Petkov , Daniel Vetter , David Airlie , Jani Nikula , John Harrison , Joonas Lahtinen , Matthew Brost , Michal Wajdeczko , Prathap Kumar Valsan , Rodrigo Vivi , Tvrtko Ursulin , dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, linux-kernel@vger.kernel.org Subject: [PATCH 10/21] drm/i915/guc: use kernel-doc for enum intel_guc_tlb_inval_mode Date: Wed, 13 Jul 2022 10:30:07 +0100 Message-Id: <39ff9de11bef9a30205fa340f89fa36de783be55.1657703926.git.mchehab@kernel.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Transform the comments for intel_guc_tlb_inval_mode into a kernel-doc markup. Signed-off-by: Mauro Carvalho Chehab --- See [PATCH 00/21] at: https://lore.kernel.org/all/cover.1657703926.git.mche= hab@kernel.org/ drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h b/drivers/gpu= /drm/i915/gt/uc/abi/guc_actions_abi.h index 2e39d8df4c82..14e35a2f8306 100644 --- a/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h +++ b/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h @@ -190,15 +190,18 @@ enum intel_guc_tlb_invalidation_type { INTEL_GUC_TLB_INVAL_GUC =3D 0x3, }; =20 -/* - * 0: Heavy mode of Invalidation: +/** + * enum intel_guc_tlb_inval_mode - define the mode for TLB cache invlidati= on + * + * @INTEL_GUC_TLB_INVAL_MODE_HEAVY: Heavy Invalidation Mode. * The pipeline of the engine(s) for which the invalidation is targeted to= is * blocked, and all the in-flight transactions are guaranteed to be Global= ly - * Observed before completing the TLB invalidation - * 1: Lite mode of Invalidation: + * Observed before completing the TLB invalidation. + * @INTEL_GUC_TLB_INVAL_MODE_LITE: Light Invalidation Mode. * TLBs of the targeted engine(s) are immediately invalidated. * In-flight transactions are NOT guaranteed to be Globally Observed before * completing TLB invalidation. + * * Light Invalidation Mode is to be used only when * it can be guaranteed (by SW) that the address translations remain invar= iant * for the in-flight transactions across the TLB invalidation. In other wo= rds, --=20 2.36.1 From nobody Sat Apr 18 14:11:47 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71B97C433EF for ; Wed, 13 Jul 2022 09:30:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235615AbiGMJab (ORCPT ); Wed, 13 Jul 2022 05:30:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43742 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235271AbiGMJaY (ORCPT ); Wed, 13 Jul 2022 05:30:24 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D546BEE1F8 for ; Wed, 13 Jul 2022 02:30:23 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 7229861CAE for ; Wed, 13 Jul 2022 09:30:23 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 26A02C341D9; Wed, 13 Jul 2022 09:30:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1657704622; bh=numqK226xRmWDfGwHz4d5UBb1H2QKPgM+SkFWYMj/wI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=SAbvoxeX0Ggt9W/GCv9SVYOeovlGpOZD7jsBImb7PhuMpPL2NV8AhwQ++WOQu4lrB SUFyClvGEs4Jh5lWKYfxQh10iKBOLOeePFezzCpaFRTjbNYFrfs1Hk1hfPScgpx01h U8CkhvAz/xu6hzblOMsgFz0qlGbul9WvQZjHYE3/1RpfcMB1Qlr+P02h3BFpKqcjHS sS20FeC369U3qu6cIZ6Dgj4d1dEaOEH60H2bj5eHrKshkRYqww6UBRRXQOn1GXeKyr Eyf4Z9cki4QvvUYKZtXqo8fxAGZXuhkmnP5p4jUvpuEuQAuv9m7PFCVY4bfaWGQw+k FPVMJi2zh3RpQ== Received: from mchehab by mail.kernel.org with local (Exim 4.95) (envelope-from ) id 1oBYhH-0050Lg-PO; Wed, 13 Jul 2022 10:30:19 +0100 From: Mauro Carvalho Chehab Cc: Mauro Carvalho Chehab , Alan Previn , Daniel Vetter , Daniele Ceraolo Spurio , David Airlie , Jani Nikula , John Harrison , Joonas Lahtinen , Lucas De Marchi , Matthew Brost , Prathap Kumar Valsan , Rodrigo Vivi , Tvrtko Ursulin , Umesh Nerlige Ramappa , dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, linux-kernel@vger.kernel.org Subject: [PATCH 11/21] drm/i915/guc: document the TLB invalidation struct members Date: Wed, 13 Jul 2022 10:30:08 +0100 Message-Id: X-Mailer: git-send-email 2.36.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add documentation for the 3 new members of struct intel_guc that are used to handle TLB cache invalidation logic. Signed-off-by: Mauro Carvalho Chehab --- See [PATCH 00/21] at: https://lore.kernel.org/all/cover.1657703926.git.mche= hab@kernel.org/ drivers/gpu/drm/i915/gt/uc/intel_guc.h | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc.h b/drivers/gpu/drm/i915/= gt/uc/intel_guc.h index f82a121b0838..73c46d405dc4 100644 --- a/drivers/gpu/drm/i915/gt/uc/intel_guc.h +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc.h @@ -76,11 +76,23 @@ struct intel_guc { */ atomic_t outstanding_submission_g2h; =20 - /** @interrupts: pointers to GuC interrupt-managing functions. */ + /** + * @tlb_lookup: TLB cache invalidation lookup table. + */ struct xarray tlb_lookup; + + /** + * @serial_slot: index to the latest allocated element at the + * @tlb_lookup xarray. + */ u32 serial_slot; + + /** + * @next_seqno: next index to be allocated at the @tlb_lookup xarray. + */ u32 next_seqno; =20 + /** @interrupts: pointers to GuC interrupt-managing functions. */ struct { void (*reset)(struct intel_guc *guc); void (*enable)(struct intel_guc *guc); --=20 2.36.1 From nobody Sat Apr 18 14:11:47 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07DAAC43334 for ; Wed, 13 Jul 2022 09:30:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235922AbiGMJar (ORCPT ); Wed, 13 Jul 2022 05:30:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43798 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231936AbiGMJaZ (ORCPT ); Wed, 13 Jul 2022 05:30:25 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5487CF2E20 for ; Wed, 13 Jul 2022 02:30:24 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id E162A61CC0 for ; Wed, 13 Jul 2022 09:30:23 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5700FC341E1; Wed, 13 Jul 2022 09:30:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1657704622; bh=Fkt5pO3rEYdJ3e4kKvqhVhroFKgVT0cB8fjVsWq7iJ8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RtajU/xM0JkvFZqJ7Nf4uF2iZiE0nK5yPJQ8l7fIR1i/sE7K9j+uHglsYT5rjO/4G LUq9n3d3msgBx+dsGR28QhSXXaQtpSA05/M1Os/bRWtjpITdX33ms66GPMx4jjHfSp N/gyvlDR0Z0Sa9HGNzsrW7PFhaEJ6bXOnDiaqcc2bBz4qxnF5mDYFo4vIjLSdUIlYS hLuSWjCx7wYdmbuOVDuo4knHlVBqZ0X43uXNf6uBQwciTyK4rua1zJpO9VCAloIxjy CJB5s45etREBQlrAIYhFvuYZS2hrBrm2Uq66BCtgYkUmttfX0yqF2uyTksbIkjEW4G R7dBFJyiC/J7Q== Received: from mchehab by mail.kernel.org with local (Exim 4.95) (envelope-from ) id 1oBYhH-0050Ll-Q7; Wed, 13 Jul 2022 10:30:19 +0100 From: Mauro Carvalho Chehab Cc: =?UTF-8?q?Piotr=20Pi=C3=B3rkowski?= , Alan Previn , Daniel Vetter , Daniele Ceraolo Spurio , David Airlie , Jani Nikula , John Harrison , Joonas Lahtinen , Lucas De Marchi , Matt Roper , Matthew Brost , Mauro Carvalho Chehab , Michal Wajdeczko , Prathap Kumar Valsan , Rodrigo Vivi , Tvrtko Ursulin , Umesh Nerlige Ramappa , Vinay Belgaumkar , dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, linux-kernel@vger.kernel.org Subject: [PATCH 12/21] drm/i915/guc: Introduce TLB_INVALIDATION_ALL action Date: Wed, 13 Jul 2022 10:30:09 +0100 Message-Id: X-Mailer: git-send-email 2.36.1 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Piotr Pi=C3=B3rkowski Add a new way to invalidate TLB via GuC using actions 0x7002 (TLB_INVALIDATION_ALL). Those actions will be used on upcoming patches. Signed-off-by: Piotr Pi=C3=B3rkowski Cc: Michal Wajdeczko Signed-off-by: Mauro Carvalho Chehab --- See [PATCH 00/21] at: https://lore.kernel.org/all/cover.1657703926.git.mche= hab@kernel.org/ drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h | 1 + drivers/gpu/drm/i915/gt/uc/intel_guc.c | 14 ++++++++++++++ drivers/gpu/drm/i915/gt/uc/intel_guc.h | 1 + 3 files changed, 16 insertions(+) diff --git a/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h b/drivers/gpu= /drm/i915/gt/uc/abi/guc_actions_abi.h index 14e35a2f8306..fb0af33e43cc 100644 --- a/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h +++ b/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h @@ -138,6 +138,7 @@ enum intel_guc_action { INTEL_GUC_ACTION_PAGE_FAULT_NOTIFICATION =3D 0x6001, INTEL_GUC_ACTION_TLB_INVALIDATION =3D 0x7000, INTEL_GUC_ACTION_TLB_INVALIDATION_DONE =3D 0x7001, + INTEL_GUC_ACTION_TLB_INVALIDATION_ALL =3D 0x7002, INTEL_GUC_ACTION_STATE_CAPTURE_NOTIFICATION =3D 0x8002, INTEL_GUC_ACTION_NOTIFY_FLUSH_LOG_BUFFER_TO_FILE =3D 0x8003, INTEL_GUC_ACTION_NOTIFY_CRASH_DUMP_POSTED =3D 0x8004, diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc.c b/drivers/gpu/drm/i915/= gt/uc/intel_guc.c index 5c59f9b144a3..8a104a292598 100644 --- a/drivers/gpu/drm/i915/gt/uc/intel_guc.c +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc.c @@ -945,6 +945,20 @@ int intel_guc_invalidate_tlb_guc(struct intel_guc *guc, return guc_send_invalidate_tlb(guc, action, ARRAY_SIZE(action)); } =20 +int intel_guc_invalidate_tlb_all(struct intel_guc *guc) +{ + u32 action[] =3D { + INTEL_GUC_ACTION_TLB_INVALIDATION_ALL, + 0, + INTEL_GUC_TLB_INVAL_MODE_HEAVY << INTEL_GUC_TLB_INVAL_MODE_SHIFT | + INTEL_GUC_TLB_INVAL_FLUSH_CACHE, + }; + + GEM_BUG_ON(!INTEL_GUC_SUPPORTS_TLB_INVALIDATION(guc)); + + return guc_send_invalidate_tlb(guc, action, ARRAY_SIZE(action)); +} + /** * intel_guc_load_status - dump information about GuC load status * @guc: the GuC diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc.h b/drivers/gpu/drm/i915/= gt/uc/intel_guc.h index 73c46d405dc4..01c6478451cc 100644 --- a/drivers/gpu/drm/i915/gt/uc/intel_guc.h +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc.h @@ -386,6 +386,7 @@ int intel_guc_self_cfg64(struct intel_guc *guc, u16 key= , u64 value); =20 int intel_guc_invalidate_tlb_guc(struct intel_guc *guc, enum intel_guc_tlb_inval_mode mode); +int intel_guc_invalidate_tlb_all(struct intel_guc *guc); =20 static inline bool intel_guc_is_supported(struct intel_guc *guc) { --=20 2.36.1 From nobody Sat Apr 18 14:11:47 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6A4AACCA479 for ; Wed, 13 Jul 2022 09:31:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236124AbiGMJbH (ORCPT ); Wed, 13 Jul 2022 05:31:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43860 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235450AbiGMJa0 (ORCPT ); Wed, 13 Jul 2022 05:30:26 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EBC72F32D5 for ; Wed, 13 Jul 2022 02:30:24 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 068C661CC9 for ; Wed, 13 Jul 2022 09:30:24 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 54376C341DE; Wed, 13 Jul 2022 09:30:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1657704622; bh=13GXq2zQb7Bxagr/w8vw9akDrkJzNGqiR7Sg4+9Z8Ds=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uDR+oj1Uo0dd07KKa07FD91ksV5rjZafhgb75JpGn7nLZpTmaVt2d4I73XcSC1T90 38YlhonG421daCMfaODah47ilSlJC8NMlpB6YOUCCSlxTXYN3TOEZ5jU/JhP3RE1wI t4y21oq+GLPNWcWE4NQjFz4q22qEG3ljPjs6uxe8fXHKCsOCAxUtgp44dN0X9IAIPp ix1g/qrc5z7hD+TzOjSQo/2M3AaoTS0J7JafwyPRBarpeiTQSgfXThijjwbKc1hEe1 A6ZSsy2lcaOeRPJ6QljaCNgGCfBiVvKl1qwYybSLk2IXx483dACp37lVE/WqzbbFQY uVr+zS4SQCXnQ== Received: from mchehab by mail.kernel.org with local (Exim 4.95) (envelope-from ) id 1oBYhH-0050Lo-R1; Wed, 13 Jul 2022 10:30:19 +0100 From: Mauro Carvalho Chehab Cc: Chris Wilson , =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= , Andi Shyti , Ashutosh Dixit , Ayaz A Siddiqui , Daniel Vetter , Daniele Ceraolo Spurio , David Airlie , Jani Nikula , Jason Ekstrand , John Harrison , Joonas Lahtinen , Lucas De Marchi , Maarten Lankhorst , Matt Roper , Matthew Auld , Matthew Brost , Mauro Carvalho Chehab , Michael Cheng , Prathap Kumar Valsan , Rodrigo Vivi , Tvrtko Ursulin , dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, linux-kernel@vger.kernel.org, Fei Yang Subject: [PATCH 13/21] drm/i915: Invalidate the TLBs on each GT Date: Wed, 13 Jul 2022 10:30:10 +0100 Message-Id: <5b44379c3ec1960000a11b30215d22ff1e286df6.1657703926.git.mchehab@kernel.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Chris Wilson With multi-GT devices, the object may have been bound on each GT. Invalidate the TLBs across all GT before releasing the pages back to the system. Signed-off-by: Chris Wilson Cc: Fei Yang Signed-off-by: Mauro Carvalho Chehab --- See [PATCH 00/21] at: https://lore.kernel.org/all/cover.1657703926.git.mche= hab@kernel.org/ drivers/gpu/drm/i915/gem/i915_gem_object_types.h | 4 +++- drivers/gpu/drm/i915/gem/i915_gem_pages.c | 13 ++++++++----- drivers/gpu/drm/i915/gt/intel_engine.h | 1 + drivers/gpu/drm/i915/gt/intel_gt_buffer_pool.h | 3 ++- drivers/gpu/drm/i915/gt/intel_gt_defines.h | 11 +++++++++++ drivers/gpu/drm/i915/gt/intel_gt_types.h | 4 +++- drivers/gpu/drm/i915/gt/intel_ppgtt.c | 4 ++-- drivers/gpu/drm/i915/i915_drv.h | 1 - drivers/gpu/drm/i915/i915_vma.c | 14 +++++++++++--- drivers/gpu/drm/i915/i915_vma.h | 2 +- 10 files changed, 42 insertions(+), 15 deletions(-) create mode 100644 drivers/gpu/drm/i915/gt/intel_gt_defines.h diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object_types.h b/drivers/gpu= /drm/i915/gem/i915_gem_object_types.h index 9f6b14ec189a..3c1d0b750a67 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_object_types.h +++ b/drivers/gpu/drm/i915/gem/i915_gem_object_types.h @@ -17,6 +17,8 @@ #include "i915_selftest.h" #include "i915_vma_resource.h" =20 +#include "gt/intel_gt_defines.h" + struct drm_i915_gem_object; struct intel_fronbuffer; struct intel_memory_region; @@ -616,7 +618,7 @@ struct drm_i915_gem_object { */ bool dirty:1; =20 - u32 tlb; + u32 tlb[I915_MAX_GT]; } mm; =20 struct { diff --git a/drivers/gpu/drm/i915/gem/i915_gem_pages.c b/drivers/gpu/drm/i9= 15/gem/i915_gem_pages.c index 1cd76cc5d9f3..4a6a2f2e8148 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_pages.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_pages.c @@ -194,13 +194,16 @@ static void unmap_object(struct drm_i915_gem_object *= obj, void *ptr) static void flush_tlb_invalidate(struct drm_i915_gem_object *obj) { struct drm_i915_private *i915 =3D to_i915(obj->base.dev); - struct intel_gt *gt =3D to_gt(i915); + struct intel_gt *gt; + int id; =20 - if (!obj->mm.tlb) - return; + for_each_gt(gt, i915, id) { + if (!obj->mm.tlb[id]) + continue; =20 - intel_gt_invalidate_tlb_full(gt, obj->mm.tlb); - obj->mm.tlb =3D 0; + intel_gt_invalidate_tlb_full(gt, obj->mm.tlb[id]); + obj->mm.tlb[id] =3D 0; + } } =20 struct sg_table * diff --git a/drivers/gpu/drm/i915/gt/intel_engine.h b/drivers/gpu/drm/i915/= gt/intel_engine.h index 04e435bce79b..fe1dc55bf8f7 100644 --- a/drivers/gpu/drm/i915/gt/intel_engine.h +++ b/drivers/gpu/drm/i915/gt/intel_engine.h @@ -18,6 +18,7 @@ #include "intel_gt_types.h" #include "intel_timeline.h" #include "intel_workarounds.h" +#include "uc/intel_guc_submission.h" =20 struct drm_printer; struct intel_context; diff --git a/drivers/gpu/drm/i915/gt/intel_gt_buffer_pool.h b/drivers/gpu/d= rm/i915/gt/intel_gt_buffer_pool.h index 487b8a5520f1..8d41cf0c937a 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt_buffer_pool.h +++ b/drivers/gpu/drm/i915/gt/intel_gt_buffer_pool.h @@ -11,8 +11,9 @@ #include "i915_active.h" #include "intel_gt_buffer_pool_types.h" =20 -struct intel_gt; +enum i915_map_type; struct i915_request; +struct intel_gt; =20 struct intel_gt_buffer_pool_node * intel_gt_get_buffer_pool(struct intel_gt *gt, size_t size, diff --git a/drivers/gpu/drm/i915/gt/intel_gt_defines.h b/drivers/gpu/drm/i= 915/gt/intel_gt_defines.h new file mode 100644 index 000000000000..7c711726d663 --- /dev/null +++ b/drivers/gpu/drm/i915/gt/intel_gt_defines.h @@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: MIT */ +/* + * Copyright =C2=A9 2019 Intel Corporation + */ + +#ifndef __INTEL_GT_DEFINES__ +#define __INTEL_GT_DEFINES__ + +#define I915_MAX_GT 4 + +#endif diff --git a/drivers/gpu/drm/i915/gt/intel_gt_types.h b/drivers/gpu/drm/i91= 5/gt/intel_gt_types.h index 3804a583382b..b857c3972251 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt_types.h +++ b/drivers/gpu/drm/i915/gt/intel_gt_types.h @@ -19,7 +19,6 @@ #include "uc/intel_uc.h" #include "intel_gsc.h" =20 -#include "i915_vma.h" #include "intel_engine_types.h" #include "intel_gt_buffer_pool_types.h" #include "intel_hwconfig.h" @@ -31,8 +30,11 @@ #include "intel_wakeref.h" #include "pxp/intel_pxp_types.h" =20 +#include "intel_gt_defines.h" + struct drm_i915_private; struct i915_ggtt; +struct i915_vma; struct intel_engine_cs; struct intel_uncore; =20 diff --git a/drivers/gpu/drm/i915/gt/intel_ppgtt.c b/drivers/gpu/drm/i915/g= t/intel_ppgtt.c index 2da6c82a8bd2..f764d250e929 100644 --- a/drivers/gpu/drm/i915/gt/intel_ppgtt.c +++ b/drivers/gpu/drm/i915/gt/intel_ppgtt.c @@ -8,6 +8,7 @@ #include "gem/i915_gem_lmem.h" =20 #include "i915_trace.h" +#include "intel_gt.h" #include "intel_gtt.h" #include "gen6_ppgtt.h" #include "gen8_ppgtt.h" @@ -210,8 +211,7 @@ void ppgtt_unbind_vma(struct i915_address_space *vm, return; =20 vm->clear_range(vm, vma_res->start, vma_res->vma_size); - if (vma_res->tlb) - vma_invalidate_tlb(vm, *vma_res->tlb); + vma_invalidate_tlb(vm, vma_res->tlb); } =20 static unsigned long pd_count(u64 size, int shift) diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_dr= v.h index d25647be25d1..f1f70257dbe0 100644 --- a/drivers/gpu/drm/i915/i915_drv.h +++ b/drivers/gpu/drm/i915/i915_drv.h @@ -711,7 +711,6 @@ struct drm_i915_private { /* * i915->gt[0] =3D=3D &i915->gt0 */ -#define I915_MAX_GT 4 struct intel_gt *gt[I915_MAX_GT]; =20 struct kobject *sysfs_gt; diff --git a/drivers/gpu/drm/i915/i915_vma.c b/drivers/gpu/drm/i915/i915_vm= a.c index fe947d1456d5..5edc745dcc51 100644 --- a/drivers/gpu/drm/i915/i915_vma.c +++ b/drivers/gpu/drm/i915/i915_vma.c @@ -1309,8 +1309,14 @@ I915_SELFTEST_EXPORT int i915_vma_get_pages(struct i= 915_vma *vma) return err; } =20 -void vma_invalidate_tlb(struct i915_address_space *vm, u32 tlb) +void vma_invalidate_tlb(struct i915_address_space *vm, u32 *tlb) { + struct intel_gt *gt; + int id; + + if (!tlb) + return; + /* * Before we release the pages that were bound by this vma, we * must invalidate all the TLBs that may still have a reference @@ -1319,7 +1325,9 @@ void vma_invalidate_tlb(struct i915_address_space *vm= , u32 tlb) * the most recent TLB invalidation seqno, and if we have not yet * flushed the TLBs upon release, perform a full invalidation. */ - WRITE_ONCE(tlb, intel_gt_next_invalidate_tlb_full(vm->gt)); + for_each_gt(gt, vm->i915, id) + WRITE_ONCE(tlb[id], + intel_gt_next_invalidate_tlb_full(vm->gt)); } =20 static void __vma_put_pages(struct i915_vma *vma, unsigned int count) @@ -1955,7 +1963,7 @@ struct dma_fence *__i915_vma_evict(struct i915_vma *v= ma, bool async) =20 if (async) unbind_fence =3D i915_vma_resource_unbind(vma_res, - &vma->obj->mm.tlb); + vma->obj->mm.tlb); else unbind_fence =3D i915_vma_resource_unbind(vma_res, NULL); =20 diff --git a/drivers/gpu/drm/i915/i915_vma.h b/drivers/gpu/drm/i915/i915_vm= a.h index 5048eed536da..33a58f605d75 100644 --- a/drivers/gpu/drm/i915/i915_vma.h +++ b/drivers/gpu/drm/i915/i915_vma.h @@ -213,7 +213,7 @@ bool i915_vma_misplaced(const struct i915_vma *vma, u64 size, u64 alignment, u64 flags); void __i915_vma_set_map_and_fenceable(struct i915_vma *vma); void i915_vma_revoke_mmap(struct i915_vma *vma); -void vma_invalidate_tlb(struct i915_address_space *vm, u32 tlb); +void vma_invalidate_tlb(struct i915_address_space *vm, u32 *tlb); struct dma_fence *__i915_vma_evict(struct i915_vma *vma, bool async); int __i915_vma_unbind(struct i915_vma *vma); int __must_check i915_vma_unbind(struct i915_vma *vma); --=20 2.36.1 From nobody Sat Apr 18 14:11:47 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2EC06C43334 for ; Wed, 13 Jul 2022 09:31:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236008AbiGMJbb (ORCPT ); Wed, 13 Jul 2022 05:31:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44210 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235709AbiGMJag (ORCPT ); Wed, 13 Jul 2022 05:30:36 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 39B6CF422A for ; Wed, 13 Jul 2022 02:30:27 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 0B2E8B81D6C for ; Wed, 13 Jul 2022 09:30:24 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0BCBCC341CA; Wed, 13 Jul 2022 09:30:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1657704622; bh=gq1c2dnJ31eBwAsmdC8JxTbRoY8LDtkL7UuzgIkibDk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cA2NBftAeUC1eYlwzo/9T8vdUOsjh023WYuR14qau0WsbUh7GpuL44NcNhlO0NTVV IeXq9o91B2mS4Q/9yyDit4DMjlX1JTLT1k7wSD7rE4rcc1exQuxHri5knL+cva9SoX MNMJ89Ul+un/KEsJcfDWbN1p9vLknnxqsTky9VwO0FKAHle+rIkTa79wHkjlDuvGgv 5DyfHKd8aPwNGJ4UqR71bk71xMTXLbaF3bdJPp6aZgva6sPrFmAan0UlJEZfIYz9nj cWECJJBttuAtiju1XCZqlmaEk7IVwvoFqDFocMRas8txwQzK8C/0hV4ds/zngekuBZ 4hejkK2f98dDA== Received: from mchehab by mail.kernel.org with local (Exim 4.95) (envelope-from ) id 1oBYhH-0050Ls-Rm; Wed, 13 Jul 2022 10:30:19 +0100 From: Mauro Carvalho Chehab Cc: Mauro Carvalho Chehab , =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= , Chris Wilson , Daniel Vetter , David Airlie , Jani Nikula , Joonas Lahtinen , Matthew Auld , Rodrigo Vivi , Tvrtko Ursulin , dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, linux-kernel@vger.kernel.org Subject: [PATCH 14/21] drm/i915: document tlb field at struct drm_i915_gem_object Date: Wed, 13 Jul 2022 10:30:11 +0100 Message-Id: X-Mailer: git-send-email 2.36.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add documentation to the TLB field inside struct drm_i915_gem_object. Signed-off-by: Mauro Carvalho Chehab --- See [PATCH 00/21] at: https://lore.kernel.org/all/cover.1657703926.git.mche= hab@kernel.org/ drivers/gpu/drm/i915/gem/i915_gem_object_types.h | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object_types.h b/drivers/gpu= /drm/i915/gem/i915_gem_object_types.h index 3c1d0b750a67..6f5b9e34a4d7 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_object_types.h +++ b/drivers/gpu/drm/i915/gem/i915_gem_object_types.h @@ -618,6 +618,7 @@ struct drm_i915_gem_object { */ bool dirty:1; =20 + /** @mm.tlb: array with TLB invalidate IDs */ u32 tlb[I915_MAX_GT]; } mm; =20 --=20 2.36.1 From nobody Sat Apr 18 14:11:47 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE103C433EF for ; Wed, 13 Jul 2022 09:30:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235665AbiGMJaf (ORCPT ); Wed, 13 Jul 2022 05:30:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43754 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235294AbiGMJaY (ORCPT ); Wed, 13 Jul 2022 05:30:24 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D542CEA14B for ; Wed, 13 Jul 2022 02:30:23 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 41D2161C5E for ; Wed, 13 Jul 2022 09:30:23 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 20A05C341D3; Wed, 13 Jul 2022 09:30:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1657704622; bh=d7Ye7f/yZMeD0fzLWJnDFqz2oIBKUGw1N39MIHO+bsk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Yp/pZSEkoUvsGlC4rq7mlKo7cz2NpoAbkOwvGR8oKgWgtsy9Y0PYmJlYtjLh1w3iL Mh1wKOxxtP+OtWLAcjPRrPFjDKJDFr69KeSIN9wh+XG0JFNv66PSzS3wOZCNLYa2di hJ1AhfNhbgm2tMw5M4tBNjhTlL0b4LqziQhCSLpKcj68m9fshW+z26XHaSX0NWlr1z JY7+6wDu1u9TBhJwhEcBY+4W3w/BjiThWWS1FxilJIYUXJmyhNUr3T/hL3hZEuqCNr 8jZvfKp22fAha3A/Q8/IXLsURheYzJs/LvP4XE6dpAMFOq7h3mUtmLUgzthQAii3a7 e8v6uoFpP5ddA== Received: from mchehab by mail.kernel.org with local (Exim 4.95) (envelope-from ) id 1oBYhH-0050Lx-ST; Wed, 13 Jul 2022 10:30:19 +0100 From: Mauro Carvalho Chehab Cc: Prathap Kumar Valsan , Daniel Vetter , David Airlie , Jani Nikula , Joonas Lahtinen , Rodrigo Vivi , Tvrtko Ursulin , dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, linux-kernel@vger.kernel.org, Niranjana Vishwanathapura , Mauro Carvalho Chehab Subject: [PATCH 15/21] drm/i915: Add platform macro for selective tlb flush Date: Wed, 13 Jul 2022 10:30:12 +0100 Message-Id: <62c27558f7add68cc60fcf765e591f8886a195f9.1657703926.git.mchehab@kernel.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Prathap Kumar Valsan Add support for selective TLB invalidation, which is a platform feature supported on XeHP. Signed-off-by: Prathap Kumar Valsan Cc: Niranjana Vishwanathapura Signed-off-by: Mauro Carvalho Chehab --- See [PATCH 00/21] at: https://lore.kernel.org/all/cover.1657703926.git.mche= hab@kernel.org/ drivers/gpu/drm/i915/i915_drv.h | 3 +++ drivers/gpu/drm/i915/i915_pci.c | 1 + drivers/gpu/drm/i915/intel_device_info.h | 1 + 3 files changed, 5 insertions(+) diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_dr= v.h index f1f70257dbe0..73494960a3a8 100644 --- a/drivers/gpu/drm/i915/i915_drv.h +++ b/drivers/gpu/drm/i915/i915_drv.h @@ -1312,6 +1312,9 @@ IS_SUBPLATFORM(const struct drm_i915_private *i915, =20 #define HAS_GT_UC(dev_priv) (INTEL_INFO(dev_priv)->has_gt_uc) =20 +#define HAS_SELECTIVE_TLB_INVALIDATION(dev_priv) \ + (INTEL_INFO(dev_priv)->has_selective_tlb_invalidation) + #define HAS_POOLED_EU(dev_priv) (INTEL_INFO(dev_priv)->has_pooled_eu) =20 #define HAS_GLOBAL_MOCS_REGISTERS(dev_priv) (INTEL_INFO(dev_priv)->has_glo= bal_mocs) diff --git a/drivers/gpu/drm/i915/i915_pci.c b/drivers/gpu/drm/i915/i915_pc= i.c index aacc10f2e73f..30d945fe384b 100644 --- a/drivers/gpu/drm/i915/i915_pci.c +++ b/drivers/gpu/drm/i915/i915_pci.c @@ -1022,6 +1022,7 @@ static const struct intel_device_info adl_p_info =3D { .has_reset_engine =3D 1, \ .has_rps =3D 1, \ .has_runtime_pm =3D 1, \ + .has_selective_tlb_invalidation =3D 1, \ .ppgtt_size =3D 48, \ .ppgtt_type =3D INTEL_PPGTT_FULL =20 diff --git a/drivers/gpu/drm/i915/intel_device_info.h b/drivers/gpu/drm/i91= 5/intel_device_info.h index 23bf230aa104..92a38b8f7c47 100644 --- a/drivers/gpu/drm/i915/intel_device_info.h +++ b/drivers/gpu/drm/i915/intel_device_info.h @@ -170,6 +170,7 @@ enum intel_ppgtt_type { func(has_rc6p); \ func(has_rps); \ func(has_runtime_pm); \ + func(has_selective_tlb_invalidation); \ func(has_snoop); \ func(has_coherent_ggtt); \ func(unfenced_needs_alignment); \ --=20 2.36.1 From nobody Sat Apr 18 14:11:47 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D333CC433EF for ; Wed, 13 Jul 2022 09:30:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235874AbiGMJao (ORCPT ); Wed, 13 Jul 2022 05:30:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43780 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235369AbiGMJaZ (ORCPT ); Wed, 13 Jul 2022 05:30:25 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 19275EEA9C for ; Wed, 13 Jul 2022 02:30:24 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 9FB7E61CB1 for ; Wed, 13 Jul 2022 09:30:23 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6096BC341E3; Wed, 13 Jul 2022 09:30:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1657704622; bh=J2Sa3hm2zBlgOEeZya1e1BBIqVwbiPlLAjT/AUvYXTE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lYftaMgrB8JRa6YDJa9IORDSoDop3EuYuVcAN2TD2kSI5wZ4T/PNkcPY2yLUWFSj1 ViBSrvqxZoGwFOzJHGOtSOw9LEP1lcFMYMCyMobcaQlEK68votQfeE4sfulYt9mchC AeeT6e+mX23zVbUX0HGZkNzJYmD7f2K68lhf6EuinbUGKURGdYUxkArnUbKbBqYgOm CUoKG9/nDljTVTTI8jWZgxW1rxAieP9gPvL9pmN3H1ZL9DRaBwDxGa5tcBLEXD44hH tEsj45BUf1P4iSOvYKfhweIggJaopyHmgHXRtT91sAQaGV4XrwkD6+JbI+lSirmHNc Kp0n7eL+Z38CA== Received: from mchehab by mail.kernel.org with local (Exim 4.95) (envelope-from ) id 1oBYhH-0050M0-TG; Wed, 13 Jul 2022 10:30:19 +0100 From: Mauro Carvalho Chehab Cc: Prathap Kumar Valsan , Alan Previn , Borislav Petkov , Daniel Vetter , Daniele Ceraolo Spurio , David Airlie , Jani Nikula , John Harrison , Joonas Lahtinen , Lucas De Marchi , Matt Roper , Matthew Brost , Mauro Carvalho Chehab , Michal Wajdeczko , Rodrigo Vivi , Tvrtko Ursulin , Umesh Nerlige Ramappa , Vinay Belgaumkar , dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, linux-kernel@vger.kernel.org Subject: [PATCH 16/21] drm/i915: Define GuC Based TLB invalidation routines Date: Wed, 13 Jul 2022 10:30:13 +0100 Message-Id: <06135f47cc14dc64a370d9800ae1ee689689cb71.1657703926.git.mchehab@kernel.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Prathap Kumar Valsan Add routines to interface with GuC firmware for selective TLB invalidation supported on XeHP. Signed-off-by: Prathap Kumar Valsan Cc: Matthew Brost Signed-off-by: Mauro Carvalho Chehab --- See [PATCH 00/21] at: https://lore.kernel.org/all/cover.1657703926.git.mche= hab@kernel.org/ .../gpu/drm/i915/gt/uc/abi/guc_actions_abi.h | 3 + drivers/gpu/drm/i915/gt/uc/intel_guc.c | 90 +++++++++++++++++++ drivers/gpu/drm/i915/gt/uc/intel_guc.h | 10 +++ drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h | 3 + 4 files changed, 106 insertions(+) diff --git a/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h b/drivers/gpu= /drm/i915/gt/uc/abi/guc_actions_abi.h index fb0af33e43cc..5c019856a269 100644 --- a/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h +++ b/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h @@ -188,6 +188,9 @@ enum intel_guc_state_capture_event_status { #define INTEL_GUC_TLB_INVAL_FLUSH_CACHE (1 << 31) =20 enum intel_guc_tlb_invalidation_type { + INTEL_GUC_TLB_INVAL_FULL =3D 0x0, + INTEL_GUC_TLB_INVAL_PAGE_SELECTIVE =3D 0x1, + INTEL_GUC_TLB_INVAL_PAGE_SELECTIVE_CTX =3D 0x2, INTEL_GUC_TLB_INVAL_GUC =3D 0x3, }; =20 diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc.c b/drivers/gpu/drm/i915/= gt/uc/intel_guc.c index 8a104a292598..98260a7bc90b 100644 --- a/drivers/gpu/drm/i915/gt/uc/intel_guc.c +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc.c @@ -923,6 +923,96 @@ static int guc_send_invalidate_tlb(struct intel_guc *g= uc, u32 *action, u32 size) return err; } =20 + /* Full TLB invalidation */ +int intel_guc_invalidate_tlb_full(struct intel_guc *guc, + enum intel_guc_tlb_inval_mode mode) +{ + u32 action[] =3D { + INTEL_GUC_ACTION_TLB_INVALIDATION, + 0, + INTEL_GUC_TLB_INVAL_FULL << INTEL_GUC_TLB_INVAL_TYPE_SHIFT | + mode << INTEL_GUC_TLB_INVAL_MODE_SHIFT | + INTEL_GUC_TLB_INVAL_FLUSH_CACHE, + }; + + if (!INTEL_GUC_SUPPORTS_TLB_INVALIDATION(guc)) { + DRM_ERROR("Tlb invalidation: Operation not supported in this platform!\n= "); + return 0; + } + + return guc_send_invalidate_tlb(guc, action, ARRAY_SIZE(action)); +} + +/* + * Selective TLB Invalidation for Address Range: + * TLB's in the Address Range is Invalidated across all engines. + */ +int intel_guc_invalidate_tlb_page_selective(struct intel_guc *guc, + enum intel_guc_tlb_inval_mode mode, + u64 start, u64 length) +{ + u64 vm_total =3D BIT_ULL(INTEL_INFO(guc_to_gt(guc)->i915)->ppgtt_size); + u32 address_mask =3D (ilog2(length) - ilog2(I915_GTT_PAGE_SIZE_4K)); + u32 full_range =3D vm_total =3D=3D length; + u32 action[] =3D { + INTEL_GUC_ACTION_TLB_INVALIDATION, + 0, + INTEL_GUC_TLB_INVAL_PAGE_SELECTIVE << INTEL_GUC_TLB_INVAL_TYPE_SHIFT | + mode << INTEL_GUC_TLB_INVAL_MODE_SHIFT | + INTEL_GUC_TLB_INVAL_FLUSH_CACHE, + 0, + full_range ? full_range : lower_32_bits(start), + full_range ? 0 : upper_32_bits(start), + full_range ? 0 : address_mask, + }; + + if (!INTEL_GUC_SUPPORTS_TLB_INVALIDATION_SELECTIVE(guc)) { + DRM_ERROR("Tlb invalidation: Operation not supported in this platform!\n= "); + return 0; + } + + GEM_BUG_ON(!IS_ALIGNED(start, I915_GTT_PAGE_SIZE_4K)); + GEM_BUG_ON(!IS_ALIGNED(length, I915_GTT_PAGE_SIZE_4K)); + GEM_BUG_ON(range_overflows(start, length, vm_total)); + + return guc_send_invalidate_tlb(guc, action, ARRAY_SIZE(action)); +} + +/* + * Selective TLB Invalidation for Context: + * Invalidates all TLB's for a specific context across all engines. + */ +int intel_guc_invalidate_tlb_page_selective_ctx(struct intel_guc *guc, + enum intel_guc_tlb_inval_mode mode, + u64 start, u64 length, u32 ctxid) +{ + u64 vm_total =3D BIT_ULL(INTEL_INFO(guc_to_gt(guc)->i915)->ppgtt_size); + u32 address_mask =3D (ilog2(length) - ilog2(I915_GTT_PAGE_SIZE_4K)); + u32 full_range =3D vm_total =3D=3D length; + u32 action[] =3D { + INTEL_GUC_ACTION_TLB_INVALIDATION, + 0, + INTEL_GUC_TLB_INVAL_PAGE_SELECTIVE_CTX << INTEL_GUC_TLB_INVAL_TYPE_SHIFT= | + mode << INTEL_GUC_TLB_INVAL_MODE_SHIFT | + INTEL_GUC_TLB_INVAL_FLUSH_CACHE, + ctxid, + full_range ? full_range : lower_32_bits(start), + full_range ? 0 : upper_32_bits(start), + full_range ? 0 : address_mask, + }; + + if (!INTEL_GUC_SUPPORTS_TLB_INVALIDATION_SELECTIVE(guc)) { + DRM_ERROR("Tlb invalidation: Operation not supported in this platform!\n= "); + return 0; + } + + GEM_BUG_ON(!IS_ALIGNED(start, I915_GTT_PAGE_SIZE_4K)); + GEM_BUG_ON(!IS_ALIGNED(length, I915_GTT_PAGE_SIZE_4K)); + GEM_BUG_ON(range_overflows(start, length, vm_total)); + + return guc_send_invalidate_tlb(guc, action, ARRAY_SIZE(action)); +} + /* * Guc TLB Invalidation: Invalidate the TLB's of GuC itself. */ diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc.h b/drivers/gpu/drm/i915/= gt/uc/intel_guc.h index 01c6478451cc..df6ba1c32808 100644 --- a/drivers/gpu/drm/i915/gt/uc/intel_guc.h +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc.h @@ -384,6 +384,16 @@ int intel_guc_allocate_and_map_vma(struct intel_guc *g= uc, u32 size, int intel_guc_self_cfg32(struct intel_guc *guc, u16 key, u32 value); int intel_guc_self_cfg64(struct intel_guc *guc, u16 key, u64 value); =20 +int intel_guc_g2g_register(struct intel_guc *guc); + +int intel_guc_invalidate_tlb_full(struct intel_guc *guc, + enum intel_guc_tlb_inval_mode mode); +int intel_guc_invalidate_tlb_page_selective(struct intel_guc *guc, + enum intel_guc_tlb_inval_mode mode, + u64 start, u64 length); +int intel_guc_invalidate_tlb_page_selective_ctx(struct intel_guc *guc, + enum intel_guc_tlb_inval_mode mode, + u64 start, u64 length, u32 ctxid); int intel_guc_invalidate_tlb_guc(struct intel_guc *guc, enum intel_guc_tlb_inval_mode mode); int intel_guc_invalidate_tlb_all(struct intel_guc *guc); diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h b/drivers/gpu/drm/= i915/gt/uc/intel_guc_fwif.h index 3edf567b3f65..29e402f70a94 100644 --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h @@ -436,5 +436,8 @@ enum intel_guc_recv_message { ((intel_guc_ct_enabled(&(guc)->ct)) && \ (intel_guc_submission_is_used(guc)) && \ (GRAPHICS_VER(guc_to_gt((guc))->i915) >=3D 12)) +#define INTEL_GUC_SUPPORTS_TLB_INVALIDATION_SELECTIVE(guc) \ + (INTEL_GUC_SUPPORTS_TLB_INVALIDATION(guc) && \ + HAS_SELECTIVE_TLB_INVALIDATION(guc_to_gt(guc)->i915)) =20 #endif --=20 2.36.1 From nobody Sat Apr 18 14:11:47 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 83DAEC43334 for ; Wed, 13 Jul 2022 09:31:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236111AbiGMJbj (ORCPT ); Wed, 13 Jul 2022 05:31:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43826 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235705AbiGMJag (ORCPT ); Wed, 13 Jul 2022 05:30:36 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F3C2BF4223 for ; Wed, 13 Jul 2022 02:30:26 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 71804B81D6F for ; Wed, 13 Jul 2022 09:30:24 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 494E1C341DF; Wed, 13 Jul 2022 09:30:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1657704622; bh=FzHmmGEGNNUQo1E9IcdaPTJZyRSYl5vZePFi9GRAEHc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lIDUyc5sj0v2wkHK56P1equ4UXbgwyIAXhnE3oNV2uyRPpIj8R+5fPCLPA+GU1bZ5 EFtt1JyvpN1R2WIVfgma2jTVSj8JlA6/0JattwdNRm32XFF9+Qu7kEe/faIq2mV4KK r8SxafBS3sF10yblxA6p/qvys6BaWG72hDVPmRorIFuKPUl1lo+Ta/X/dihpSsnrWU iruwCXI3+AaO0eiOFn2RilKWStlvCaeXwu2v/ljGgpB+LBB887/CdXlFliuKq7x2nO fF49RtdliWmCE+SFFcZXpOCCPEmbSrz4Z+bDDXw6q+TvEMOM9SVrII8Xe3AQNg90nD PuUjAF8ka2pPw== Received: from mchehab by mail.kernel.org with local (Exim 4.95) (envelope-from ) id 1oBYhH-0050M3-U5; Wed, 13 Jul 2022 10:30:19 +0100 From: Mauro Carvalho Chehab Cc: Prathap Kumar Valsan , Ashutosh Dixit , Chris Wilson , Daniel Vetter , Daniele Ceraolo Spurio , David Airlie , Jani Nikula , Joonas Lahtinen , Lucas De Marchi , Matt Atwood , Matt Roper , Mauro Carvalho Chehab , Rodrigo Vivi , Tvrtko Ursulin , dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, linux-kernel@vger.kernel.org, Niranjana Vishwanathapura , Fei Yang Subject: [PATCH 17/21] drm/i915: Add generic interface for tlb invalidation for XeHP Date: Wed, 13 Jul 2022 10:30:14 +0100 Message-Id: X-Mailer: git-send-email 2.36.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Prathap Kumar Valsan Add an interface for GuC TLB actions, supporting both selective and full TLB invalidations. After this change, when GuC is enabled, tlb invalidations use GuC ct. Otherwise, use mmio interface. Signed-off-by: Prathap Kumar Valsan Cc: Niranjana Vishwanathapura Cc: Fei Yang Signed-off-by: Mauro Carvalho Chehab --- See [PATCH 00/21] at: https://lore.kernel.org/all/cover.1657703926.git.mche= hab@kernel.org/ drivers/gpu/drm/i915/gt/intel_gt_regs.h | 8 +++ drivers/gpu/drm/i915/gt/intel_tlb.c | 78 ++++++++++++++++++++++++- drivers/gpu/drm/i915/gt/intel_tlb.h | 1 + 3 files changed, 86 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/i915/gt/intel_gt_regs.h b/drivers/gpu/drm/i915= /gt/intel_gt_regs.h index 60d6eb5f245b..52508a9c23e5 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt_regs.h +++ b/drivers/gpu/drm/i915/gt/intel_gt_regs.h @@ -1054,6 +1054,14 @@ =20 #define GEN12_GAM_DONE _MMIO(0xcf68) =20 +#define XEHP_TLB_INV_DESC0 _MMIO(0xcf7c) +#define XEHP_TLB_INV_DESC0_ADDR_LO REG_GENMASK(31, 12) +#define XEHP_TLB_INV_DESC0_ADDR_MASK REG_GENMASK(8, 3) +#define XEHP_TLB_INV_DESC0_G REG_GENMASK(2, 1) +#define XEHP_TLB_INV_DESC0_VALID REG_BIT(0) +#define XEHP_TLB_INV_DESC1 _MMIO(0xcf80) +#define XEHP_TLB_INV_DESC0_ADDR_HI REG_GENMASK(31, 0) + #define GEN7_HALF_SLICE_CHICKEN1 _MMIO(0xe100) /* IVB GT1 + VLV */ #define GEN7_MAX_PS_THREAD_DEP (8 << 12) #define GEN7_SINGLE_SUBSCAN_DISPATCH_ENABLE (1 << 10) diff --git a/drivers/gpu/drm/i915/gt/intel_tlb.c b/drivers/gpu/drm/i915/gt/= intel_tlb.c index af8cae979489..15ed83226676 100644 --- a/drivers/gpu/drm/i915/gt/intel_tlb.c +++ b/drivers/gpu/drm/i915/gt/intel_tlb.c @@ -10,6 +10,7 @@ #include "intel_gt_pm.h" #include "intel_gt_regs.h" #include "intel_tlb.h" +#include "uc/intel_guc.h" =20 struct reg_and_bit { i915_reg_t reg; @@ -159,11 +160,16 @@ void intel_gt_invalidate_tlb_full(struct intel_gt *gt= , u32 seqno) return; =20 with_intel_gt_pm_if_awake(gt, wakeref) { + struct intel_guc *guc =3D >->uc.guc; + mutex_lock(>->tlb.invalidate_lock); if (tlb_seqno_passed(gt, seqno)) goto unlock; =20 - mmio_invalidate_full(gt); + if (INTEL_GUC_SUPPORTS_TLB_INVALIDATION(guc)) + intel_guc_invalidate_tlb_full(guc, INTEL_GUC_TLB_INVAL_MODE_HEAVY); + else + mmio_invalidate_full(gt); =20 write_seqcount_invalidate(>->tlb.seqno); unlock: @@ -171,6 +177,76 @@ void intel_gt_invalidate_tlb_full(struct intel_gt *gt,= u32 seqno) } } =20 +static bool mmio_invalidate_range(struct intel_gt *gt, u64 start, u64 leng= th) +{ + u32 address_mask =3D (ilog2(length) - ilog2(I915_GTT_PAGE_SIZE_4K)); + u64 vm_total =3D BIT_ULL(INTEL_INFO(gt->i915)->ppgtt_size); + intel_wakeref_t wakeref; + u32 dw0, dw1; + int err; + + GEM_BUG_ON(!IS_ALIGNED(start, I915_GTT_PAGE_SIZE_4K)); + GEM_BUG_ON(!IS_ALIGNED(length, I915_GTT_PAGE_SIZE_4K)); + GEM_BUG_ON(range_overflows(start, length, vm_total)); + + dw0 =3D FIELD_PREP(XEHP_TLB_INV_DESC0_ADDR_LO, (lower_32_bits(start) >> 1= 2)) | + FIELD_PREP(XEHP_TLB_INV_DESC0_ADDR_MASK, address_mask) | + FIELD_PREP(XEHP_TLB_INV_DESC0_G, 0x3) | + FIELD_PREP(XEHP_TLB_INV_DESC0_VALID, 0x1); + dw1 =3D upper_32_bits(start); + + err =3D 0; + with_intel_gt_pm_if_awake(gt, wakeref) { + struct intel_uncore *uncore =3D gt->uncore; + + intel_uncore_forcewake_get(uncore, FORCEWAKE_ALL); + + mutex_lock(>->tlb.invalidate_lock); + intel_uncore_write_fw(uncore, XEHP_TLB_INV_DESC1, dw1); + intel_uncore_write_fw(uncore, XEHP_TLB_INV_DESC0, dw0); + err =3D __intel_wait_for_register_fw(uncore, + XEHP_TLB_INV_DESC0, + XEHP_TLB_INV_DESC0_VALID, + 0, 100, 10, NULL); + mutex_unlock(>->tlb.invalidate_lock); + + intel_uncore_forcewake_put_delayed(uncore, FORCEWAKE_ALL); + } + + if (err) + drm_err_ratelimited(>->i915->drm, + "TLB invalidation response timed out\n"); + + return err =3D=3D 0; +} + +bool intel_gt_invalidate_tlb_range(struct intel_gt *gt, + u64 start, u64 length) +{ + struct intel_guc *guc =3D >->uc.guc; + intel_wakeref_t wakeref; + + if (intel_gt_is_wedged(gt)) + return true; + + if (!INTEL_GUC_SUPPORTS_TLB_INVALIDATION_SELECTIVE(guc)) + return false; + + /*XXX: We are seeing timeouts on guc based tlb invalidations on XEHPSDV. + * Until we have a fix, use mmio + */ + if (IS_XEHPSDV(gt->i915)) + return mmio_invalidate_range(gt, start, length); + + with_intel_gt_pm_if_awake(gt, wakeref) { + intel_guc_invalidate_tlb_page_selective(guc, + INTEL_GUC_TLB_INVAL_MODE_HEAVY, + start, length); + } + + return true; +} + void intel_gt_init_tlb(struct intel_gt *gt) { mutex_init(>->tlb.invalidate_lock); diff --git a/drivers/gpu/drm/i915/gt/intel_tlb.h b/drivers/gpu/drm/i915/gt/= intel_tlb.h index 46ce25bf5afe..32cc79b1d8a4 100644 --- a/drivers/gpu/drm/i915/gt/intel_tlb.h +++ b/drivers/gpu/drm/i915/gt/intel_tlb.h @@ -12,6 +12,7 @@ #include "intel_gt_types.h" =20 void intel_gt_invalidate_tlb_full(struct intel_gt *gt, u32 seqno); +bool intel_gt_invalidate_tlb_range(struct intel_gt *gt, u64 start, u64 len= gth); =20 void intel_gt_init_tlb(struct intel_gt *gt); void intel_gt_fini_tlb(struct intel_gt *gt); --=20 2.36.1 From nobody Sat Apr 18 14:11:47 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CBFDCCCA483 for ; Wed, 13 Jul 2022 09:31:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236119AbiGMJbD (ORCPT ); Wed, 13 Jul 2022 05:31:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43826 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235432AbiGMJa0 (ORCPT ); Wed, 13 Jul 2022 05:30:26 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EA0DCF32C3 for ; Wed, 13 Jul 2022 02:30:24 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 0458861CC2 for ; Wed, 13 Jul 2022 09:30:24 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5E005C341E2; Wed, 13 Jul 2022 09:30:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1657704622; bh=No42/RgEW6DsepQiPc1/0fJqZ/T5P5PUlBvfd2kfCqA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pbD9lHpKoIDbugthGJ+HYyhMHuBtLIO3OVFhVRsWIvnwfTn65Nda3/T0bmcI6GwLd pQp7JJVzz6dnN+s3wa3Ci6mYBSoUvEaYQYLJD2U9hZlbbEa82Os3iUD4LhFswTg0is 93xhBa6TQOCRWWsbIyU+vA+QOfQjSb7td6PX1T+1XbJSxhNKjdA2pIP1BPgF8R0Xp5 6D2Zxof3B6myg64sRQ+Z9v9j8qCDO6WfTLSMnj7P6pJYmrnOkNwOge/ToXQcO8+KIl 4llo3nS9jJy3qkd2kGMThP6RLTurUB//aJ30IOk/fEoXEcsPjWeXhMOZlYxMSp7iKQ Og/AbDjfrkNRw== Received: from mchehab by mail.kernel.org with local (Exim 4.95) (envelope-from ) id 1oBYhH-0050M7-Ur; Wed, 13 Jul 2022 10:30:19 +0100 From: Mauro Carvalho Chehab Cc: Prathap Kumar Valsan , =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= , Chris Wilson , Daniel Vetter , David Airlie , Jani Nikula , Joonas Lahtinen , Lucas De Marchi , Maarten Lankhorst , Matthew Auld , Mauro Carvalho Chehab , Ramalingam C , Rodrigo Vivi , Tvrtko Ursulin , dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, linux-kernel@vger.kernel.org, Niranjana Vishwanathapura , Fei Yang Subject: [PATCH 18/21] drm/i915: Use selective tlb invalidations where supported Date: Wed, 13 Jul 2022 10:30:15 +0100 Message-Id: <1eddd79483814571bdd6a6d9781b1637ef05ed6e.1657703926.git.mchehab@kernel.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Prathap Kumar Valsan For platforms supporting selective tlb invalidations, we don't need to do a full tlb invalidation. Rather do a range based tlb invalidation for every unbind of purged vma belongs to an active vm. [mchehab: change moved from intel_ppgtt.c to i915_vma.c] Signed-off-by: Prathap Kumar Valsan Cc: Niranjana Vishwanathapura Cc: Fei Yang Signed-off-by: Mauro Carvalho Chehab --- See [PATCH 00/21] at: https://lore.kernel.org/all/cover.1657703926.git.mche= hab@kernel.org/ drivers/gpu/drm/i915/gt/intel_ppgtt.c | 2 +- drivers/gpu/drm/i915/i915_vma.c | 14 +++++++++----- drivers/gpu/drm/i915/i915_vma.h | 3 ++- 3 files changed, 12 insertions(+), 7 deletions(-) diff --git a/drivers/gpu/drm/i915/gt/intel_ppgtt.c b/drivers/gpu/drm/i915/g= t/intel_ppgtt.c index f764d250e929..74782fb2ccbd 100644 --- a/drivers/gpu/drm/i915/gt/intel_ppgtt.c +++ b/drivers/gpu/drm/i915/gt/intel_ppgtt.c @@ -211,7 +211,7 @@ void ppgtt_unbind_vma(struct i915_address_space *vm, return; =20 vm->clear_range(vm, vma_res->start, vma_res->vma_size); - vma_invalidate_tlb(vm, vma_res->tlb); + vma_invalidate_tlb(vm, vma_res->tlb, vma_res->start, vma_res->vma_size); } =20 static unsigned long pd_count(u64 size, int shift) diff --git a/drivers/gpu/drm/i915/i915_vma.c b/drivers/gpu/drm/i915/i915_vm= a.c index 5edc745dcc51..6d881a6b403a 100644 --- a/drivers/gpu/drm/i915/i915_vma.c +++ b/drivers/gpu/drm/i915/i915_vma.c @@ -1309,7 +1309,8 @@ I915_SELFTEST_EXPORT int i915_vma_get_pages(struct i9= 15_vma *vma) return err; } =20 -void vma_invalidate_tlb(struct i915_address_space *vm, u32 *tlb) +void vma_invalidate_tlb(struct i915_address_space *vm, u32 *tlb, + u64 start, u64 size) { struct intel_gt *gt; int id; @@ -1325,9 +1326,11 @@ void vma_invalidate_tlb(struct i915_address_space *v= m, u32 *tlb) * the most recent TLB invalidation seqno, and if we have not yet * flushed the TLBs upon release, perform a full invalidation. */ - for_each_gt(gt, vm->i915, id) - WRITE_ONCE(tlb[id], - intel_gt_next_invalidate_tlb_full(vm->gt)); + for_each_gt(gt, vm->i915, id) { + if (!intel_gt_invalidate_tlb_range(gt, start, size)) + WRITE_ONCE(tlb[id], + intel_gt_next_invalidate_tlb_full(vm->gt)); + } } =20 static void __vma_put_pages(struct i915_vma *vma, unsigned int count) @@ -1980,7 +1983,8 @@ struct dma_fence *__i915_vma_evict(struct i915_vma *v= ma, bool async) dma_fence_put(unbind_fence); unbind_fence =3D NULL; } - vma_invalidate_tlb(vma->vm, vma->obj->mm.tlb); + vma_invalidate_tlb(vma->vm, vma->obj->mm.tlb, + vma->node.start, vma->size); } =20 /* diff --git a/drivers/gpu/drm/i915/i915_vma.h b/drivers/gpu/drm/i915/i915_vm= a.h index 33a58f605d75..3f0af9595e59 100644 --- a/drivers/gpu/drm/i915/i915_vma.h +++ b/drivers/gpu/drm/i915/i915_vma.h @@ -213,7 +213,8 @@ bool i915_vma_misplaced(const struct i915_vma *vma, u64 size, u64 alignment, u64 flags); void __i915_vma_set_map_and_fenceable(struct i915_vma *vma); void i915_vma_revoke_mmap(struct i915_vma *vma); -void vma_invalidate_tlb(struct i915_address_space *vm, u32 *tlb); +void vma_invalidate_tlb(struct i915_address_space *vm, u32 *tlb, + u64 start, u64 size); struct dma_fence *__i915_vma_evict(struct i915_vma *vma, bool async); int __i915_vma_unbind(struct i915_vma *vma); int __must_check i915_vma_unbind(struct i915_vma *vma); --=20 2.36.1 From nobody Sat Apr 18 14:11:47 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A9CECCA479 for ; Wed, 13 Jul 2022 09:30:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235751AbiGMJai (ORCPT ); Wed, 13 Jul 2022 05:30:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43778 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235359AbiGMJaZ (ORCPT ); Wed, 13 Jul 2022 05:30:25 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 540C1F2E0D for ; Wed, 13 Jul 2022 02:30:24 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id DB85761CBE for ; Wed, 13 Jul 2022 09:30:23 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 57759C385A9; Wed, 13 Jul 2022 09:30:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1657704622; bh=YZ+rekclkz0PmGszpxR0kr32E1UilqxXVk+khwXJKBk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=D8IOZ3Ei1SBs13Sjy4Qij9irRpN3hW3b5vUuhYpTdLu9YQnrMZ+KvNs9AvdvMe6UI NHuwsSDqWB7ri0Bx0sQy1ijNvoPVuEpGfcZ5r6byyZFgRjrEP3yLbfmWG5e9GcgHqw 7ZvxwqkQ8W1a0Ad4N0MD+8Jkv+Ult97rVKirgOazolj+OCq67AiBex+zn1KiqYiByF Q0Ka+07d8H8G70Mh19WMggzLLZ5Varl8kkO3Q+O0Vcdx5lC0b5nQMrTGCNmQVvcU2P mKtdOGXdcUuyAeKlnp4T1QwuF+zAiHcmpLM2E8mynyFRN6v1K74ToaWjd/0oLykTUV ji0jhwbdFgdrg== Received: from mchehab by mail.kernel.org with local (Exim 4.95) (envelope-from ) id 1oBYhH-0050MB-Vc; Wed, 13 Jul 2022 10:30:19 +0100 From: Mauro Carvalho Chehab Cc: Mauro Carvalho Chehab , Chris Wilson , Daniel Vetter , David Airlie , Jani Nikula , Joonas Lahtinen , Prathap Kumar Valsan , Rodrigo Vivi , Tvrtko Ursulin , dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, linux-kernel@vger.kernel.org Subject: [PATCH 19/21] drm/i915/gt: document TLB cache invalidation functions Date: Wed, 13 Jul 2022 10:30:16 +0100 Message-Id: X-Mailer: git-send-email 2.36.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add a description for the kAPI functions inside intel_tlb.c. Signed-off-by: Mauro Carvalho Chehab --- See [PATCH 00/21] at: https://lore.kernel.org/all/cover.1657703926.git.mche= hab@kernel.org/ drivers/gpu/drm/i915/gt/intel_tlb.c | 36 +++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/drivers/gpu/drm/i915/gt/intel_tlb.c b/drivers/gpu/drm/i915/gt/= intel_tlb.c index 15ed83226676..aa2e0086ae88 100644 --- a/drivers/gpu/drm/i915/gt/intel_tlb.c +++ b/drivers/gpu/drm/i915/gt/intel_tlb.c @@ -146,6 +146,18 @@ static void mmio_invalidate_full(struct intel_gt *gt) intel_uncore_forcewake_put_delayed(uncore, FORCEWAKE_ALL); } =20 +/** + * intel_gt_invalidate_tlb_full - do full TLB cache invalidation + * @gt: GT structure + * @seqno: sequence number + * + * Do a full TLB cache invalidation if the @seqno is bigger than the last + * full TLB cache invalidation. + * + * Note: + * The TLB cache invalidation logic depends on GEN-specific registers. + * It currently supports GEN8 to GEN12 and GuC-based TLB cache invalidatio= n. + */ void intel_gt_invalidate_tlb_full(struct intel_gt *gt, u32 seqno) { intel_wakeref_t wakeref; @@ -220,6 +232,17 @@ static bool mmio_invalidate_range(struct intel_gt *gt,= u64 start, u64 length) return err =3D=3D 0; } =20 +/** + * intel_gt_invalidate_tlb_range - do full TLB cache invalidation + * @gt: GT structure + * @start: range start + * @length: range length + * + * Do a selected TLB cache invalidation on a range pointed by @start + * with @length size. + * + * Only some GuC-based GPUs can do a selective cache invalidation. + */ bool intel_gt_invalidate_tlb_range(struct intel_gt *gt, u64 start, u64 length) { @@ -247,12 +270,25 @@ bool intel_gt_invalidate_tlb_range(struct intel_gt *g= t, return true; } =20 +/** + * intel_gt_init_tlb - initialize TLB-specific vars + * @gt: GT structure + * + * TLB cache invalidation logic internally uses some resources that require + * initialization. Should be called before doing any TLB cache invalidatio= n. + */ void intel_gt_init_tlb(struct intel_gt *gt) { mutex_init(>->tlb.invalidate_lock); seqcount_mutex_init(>->tlb.seqno, >->tlb.invalidate_lock); } =20 +/** + * intel_gt_fini_tlb - initialize TLB-specific vars + * @gt: GT structure + * + * Frees any resources needed by TLB cache invalidation logic. + */ void intel_gt_fini_tlb(struct intel_gt *gt) { mutex_destroy(>->tlb.invalidate_lock); --=20 2.36.1 From nobody Sat Apr 18 14:11:47 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0500DC43334 for ; Wed, 13 Jul 2022 09:30:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235778AbiGMJal (ORCPT ); Wed, 13 Jul 2022 05:30:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43764 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235315AbiGMJaZ (ORCPT ); Wed, 13 Jul 2022 05:30:25 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 547C2F2E0F for ; Wed, 13 Jul 2022 02:30:24 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id CE08661CBB for ; Wed, 13 Jul 2022 09:30:23 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 74B90C341E6; Wed, 13 Jul 2022 09:30:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1657704622; bh=CCKKzwB0qHXF+WGVUFGTqABjxYBM8eht5j02XIx4jVY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uTJLFfDrWmXusPvlk/y7wR2q/VA5ySyre0cG0m02FfgFLLaJ5BrBmOK7FqmakpLJS xYVl9PR4l2pc/7fAeiOqKtV17Jx6+jBmlvDzKuRGhYHZyfZ1yaONRl9GyhIJv+USmx nOub+MnFujdOK7+c1oh21empX7bgoMedy0TclG2SLbMpSZGbu6P57Oaz6saHaYp8QW ZOMi1JMwjeYCR55Z9VEGP+aeUYlnozoMRLjNUjIa9pxNcQCrp3JeZf5bgSTeCpA2Fa 8UDVuTgFBEOfJXuxw7gg9JfHzm8HER+L562SdDQl8O4iLnZwg5+1vEyB8ybY9c8gnL YYenUXTz8VkUg== Received: from mchehab by mail.kernel.org with local (Exim 4.95) (envelope-from ) id 1oBYhI-0050MF-04; Wed, 13 Jul 2022 10:30:20 +0100 From: Mauro Carvalho Chehab Cc: Mauro Carvalho Chehab , Alan Previn , Borislav Petkov , Daniel Vetter , David Airlie , Jani Nikula , John Harrison , Joonas Lahtinen , Matthew Brost , Michal Wajdeczko , Prathap Kumar Valsan , Rodrigo Vivi , Tvrtko Ursulin , Vinay Belgaumkar , dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, linux-kernel@vger.kernel.org Subject: [PATCH 20/21] drm/i915/guc: describe enum intel_guc_tlb_invalidation_type Date: Wed, 13 Jul 2022 10:30:17 +0100 Message-Id: <73d51e0580ba98a466f951473902eda3cfa97053.1657703926.git.mchehab@kernel.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add a description for intel_guc_tlb_invalidation_type enum. Signed-off-by: Mauro Carvalho Chehab --- See [PATCH 00/21] at: https://lore.kernel.org/all/cover.1657703926.git.mche= hab@kernel.org/ drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h b/drivers/gpu= /drm/i915/gt/uc/abi/guc_actions_abi.h index 5c019856a269..e97065c62d28 100644 --- a/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h +++ b/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h @@ -187,6 +187,18 @@ enum intel_guc_state_capture_event_status { /* Flush PPC or SMRO caches along with TLB invalidation request */ #define INTEL_GUC_TLB_INVAL_FLUSH_CACHE (1 << 31) =20 +/** + * enum intel_guc_tlb_invalidation_type - type of TLB cache invalidation + * + * @INTEL_GUC_TLB_INVAL_FULL: + * Global TLB invalidation + * @INTEL_GUC_TLB_INVAL_PAGE_SELECTIVE: + * Page-selective TLB cache invalidation + * @INTEL_GUC_TLB_INVAL_PAGE_SELECTIVE_CTX: + * Context-selective TLB cache invalidation + * @INTEL_GUC_TLB_INVAL_GUC: + * Invalidate TLB on GuC itself + */ enum intel_guc_tlb_invalidation_type { INTEL_GUC_TLB_INVAL_FULL =3D 0x0, INTEL_GUC_TLB_INVAL_PAGE_SELECTIVE =3D 0x1, --=20 2.36.1 From nobody Sat Apr 18 14:11:47 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EDFF9C433EF for ; Wed, 13 Jul 2022 09:31:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236154AbiGMJbW (ORCPT ); Wed, 13 Jul 2022 05:31:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43998 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235545AbiGMJaa (ORCPT ); Wed, 13 Jul 2022 05:30:30 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 85977F32E1 for ; Wed, 13 Jul 2022 02:30:25 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 1694C61CCE for ; Wed, 13 Jul 2022 09:30:24 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6567CC341E0; Wed, 13 Jul 2022 09:30:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1657704622; bh=MTo5LFJQmGswJidJuW/ez8S5JqY/AKBuALalK+cXQ8g=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=i6VeyuVSXZZSy5+N68xJ7UWpZDSidffSrdw4xCYZWbZ1dHZz1Py5khbHhDZLWvr6O 6DMNp8RJ0NKHZ3l9QN4O2WIrIduhDgmLtgo3Pag0KSkdB5kEU8k4bL1k/wmcZOxzTe GLQ94XCBNRELc17vCSdYPiiCLjHu5xHZCBalbCkPzC85Z13M3wXjxI7EE5bVJcMvzA kjCZJ2fK27RTma6tNZpFTPVrZrkPa4CtAiQ6tEQTXkDZYBI0qOiLTloF0VvkCEqF8o Lt6wZhCMdT8rzT1qmB25wyhWM2WY27BnZ0AhU/PA1zqnRRX7Hk7hSEs4Z/4p5XwvMU UQucqp0c2+uJg== Received: from mchehab by mail.kernel.org with local (Exim 4.95) (envelope-from ) id 1oBYhI-0050MJ-0i; Wed, 13 Jul 2022 10:30:20 +0100 From: Mauro Carvalho Chehab Cc: Mauro Carvalho Chehab , Daniel Vetter , Daniele Ceraolo Spurio , David Airlie , Jani Nikula , John Harrison , Joonas Lahtinen , Matthew Brost , Michal Wajdeczko , Prathap Kumar Valsan , Rodrigo Vivi , Tvrtko Ursulin , Umesh Nerlige Ramappa , Vinay Belgaumkar , dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, linux-kernel@vger.kernel.org Subject: [PATCH 21/21] drm/i915/guc: document TLB cache invalidation functions Date: Wed, 13 Jul 2022 10:30:18 +0100 Message-Id: <51f5a4d4e209e27eaf82683d56e7747a183fb051.1657703926.git.mchehab@kernel.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add documentation for the kAPI functions that do TLB cache invalidation via GuC. Signed-off-by: Mauro Carvalho Chehab --- See [PATCH 00/21] at: https://lore.kernel.org/all/cover.1657703926.git.mche= hab@kernel.org/ drivers/gpu/drm/i915/gt/uc/intel_guc.c | 52 ++++++++++++++++++++++---- 1 file changed, 45 insertions(+), 7 deletions(-) diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc.c b/drivers/gpu/drm/i915/= gt/uc/intel_guc.c index 98260a7bc90b..173833bc3a62 100644 --- a/drivers/gpu/drm/i915/gt/uc/intel_guc.c +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc.c @@ -923,7 +923,14 @@ static int guc_send_invalidate_tlb(struct intel_guc *g= uc, u32 *action, u32 size) return err; } =20 - /* Full TLB invalidation */ +/** + * intel_guc_invalidate_tlb_full - GuC full TLB invalidation + * + * @guc: the guc + * @mode: mode of TLB cache invalidation (heavy or lite) + * + * Use GuC to do a full TLB cache invalidation if supported. + */ int intel_guc_invalidate_tlb_full(struct intel_guc *guc, enum intel_guc_tlb_inval_mode mode) { @@ -943,8 +950,17 @@ int intel_guc_invalidate_tlb_full(struct intel_guc *gu= c, return guc_send_invalidate_tlb(guc, action, ARRAY_SIZE(action)); } =20 -/* - * Selective TLB Invalidation for Address Range: +/** + * intel_guc_invalidate_tlb_page_selective - GuC selective TLB invalidation + * for an address range + * + * @guc: the guc + * @mode: mode of TLB cache invalidation (heavy or lite) + * @start: range start + * @length: range length + * + * Use GuC to do a selective TLB invalidation if supported. + * * TLB's in the Address Range is Invalidated across all engines. */ int intel_guc_invalidate_tlb_page_selective(struct intel_guc *guc, @@ -978,8 +994,18 @@ int intel_guc_invalidate_tlb_page_selective(struct int= el_guc *guc, return guc_send_invalidate_tlb(guc, action, ARRAY_SIZE(action)); } =20 -/* - * Selective TLB Invalidation for Context: +/** + * intel_guc_invalidate_tlb_page_selective_ctx - GuC selective TLB + * invalidation for a context + * + * @guc: the guc + * @mode: mode of TLB cache invalidation (heavy or lite) + * @start: range start + * @length: range length + * @ctxid: context ID + * + * Use GuC to do a selective TLB invalidation on a context if supported. + * * Invalidates all TLB's for a specific context across all engines. */ int intel_guc_invalidate_tlb_page_selective_ctx(struct intel_guc *guc, @@ -1013,8 +1039,13 @@ int intel_guc_invalidate_tlb_page_selective_ctx(stru= ct intel_guc *guc, return guc_send_invalidate_tlb(guc, action, ARRAY_SIZE(action)); } =20 -/* - * Guc TLB Invalidation: Invalidate the TLB's of GuC itself. +/** + * intel_guc_invalidate_tlb_guc - GuC self TLB invalidation + * + * @guc: the guc + * @mode: mode of TLB cache invalidation (heavy or lite) + * + * Use GuC to invalidate the TLB's of GuC itself. */ int intel_guc_invalidate_tlb_guc(struct intel_guc *guc, enum intel_guc_tlb_inval_mode mode) @@ -1035,6 +1066,13 @@ int intel_guc_invalidate_tlb_guc(struct intel_guc *g= uc, return guc_send_invalidate_tlb(guc, action, ARRAY_SIZE(action)); } =20 +/** + * intel_guc_invalidate_tlb_all - GuC global TLB invalidation + * + * @guc: the guc + * + * Use GuC to do a complete TLB invalidation on all tables + */ int intel_guc_invalidate_tlb_all(struct intel_guc *guc) { u32 action[] =3D { --=20 2.36.1