From nobody Tue Dec 16 07:32:52 2025 Received: from out199-4.us.a.mail.aliyun.com (out199-4.us.a.mail.aliyun.com [47.90.199.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 23A0CF4E2; Wed, 7 May 2025 01:16:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=47.90.199.4 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746580577; cv=none; b=GoEt/44dVKoEVSCSJ0ELjiRF6i9FWVt6Uzse4gDJL8AgvZ7vgQGqlR0tagwc7CIEjh/6x5Jfnvo1vdogWuIoU7PJO7C4XnNJgQ/eu/6VwttW2gt44OjkGXhW5U825gvFXg89B5HLRIktKF/Ijaxm2+yrJMRNjrrT3BSO2lTpEyU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746580577; c=relaxed/simple; bh=4Y/wfHCD/JmJfE1SBEtkQtHX5ZUN0fh0CH1uktm6L4k=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=o5Gu46vtGRB7AUZ0ag2jd5MI8mci85cXrJE5uW59a77zbAWYFkOdnEjCWLh3Xmf2uJuDDOc4tl7+FxXoH48269/Wg0N0BkyXvpB7nR9Upd/JmhGTr3s3I89wXKHMnEOzbCDRtKKSVgVfz+la7ifXp74NmaRwnPt5EZr6ZkiwmKg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=UjPwzL2e; arc=none smtp.client-ip=47.90.199.4 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="UjPwzL2e" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1746580551; h=From:To:Subject:Date:Message-ID:MIME-Version; bh=BhkBg8BHNtFzJ9+ijMhzKHUaq64bGU+JAPv//d4vJvM=; b=UjPwzL2exoui+xntVmaD82fl9MKZRNmHOtzVksh1TLNxVdxY4N84oZHz2hT8Rcez93C/JymMNeKTIPnURCMQu6cqg9V+40KUnFtUR+O+ULUnuexQlxq3ElaLqPb/fVPnRMOtrk9nQRSiF9EOzsZkSfMx20zkrFc7Usfp5opzjII= Received: from localhost.localdomain(mailfrom:xueshuai@linux.alibaba.com fp:SMTPD_---0WZh9L8X_1746580536 cluster:ay36) by smtp.aliyun-inc.com; Wed, 07 May 2025 09:15:49 +0800 From: Shuai Xue To: rostedt@goodmis.org, lukas@wunner.de, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-trace-kernel@vger.kernel.org, helgaas@kernel.org Cc: bhelgaas@google.com, tony.luck@intel.com, bp@alien8.de, xueshuai@linux.alibaba.com, mhiramat@kernel.org, mathieu.desnoyers@efficios.com, oleg@redhat.com, naveen@kernel.org, davem@davemloft.net, anil.s.keshavamurthy@intel.com, mark.rutland@arm.com, peterz@infradead.org, tianruidong@linux.alibaba.com Subject: [PATCH v7] PCI: hotplug: Add a generic RAS tracepoint for hotplug event Date: Wed, 7 May 2025 09:15:35 +0800 Message-ID: <20250507011535.43800-1-xueshuai@linux.alibaba.com> X-Mailer: git-send-email 2.44.0 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Hotplug events are critical indicators for analyzing hardware health, particularly in AI supercomputers where surprise link downs can significantly impact system performance and reliability. To this end, define a new TRACING_SYSTEM named pci, add a generic RAS tracepoint for hotplug event to help healthy check, and generate tracepoints for pcie hotplug event. Add enum pci_hotplug_event in include/uapi/linux/pci.h so applications like rasdaemon can register tracepoint event handlers for it. The output like below: $ echo 1 > /sys/kernel/debug/tracing/events/pci/pci_hp_event/enable $ cat /sys/kernel/debug/tracing/trace_pipe <...>-206 [001] ..... 40.373870: pci_hp_event: 0000:00:02.0 slot= :10, event:Link Down <...>-206 [001] ..... 40.374871: pci_hp_event: 0000:00:02.0 slot= :10, event:Card not present Suggested-by: Lukas Wunner Suggested-by: Steven Rostedt Reviewed-by: Lukas Wunner Signed-off-by: Shuai Xue Reviewed-by: Jonathan Cameron --- changes since v6: - move defination of tracepoint enum back to trace.h. Otherwise, it will cause a build error when CONFIG_UAPI_HEADER_TEST=3Dy. The problem is that the PCI_HOTPLUG_EVENT macro needs to include "linux/tracepoint.h" but it is not allowed in userspace. - No code changes compared to v5. change since v5:=20 - move define of enum to include/uapi/linux/pci.h - link: https://lore.kernel.org/lkml/202501190108.tRReJA1Z-lkp@intel.com/ v5: - link: https://patchwork.kernel.org/project/linux-pci/patch/20250109025543= .56830-1-xueshuai@linux.alibaba.com/ --- drivers/pci/hotplug/pciehp_ctrl.c | 33 ++++++++++++--- drivers/pci/hotplug/trace.h | 68 +++++++++++++++++++++++++++++++ include/uapi/linux/pci.h | 7 ++++ 3 files changed, 102 insertions(+), 6 deletions(-) create mode 100644 drivers/pci/hotplug/trace.h diff --git a/drivers/pci/hotplug/pciehp_ctrl.c b/drivers/pci/hotplug/pciehp= _ctrl.c index d603a7aa7483..f9beb4d3a9b8 100644 --- a/drivers/pci/hotplug/pciehp_ctrl.c +++ b/drivers/pci/hotplug/pciehp_ctrl.c @@ -23,6 +23,9 @@ #include "../pci.h" #include "pciehp.h" =20 +#define CREATE_TRACE_POINTS +#include "trace.h" + /* The following routines constitute the bulk of the hotplug controller logic */ @@ -244,12 +247,20 @@ void pciehp_handle_presence_or_link_change(struct con= troller *ctrl, u32 events) case ON_STATE: ctrl->state =3D POWEROFF_STATE; mutex_unlock(&ctrl->state_lock); - if (events & PCI_EXP_SLTSTA_DLLSC) + if (events & PCI_EXP_SLTSTA_DLLSC) { ctrl_info(ctrl, "Slot(%s): Link Down\n", slot_name(ctrl)); - if (events & PCI_EXP_SLTSTA_PDC) + trace_pci_hp_event(pci_name(ctrl->pcie->port), + slot_name(ctrl), + PCI_HOTPLUG_LINK_DOWN); + } + if (events & PCI_EXP_SLTSTA_PDC) { ctrl_info(ctrl, "Slot(%s): Card not present\n", slot_name(ctrl)); + trace_pci_hp_event(pci_name(ctrl->pcie->port), + slot_name(ctrl), + PCI_HOTPLUG_CARD_NOT_PRESENT); + } pciehp_disable_slot(ctrl, SURPRISE_REMOVAL); break; default: @@ -269,6 +280,9 @@ void pciehp_handle_presence_or_link_change(struct contr= oller *ctrl, u32 events) INDICATOR_NOOP); ctrl_info(ctrl, "Slot(%s): Card not present\n", slot_name(ctrl)); + trace_pci_hp_event(pci_name(ctrl->pcie->port), + slot_name(ctrl), + PCI_HOTPLUG_CARD_NOT_PRESENT); } mutex_unlock(&ctrl->state_lock); return; @@ -281,12 +295,19 @@ void pciehp_handle_presence_or_link_change(struct con= troller *ctrl, u32 events) case OFF_STATE: ctrl->state =3D POWERON_STATE; mutex_unlock(&ctrl->state_lock); - if (present) + if (present) { ctrl_info(ctrl, "Slot(%s): Card present\n", slot_name(ctrl)); - if (link_active) - ctrl_info(ctrl, "Slot(%s): Link Up\n", - slot_name(ctrl)); + trace_pci_hp_event(pci_name(ctrl->pcie->port), + slot_name(ctrl), + PCI_HOTPLUG_CARD_PRESENT); + } + if (link_active) { + ctrl_info(ctrl, "Slot(%s): Link Up\n", slot_name(ctrl)); + trace_pci_hp_event(pci_name(ctrl->pcie->port), + slot_name(ctrl), + PCI_HOTPLUG_LINK_UP); + } ctrl->request_result =3D pciehp_enable_slot(ctrl); break; default: diff --git a/drivers/pci/hotplug/trace.h b/drivers/pci/hotplug/trace.h new file mode 100644 index 000000000000..5b60cd7bcffb --- /dev/null +++ b/drivers/pci/hotplug/trace.h @@ -0,0 +1,68 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#if !defined(_TRACE_HW_EVENT_PCI_HP_H) || defined(TRACE_HEADER_MULTI_READ) +#define _TRACE_HW_EVENT_PCI_HP_H + +#include + +#undef TRACE_SYSTEM +#define TRACE_SYSTEM pci + +#define PCI_HOTPLUG_EVENT \ + EM(PCI_HOTPLUG_LINK_UP, "Link Up") \ + EM(PCI_HOTPLUG_LINK_DOWN, "Link Down") \ + EM(PCI_HOTPLUG_CARD_PRESENT, "Card present") \ + EMe(PCI_HOTPLUG_CARD_NOT_PRESENT, "Card not present") + +/* Enums require being exported to userspace, for user tool parsing */ +#undef EM +#undef EMe +#define EM(a, b) TRACE_DEFINE_ENUM(a); +#define EMe(a, b) TRACE_DEFINE_ENUM(a); + +PCI_HOTPLUG_EVENT + +/* + * Now redefine the EM() and EMe() macros to map the enums to the strings + * that will be printed in the output. + */ +#undef EM +#undef EMe +#define EM(a, b) {a, b}, +#define EMe(a, b) {a, b} + +TRACE_EVENT(pci_hp_event, + + TP_PROTO(const char *port_name, + const char *slot, + const int event), + + TP_ARGS(port_name, slot, event), + + TP_STRUCT__entry( + __string( port_name, port_name ) + __string( slot, slot ) + __field( int, event ) + ), + + TP_fast_assign( + __assign_str(port_name); + __assign_str(slot); + __entry->event =3D event; + ), + + TP_printk("%s slot:%s, event:%s\n", + __get_str(port_name), + __get_str(slot), + __print_symbolic(__entry->event, PCI_HOTPLUG_EVENT) + ) +); + +#endif /* _TRACE_HW_EVENT_PCI_HP_H */ + +#undef TRACE_INCLUDE_PATH +#define TRACE_INCLUDE_PATH ../../drivers/pci/hotplug +#undef TRACE_INCLUDE_FILE +#define TRACE_INCLUDE_FILE trace + +/* This part must be outside protection */ +#include diff --git a/include/uapi/linux/pci.h b/include/uapi/linux/pci.h index a769eefc5139..4f150028965d 100644 --- a/include/uapi/linux/pci.h +++ b/include/uapi/linux/pci.h @@ -39,4 +39,11 @@ #define PCIIOC_MMAP_IS_MEM (PCIIOC_BASE | 0x02) /* Set mmap state to MEM s= pace. */ #define PCIIOC_WRITE_COMBINE (PCIIOC_BASE | 0x03) /* Enable/disable write-= combining. */ =20 +enum pci_hotplug_event { + PCI_HOTPLUG_LINK_UP, + PCI_HOTPLUG_LINK_DOWN, + PCI_HOTPLUG_CARD_PRESENT, + PCI_HOTPLUG_CARD_NOT_PRESENT, +}; + #endif /* _UAPILINUX_PCI_H */ --=20 2.39.3