From nobody Sun Dec 14 02:00:22 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 470AD8479; Mon, 28 Jul 2025 16:02:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753718542; cv=none; b=QimHIAUOPRt4Otsy+0I0AjV+mO4usCCkti35dh2Mrn6yfHQrSI6s5ERopOlBmLf+JodlFAYpwEosC8Vd1SYIVH/ETiFifZWGr8FN02CjAXEEsaOUh76q0DvbUDdGhGTuehEEtQ6hsoTHBGRViV9+b8OsiSZlfoT7XjJXI8n4g2g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753718542; c=relaxed/simple; bh=W4oDpda6kbCRnn0ODm8ONN7mKwlVwM/eB17fbg3mJL0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JWHVtkU7ibjzzAlpm6a67gUtkjgMWdwP04duli+4y038h6IgmI/vAkOLSjONZ/M4HxPi5Mpbgscczluc4u4vuvPO5k/RG/UdgTmTFuPIq50XbxjkLhjR17Nhh637C48AqxGfuIuWn0BfGw5EhIticeV0sArQCaUtBmfHuJBlutc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=YGUD9hAc; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="YGUD9hAc" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D335EC4CEE7; Mon, 28 Jul 2025 16:02:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1753718541; bh=W4oDpda6kbCRnn0ODm8ONN7mKwlVwM/eB17fbg3mJL0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YGUD9hAcYppn4ft28tj0m5ByaI8tqR8IiAzQp8jMTNEipwnyGyfFjnFGAFxk4VhGR UV70rnp+dFLHE1K6J9wxPKnLzLAcI6pauVggFT32JOhc7cF7woqLkHw6C8+iz1YWHl OGmxh4jiRq3z5mbpgTV4qZzeCUxfuYaidLeo+0KpUh+MdsSitxwO1h5ykM2QmAIdrq H96V4xtELsYZk7AekMCUod8rLt2/yERc/TmK7FeZr48kTg97O8EBuB4JlAyxso4rax RJv4hHkVLc5RkwYtT/rjflhYcoD5kjmjOwhTg88LvnludwNI7CCLWNIhZynkgCYlXi S/uAYuejmqCfQ== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1ugQIq-00000000Gch-0ygn; Mon, 28 Jul 2025 18:02:16 +0200 From: Mauro Carvalho Chehab To: "Message-ID :" , Linux Doc Mailing List Cc: Mauro Carvalho Chehab , "Akira Yokosawa" , "Breno Leitao" , "David S. Miller" , "Donald Hunter" , "Eric Dumazet" , "Ignacio Encinas Rubio" , "Jakub Kicinski" , "Jan Stancek" , "Jonathan Corbet" , "Marco Elver" , "Paolo Abeni" , "Randy Dunlap" , "Ruben Wauters" , "Shuah Khan" , "Simon Horman" , joel@joelfernandes.org, linux-kernel-mentees@lists.linux.dev, linux-kernel@vger.kernel.org, lkmm@lists.linux.dev, netdev@vger.kernel.org, peterz@infradead.org, stern@rowland.harvard.edu Subject: [PATCH v10 01/14] docs: netlink: netlink-raw.rst: use :ref: instead of :doc: Date: Mon, 28 Jul 2025 18:01:54 +0200 Message-ID: <1855229cf1ad7a8bdcab43423b5672f2fa55026f.1753718185.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" Currently, rt documents are referred with: Documentation/userspace-api/netlink/netlink-raw.rst: :doc:`rt-link<../../ne= tworking/netlink_spec/rt-link>` Documentation/userspace-api/netlink/netlink-raw.rst: :doc:`tc<../../network= ing/netlink_spec/tc>` Documentation/userspace-api/netlink/netlink-raw.rst: :doc:`tc<../../network= ing/netlink_spec/tc>` Having :doc: references with relative paths doesn't always work, as it may have troubles when O=3D is used. Also that's hard to maintain, and may break if we change the way rst files are generated from yaml. Better to use instead a reference for the netlink family. So, replace them by Sphinx cross-reference tag that are created by ynl_gen_rst.py. Signed-off-by: Mauro Carvalho Chehab Reviewed-by: Donald Hunter --- Documentation/userspace-api/netlink/netlink-raw.rst | 6 +++--- tools/net/ynl/pyynl/ynl_gen_rst.py | 5 +++-- 2 files changed, 6 insertions(+), 5 deletions(-) diff --git a/Documentation/userspace-api/netlink/netlink-raw.rst b/Document= ation/userspace-api/netlink/netlink-raw.rst index 31fc91020eb3..aae296c170c5 100644 --- a/Documentation/userspace-api/netlink/netlink-raw.rst +++ b/Documentation/userspace-api/netlink/netlink-raw.rst @@ -62,8 +62,8 @@ Sub-messages ------------ =20 Several raw netlink families such as -:doc:`rt-link<../../networking/netlink_spec/rt-link>` and -:doc:`tc<../../networking/netlink_spec/tc>` use attribute nesting as an +:ref:`rt-link` and +:ref:`tc` use attribute nesting as an abstraction to carry module specific information. =20 Conceptually it looks as follows:: @@ -162,7 +162,7 @@ then this is an error. Nested struct definitions ------------------------- =20 -Many raw netlink families such as :doc:`tc<../../networking/netlink_spec/t= c>` +Many raw netlink families such as :ref:`tc` make use of nested struct definitions. The ``netlink-raw`` schema makes it possible to embed a struct within a struct definition using the ``struct`` property. For example, the following struct definition embeds the diff --git a/tools/net/ynl/pyynl/ynl_gen_rst.py b/tools/net/ynl/pyynl/ynl_g= en_rst.py index 0cb6348e28d3..7bfb8ceeeefc 100755 --- a/tools/net/ynl/pyynl/ynl_gen_rst.py +++ b/tools/net/ynl/pyynl/ynl_gen_rst.py @@ -314,10 +314,11 @@ def parse_yaml(obj: Dict[str, Any]) -> str: =20 # Main header =20 - lines.append(rst_header()) - family =3D obj['name'] =20 + lines.append(rst_header()) + lines.append(rst_label("netlink-" + family)) + title =3D f"Family ``{family}`` netlink specification" lines.append(rst_title(title)) lines.append(rst_paragraph(".. contents:: :depth: 3\n")) --=20 2.49.0 From nobody Sun Dec 14 02:00:22 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0F33A274B2E; Mon, 28 Jul 2025 16:02:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753718543; cv=none; b=CVqfWphlFHGFsx6GpRbQZ7mqbiTSFNLyAR5pM48qZ+BP10p6E2BC5z4X5j0oNe7+gwvBREEXWriK22G4k4leD0RH4JPSgY5DJrIrb88mw3ufN6BCFYzpjI/2Z8aex2zkPlvYiYY8QXsGAHhA+tP56xUyvP3Kkm8nLntsUr0Gaas= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753718543; c=relaxed/simple; bh=UmCMr6Z9WczdVidVdmU39OC9Ym8O+WrRwYiAZlUaMu4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=RPBzca1wKXYVKLkrRf+1hYMH5pYEP/TDA4gBw/hh6T0/Ov53iGe8kSZ4y5/qiWCUjmMKGkQbJsElDFB4y8WjwjZORDpNaAvsPPRhM56A2go1YOYGJk5B3JJVb11UB9tW5U4SPyTG9e9p3N0LF5kiA3+RjatbbpP9lj9bkizM7GE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=OJ7U+pJX; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="OJ7U+pJX" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E2DC3C4CEF7; Mon, 28 Jul 2025 16:02:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1753718542; bh=UmCMr6Z9WczdVidVdmU39OC9Ym8O+WrRwYiAZlUaMu4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OJ7U+pJXDXlak0Sxs+OT6sy4sefUrUDc2DTUYf3g+Z2jTwJbi2gddPRrZIy6tWfnz 8TSXL8rtmXtQhRsWDkjpRs6f4U5AMYFA2trRlChQ/lPng+Y/gbIks+kMdaMgJDTHBZ ePStnQfi0lIaLrgfeZnHmXWaeZeS93nvFYA+mZxed1ijIHWNL4LOW2HjcOgsN3TcFu BvzKvnZQAxF0U+LM3GQkZ/iMvHpWncvYnhvwX25ZqaXy22a5W9KiN22JIYyByO5qBg W2A7J+lMfg36lh35W7DIbywi3QT+n+tibrSa9LjO0pg/OJ+KRwTbDauJgoGlh6Cgz1 LQbU6bSRnUIzw== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1ugQIq-00000000Gck-167T; Mon, 28 Jul 2025 18:02:16 +0200 From: Mauro Carvalho Chehab To: "Message-ID :" , Linux Doc Mailing List Cc: Mauro Carvalho Chehab , "Akira Yokosawa" , "Breno Leitao" , "David S. Miller" , "Donald Hunter" , "Eric Dumazet" , "Ignacio Encinas Rubio" , "Jakub Kicinski" , "Jan Stancek" , "Jonathan Corbet" , "Marco Elver" , "Paolo Abeni" , "Randy Dunlap" , "Ruben Wauters" , "Shuah Khan" , "Simon Horman" , joel@joelfernandes.org, linux-kernel-mentees@lists.linux.dev, linux-kernel@vger.kernel.org, lkmm@lists.linux.dev, netdev@vger.kernel.org, peterz@infradead.org, stern@rowland.harvard.edu Subject: [PATCH v10 02/14] tools: ynl_gen_rst.py: Split library from command line tool Date: Mon, 28 Jul 2025 18:01:55 +0200 Message-ID: <05a0a91a731d9db7a4f934ddc33f5ddb2b64e635.1753718185.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" As we'll be using the Netlink specs parser inside a Sphinx extension, move the library part from the command line parser. While here, change the code which generates an index file to parse inputs from both .rst and .yaml extensions. With that, the tool can easily be tested with: tools/net/ynl/pyynl/ynl_gen_rst.py -x -o Documentation/netlink/specs/foo.r= st Without needing to first generate a temp directory with the rst files. Signed-off-by: Mauro Carvalho Chehab Reviewed-by: Donald Hunter --- tools/net/ynl/pyynl/lib/__init__.py | 2 + tools/net/ynl/pyynl/lib/doc_generator.py | 382 +++++++++++++++++++++++ tools/net/ynl/pyynl/ynl_gen_rst.py | 375 +--------------------- 3 files changed, 400 insertions(+), 359 deletions(-) create mode 100644 tools/net/ynl/pyynl/lib/doc_generator.py diff --git a/tools/net/ynl/pyynl/lib/__init__.py b/tools/net/ynl/pyynl/lib/= __init__.py index 71518b9842ee..5f266ebe4526 100644 --- a/tools/net/ynl/pyynl/lib/__init__.py +++ b/tools/net/ynl/pyynl/lib/__init__.py @@ -4,6 +4,8 @@ from .nlspec import SpecAttr, SpecAttrSet, SpecEnumEntry, S= pecEnumSet, \ SpecFamily, SpecOperation, SpecSubMessage, SpecSubMessageFormat from .ynl import YnlFamily, Netlink, NlError =20 +from .doc_generator import YnlDocGenerator + __all__ =3D ["SpecAttr", "SpecAttrSet", "SpecEnumEntry", "SpecEnumSet", "SpecFamily", "SpecOperation", "SpecSubMessage", "SpecSubMessag= eFormat", "YnlFamily", "Netlink", "NlError"] diff --git a/tools/net/ynl/pyynl/lib/doc_generator.py b/tools/net/ynl/pyynl= /lib/doc_generator.py new file mode 100644 index 000000000000..80e468086693 --- /dev/null +++ b/tools/net/ynl/pyynl/lib/doc_generator.py @@ -0,0 +1,382 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0 +# -*- coding: utf-8; mode: python -*- + +""" + Class to auto generate the documentation for Netlink specifications. + + :copyright: Copyright (C) 2023 Breno Leitao + :license: GPL Version 2, June 1991 see linux/COPYING for details. + + This class performs extensive parsing to the Linux kernel's netlink YA= ML + spec files, in an effort to avoid needing to heavily mark up the origi= nal + YAML file. + + This code is split in two classes: + 1) RST formatters: Use to convert a string to a RST output + 2) YAML Netlink (YNL) doc generator: Generate docs from YAML data +""" + +from typing import Any, Dict, List +import os.path +import sys +import argparse +import logging +import yaml + + +# =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +# RST Formatters +# =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +class RstFormatters: + SPACE_PER_LEVEL =3D 4 + + @staticmethod + def headroom(level: int) -> str: + """Return space to format""" + return " " * (level * RstFormatters.SPACE_PER_LEVEL) + + + @staticmethod + def bold(text: str) -> str: + """Format bold text""" + return f"**{text}**" + + + @staticmethod + def inline(text: str) -> str: + """Format inline text""" + return f"``{text}``" + + + @staticmethod + def sanitize(text: str) -> str: + """Remove newlines and multiple spaces""" + # This is useful for some fields that are spread across multiple l= ines + return str(text).replace("\n", " ").strip() + + + def rst_fields(self, key: str, value: str, level: int =3D 0) -> str: + """Return a RST formatted field""" + return self.headroom(level) + f":{key}: {value}" + + + def rst_definition(self, key: str, value: Any, level: int =3D 0) -> st= r: + """Format a single rst definition""" + return self.headroom(level) + key + "\n" + self.headroom(level + 1= ) + str(value) + + + def rst_paragraph(self, paragraph: str, level: int =3D 0) -> str: + """Return a formatted paragraph""" + return self.headroom(level) + paragraph + + + def rst_bullet(self, item: str, level: int =3D 0) -> str: + """Return a formatted a bullet""" + return self.headroom(level) + f"- {item}" + + + @staticmethod + def rst_subsection(title: str) -> str: + """Add a sub-section to the document""" + return f"{title}\n" + "-" * len(title) + + + @staticmethod + def rst_subsubsection(title: str) -> str: + """Add a sub-sub-section to the document""" + return f"{title}\n" + "~" * len(title) + + + @staticmethod + def rst_section(namespace: str, prefix: str, title: str) -> str: + """Add a section to the document""" + return f".. _{namespace}-{prefix}-{title}:\n\n{title}\n" + "=3D" *= len(title) + + + @staticmethod + def rst_subtitle(title: str) -> str: + """Add a subtitle to the document""" + return "\n" + "-" * len(title) + f"\n{title}\n" + "-" * len(title)= + "\n\n" + + + @staticmethod + def rst_title(title: str) -> str: + """Add a title to the document""" + return "=3D" * len(title) + f"\n{title}\n" + "=3D" * len(title) + = "\n\n" + + + def rst_list_inline(self, list_: List[str], level: int =3D 0) -> str: + """Format a list using inlines""" + return self.headroom(level) + "[" + ", ".join(self.inline(i) for i= in list_) + "]" + + + @staticmethod + def rst_ref(namespace: str, prefix: str, name: str) -> str: + """Add a hyperlink to the document""" + mappings =3D {'enum': 'definition', + 'fixed-header': 'definition', + 'nested-attributes': 'attribute-set', + 'struct': 'definition'} + if prefix in mappings: + prefix =3D mappings[prefix] + return f":ref:`{namespace}-{prefix}-{name}`" + + + def rst_header(self) -> str: + """The headers for all the auto generated RST files""" + lines =3D [] + + lines.append(self.rst_paragraph(".. SPDX-License-Identifier: GPL-2= .0")) + lines.append(self.rst_paragraph(".. NOTE: This document was auto-g= enerated.\n\n")) + + return "\n".join(lines) + + + @staticmethod + def rst_toctree(maxdepth: int =3D 2) -> str: + """Generate a toctree RST primitive""" + lines =3D [] + + lines.append(".. toctree::") + lines.append(f" :maxdepth: {maxdepth}\n\n") + + return "\n".join(lines) + + + @staticmethod + def rst_label(title: str) -> str: + """Return a formatted label""" + return f".. _{title}:\n\n" + +# =3D=3D=3D=3D=3D=3D=3D +# Parsers +# =3D=3D=3D=3D=3D=3D=3D +class YnlDocGenerator: + + fmt =3D RstFormatters() + + def parse_mcast_group(self, mcast_group: List[Dict[str, Any]]) -> str: + """Parse 'multicast' group list and return a formatted string""" + lines =3D [] + for group in mcast_group: + lines.append(self.fmt.rst_bullet(group["name"])) + + return "\n".join(lines) + + + def parse_do(self, do_dict: Dict[str, Any], level: int =3D 0) -> str: + """Parse 'do' section and return a formatted string""" + lines =3D [] + for key in do_dict.keys(): + lines.append(self.fmt.rst_paragraph(self.fmt.bold(key), level = + 1)) + if key in ['request', 'reply']: + lines.append(self.parse_do_attributes(do_dict[key], level = + 1) + "\n") + else: + lines.append(self.fmt.headroom(level + 2) + do_dict[key] += "\n") + + return "\n".join(lines) + + + def parse_do_attributes(self, attrs: Dict[str, Any], level: int =3D 0)= -> str: + """Parse 'attributes' section""" + if "attributes" not in attrs: + return "" + lines =3D [self.fmt.rst_fields("attributes", self.fmt.rst_list_inl= ine(attrs["attributes"]), level + 1)] + + return "\n".join(lines) + + + def parse_operations(self, operations: List[Dict[str, Any]], namespace= : str) -> str: + """Parse operations block""" + preprocessed =3D ["name", "doc", "title", "do", "dump", "flags"] + linkable =3D ["fixed-header", "attribute-set"] + lines =3D [] + + for operation in operations: + lines.append(self.fmt.rst_section(namespace, 'operation', oper= ation["name"])) + lines.append(self.fmt.rst_paragraph(operation["doc"]) + "\n") + + for key in operation.keys(): + if key in preprocessed: + # Skip the special fields + continue + value =3D operation[key] + if key in linkable: + value =3D self.fmt.rst_ref(namespace, key, value) + lines.append(self.fmt.rst_fields(key, value, 0)) + if 'flags' in operation: + lines.append(self.fmt.rst_fields('flags', self.fmt.rst_lis= t_inline(operation['flags']))) + + if "do" in operation: + lines.append(self.fmt.rst_paragraph(":do:", 0)) + lines.append(self.parse_do(operation["do"], 0)) + if "dump" in operation: + lines.append(self.fmt.rst_paragraph(":dump:", 0)) + lines.append(self.parse_do(operation["dump"], 0)) + + # New line after fields + lines.append("\n") + + return "\n".join(lines) + + + def parse_entries(self, entries: List[Dict[str, Any]], level: int) -> = str: + """Parse a list of entries""" + ignored =3D ["pad"] + lines =3D [] + for entry in entries: + if isinstance(entry, dict): + # entries could be a list or a dictionary + field_name =3D entry.get("name", "") + if field_name in ignored: + continue + type_ =3D entry.get("type") + if type_: + field_name +=3D f" ({self.fmt.inline(type_)})" + lines.append( + self.fmt.rst_fields(field_name, self.fmt.sanitize(entr= y.get("doc", "")), level) + ) + elif isinstance(entry, list): + lines.append(self.fmt.rst_list_inline(entry, level)) + else: + lines.append(self.fmt.rst_bullet(self.fmt.inline(self.fmt.= sanitize(entry)), level)) + + lines.append("\n") + return "\n".join(lines) + + + def parse_definitions(self, defs: Dict[str, Any], namespace: str) -> s= tr: + """Parse definitions section""" + preprocessed =3D ["name", "entries", "members"] + ignored =3D ["render-max"] # This is not printed + lines =3D [] + + for definition in defs: + lines.append(self.fmt.rst_section(namespace, 'definition', def= inition["name"])) + for k in definition.keys(): + if k in preprocessed + ignored: + continue + lines.append(self.fmt.rst_fields(k, self.fmt.sanitize(defi= nition[k]), 0)) + + # Field list needs to finish with a new line + lines.append("\n") + if "entries" in definition: + lines.append(self.fmt.rst_paragraph(":entries:", 0)) + lines.append(self.parse_entries(definition["entries"], 1)) + if "members" in definition: + lines.append(self.fmt.rst_paragraph(":members:", 0)) + lines.append(self.parse_entries(definition["members"], 1)) + + return "\n".join(lines) + + + def parse_attr_sets(self, entries: List[Dict[str, Any]], namespace: st= r) -> str: + """Parse attribute from attribute-set""" + preprocessed =3D ["name", "type"] + linkable =3D ["enum", "nested-attributes", "struct", "sub-message"] + ignored =3D ["checks"] + lines =3D [] + + for entry in entries: + lines.append(self.fmt.rst_section(namespace, 'attribute-set', = entry["name"])) + for attr in entry["attributes"]: + type_ =3D attr.get("type") + attr_line =3D attr["name"] + if type_: + # Add the attribute type in the same line + attr_line +=3D f" ({self.fmt.inline(type_)})" + + lines.append(self.fmt.rst_subsubsection(attr_line)) + + for k in attr.keys(): + if k in preprocessed + ignored: + continue + if k in linkable: + value =3D self.fmt.rst_ref(namespace, k, attr[k]) + else: + value =3D self.fmt.sanitize(attr[k]) + lines.append(self.fmt.rst_fields(k, value, 0)) + lines.append("\n") + + return "\n".join(lines) + + + def parse_sub_messages(self, entries: List[Dict[str, Any]], namespace:= str) -> str: + """Parse sub-message definitions""" + lines =3D [] + + for entry in entries: + lines.append(self.fmt.rst_section(namespace, 'sub-message', en= try["name"])) + for fmt in entry["formats"]: + value =3D fmt["value"] + + lines.append(self.fmt.rst_bullet(self.fmt.bold(value))) + for attr in ['fixed-header', 'attribute-set']: + if attr in fmt: + lines.append(self.fmt.rst_fields(attr, + self.fmt.rst_ref(namespace= , attr, fmt[attr]), + 1)) + lines.append("\n") + + return "\n".join(lines) + + + def parse_yaml(self, obj: Dict[str, Any]) -> str: + """Format the whole YAML into a RST string""" + lines =3D [] + + # Main header + + family =3D obj['name'] + + lines.append(self.fmt.rst_header()) + lines.append(self.fmt.rst_label("netlink-" + family)) + + title =3D f"Family ``{family}`` netlink specification" + lines.append(self.fmt.rst_title(title)) + lines.append(self.fmt.rst_paragraph(".. contents:: :depth: 3\n")) + + if "doc" in obj: + lines.append(self.fmt.rst_subtitle("Summary")) + lines.append(self.fmt.rst_paragraph(obj["doc"], 0)) + + # Operations + if "operations" in obj: + lines.append(self.fmt.rst_subtitle("Operations")) + lines.append(self.parse_operations(obj["operations"]["list"], = family)) + + # Multicast groups + if "mcast-groups" in obj: + lines.append(self.fmt.rst_subtitle("Multicast groups")) + lines.append(self.parse_mcast_group(obj["mcast-groups"]["list"= ])) + + # Definitions + if "definitions" in obj: + lines.append(self.fmt.rst_subtitle("Definitions")) + lines.append(self.parse_definitions(obj["definitions"], family= )) + + # Attributes set + if "attribute-sets" in obj: + lines.append(self.fmt.rst_subtitle("Attribute sets")) + lines.append(self.parse_attr_sets(obj["attribute-sets"], famil= y)) + + # Sub-messages + if "sub-messages" in obj: + lines.append(self.fmt.rst_subtitle("Sub-messages")) + lines.append(self.parse_sub_messages(obj["sub-messages"], fami= ly)) + + return "\n".join(lines) + + + # Main functions + # =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + + + def parse_yaml_file(self, filename: str) -> str: + """Transform the YAML specified by filename into an RST-formatted = string""" + with open(filename, "r", encoding=3D"utf-8") as spec_file: + yaml_data =3D yaml.safe_load(spec_file) + content =3D self.parse_yaml(yaml_data) + + return content diff --git a/tools/net/ynl/pyynl/ynl_gen_rst.py b/tools/net/ynl/pyynl/ynl_g= en_rst.py index 7bfb8ceeeefc..010315fad498 100755 --- a/tools/net/ynl/pyynl/ynl_gen_rst.py +++ b/tools/net/ynl/pyynl/ynl_gen_rst.py @@ -10,354 +10,17 @@ =20 This script performs extensive parsing to the Linux kernel's netlink Y= AML spec files, in an effort to avoid needing to heavily mark up the origi= nal - YAML file. - - This code is split in three big parts: - 1) RST formatters: Use to convert a string to a RST output - 2) Parser helpers: Functions to parse the YAML data structure - 3) Main function and small helpers + YAML file. It uses the library code from scripts/lib. """ =20 -from typing import Any, Dict, List import os.path +import pathlib import sys import argparse import logging -import yaml - - -SPACE_PER_LEVEL =3D 4 - - -# RST Formatters -# =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -def headroom(level: int) -> str: - """Return space to format""" - return " " * (level * SPACE_PER_LEVEL) - - -def bold(text: str) -> str: - """Format bold text""" - return f"**{text}**" - - -def inline(text: str) -> str: - """Format inline text""" - return f"``{text}``" - - -def sanitize(text: str) -> str: - """Remove newlines and multiple spaces""" - # This is useful for some fields that are spread across multiple lines - return str(text).replace("\n", " ").strip() - - -def rst_fields(key: str, value: str, level: int =3D 0) -> str: - """Return a RST formatted field""" - return headroom(level) + f":{key}: {value}" - - -def rst_definition(key: str, value: Any, level: int =3D 0) -> str: - """Format a single rst definition""" - return headroom(level) + key + "\n" + headroom(level + 1) + str(value) - - -def rst_paragraph(paragraph: str, level: int =3D 0) -> str: - """Return a formatted paragraph""" - return headroom(level) + paragraph - - -def rst_bullet(item: str, level: int =3D 0) -> str: - """Return a formatted a bullet""" - return headroom(level) + f"- {item}" - - -def rst_subsection(title: str) -> str: - """Add a sub-section to the document""" - return f"{title}\n" + "-" * len(title) - - -def rst_subsubsection(title: str) -> str: - """Add a sub-sub-section to the document""" - return f"{title}\n" + "~" * len(title) - - -def rst_section(namespace: str, prefix: str, title: str) -> str: - """Add a section to the document""" - return f".. _{namespace}-{prefix}-{title}:\n\n{title}\n" + "=3D" * len= (title) - - -def rst_subtitle(title: str) -> str: - """Add a subtitle to the document""" - return "\n" + "-" * len(title) + f"\n{title}\n" + "-" * len(title) + "= \n\n" - - -def rst_title(title: str) -> str: - """Add a title to the document""" - return "=3D" * len(title) + f"\n{title}\n" + "=3D" * len(title) + "\n\= n" - - -def rst_list_inline(list_: List[str], level: int =3D 0) -> str: - """Format a list using inlines""" - return headroom(level) + "[" + ", ".join(inline(i) for i in list_) + "= ]" - - -def rst_ref(namespace: str, prefix: str, name: str) -> str: - """Add a hyperlink to the document""" - mappings =3D {'enum': 'definition', - 'fixed-header': 'definition', - 'nested-attributes': 'attribute-set', - 'struct': 'definition'} - if prefix in mappings: - prefix =3D mappings[prefix] - return f":ref:`{namespace}-{prefix}-{name}`" - - -def rst_header() -> str: - """The headers for all the auto generated RST files""" - lines =3D [] - - lines.append(rst_paragraph(".. SPDX-License-Identifier: GPL-2.0")) - lines.append(rst_paragraph(".. NOTE: This document was auto-generated.= \n\n")) - - return "\n".join(lines) - - -def rst_toctree(maxdepth: int =3D 2) -> str: - """Generate a toctree RST primitive""" - lines =3D [] - - lines.append(".. toctree::") - lines.append(f" :maxdepth: {maxdepth}\n\n") - - return "\n".join(lines) - - -def rst_label(title: str) -> str: - """Return a formatted label""" - return f".. _{title}:\n\n" - - -# Parsers -# =3D=3D=3D=3D=3D=3D=3D - - -def parse_mcast_group(mcast_group: List[Dict[str, Any]]) -> str: - """Parse 'multicast' group list and return a formatted string""" - lines =3D [] - for group in mcast_group: - lines.append(rst_bullet(group["name"])) - - return "\n".join(lines) - - -def parse_do(do_dict: Dict[str, Any], level: int =3D 0) -> str: - """Parse 'do' section and return a formatted string""" - lines =3D [] - for key in do_dict.keys(): - lines.append(rst_paragraph(bold(key), level + 1)) - if key in ['request', 'reply']: - lines.append(parse_do_attributes(do_dict[key], level + 1) + "\= n") - else: - lines.append(headroom(level + 2) + do_dict[key] + "\n") - - return "\n".join(lines) - - -def parse_do_attributes(attrs: Dict[str, Any], level: int =3D 0) -> str: - """Parse 'attributes' section""" - if "attributes" not in attrs: - return "" - lines =3D [rst_fields("attributes", rst_list_inline(attrs["attributes"= ]), level + 1)] - - return "\n".join(lines) - - -def parse_operations(operations: List[Dict[str, Any]], namespace: str) -> = str: - """Parse operations block""" - preprocessed =3D ["name", "doc", "title", "do", "dump", "flags"] - linkable =3D ["fixed-header", "attribute-set"] - lines =3D [] - - for operation in operations: - lines.append(rst_section(namespace, 'operation', operation["name"]= )) - lines.append(rst_paragraph(operation["doc"]) + "\n") - - for key in operation.keys(): - if key in preprocessed: - # Skip the special fields - continue - value =3D operation[key] - if key in linkable: - value =3D rst_ref(namespace, key, value) - lines.append(rst_fields(key, value, 0)) - if 'flags' in operation: - lines.append(rst_fields('flags', rst_list_inline(operation['fl= ags']))) - - if "do" in operation: - lines.append(rst_paragraph(":do:", 0)) - lines.append(parse_do(operation["do"], 0)) - if "dump" in operation: - lines.append(rst_paragraph(":dump:", 0)) - lines.append(parse_do(operation["dump"], 0)) - - # New line after fields - lines.append("\n") - - return "\n".join(lines) - - -def parse_entries(entries: List[Dict[str, Any]], level: int) -> str: - """Parse a list of entries""" - ignored =3D ["pad"] - lines =3D [] - for entry in entries: - if isinstance(entry, dict): - # entries could be a list or a dictionary - field_name =3D entry.get("name", "") - if field_name in ignored: - continue - type_ =3D entry.get("type") - if type_: - field_name +=3D f" ({inline(type_)})" - lines.append( - rst_fields(field_name, sanitize(entry.get("doc", "")), lev= el) - ) - elif isinstance(entry, list): - lines.append(rst_list_inline(entry, level)) - else: - lines.append(rst_bullet(inline(sanitize(entry)), level)) - - lines.append("\n") - return "\n".join(lines) - - -def parse_definitions(defs: Dict[str, Any], namespace: str) -> str: - """Parse definitions section""" - preprocessed =3D ["name", "entries", "members"] - ignored =3D ["render-max"] # This is not printed - lines =3D [] - - for definition in defs: - lines.append(rst_section(namespace, 'definition', definition["name= "])) - for k in definition.keys(): - if k in preprocessed + ignored: - continue - lines.append(rst_fields(k, sanitize(definition[k]), 0)) - - # Field list needs to finish with a new line - lines.append("\n") - if "entries" in definition: - lines.append(rst_paragraph(":entries:", 0)) - lines.append(parse_entries(definition["entries"], 1)) - if "members" in definition: - lines.append(rst_paragraph(":members:", 0)) - lines.append(parse_entries(definition["members"], 1)) - - return "\n".join(lines) - - -def parse_attr_sets(entries: List[Dict[str, Any]], namespace: str) -> str: - """Parse attribute from attribute-set""" - preprocessed =3D ["name", "type"] - linkable =3D ["enum", "nested-attributes", "struct", "sub-message"] - ignored =3D ["checks"] - lines =3D [] - - for entry in entries: - lines.append(rst_section(namespace, 'attribute-set', entry["name"]= )) - for attr in entry["attributes"]: - type_ =3D attr.get("type") - attr_line =3D attr["name"] - if type_: - # Add the attribute type in the same line - attr_line +=3D f" ({inline(type_)})" - - lines.append(rst_subsubsection(attr_line)) - - for k in attr.keys(): - if k in preprocessed + ignored: - continue - if k in linkable: - value =3D rst_ref(namespace, k, attr[k]) - else: - value =3D sanitize(attr[k]) - lines.append(rst_fields(k, value, 0)) - lines.append("\n") - - return "\n".join(lines) - - -def parse_sub_messages(entries: List[Dict[str, Any]], namespace: str) -> s= tr: - """Parse sub-message definitions""" - lines =3D [] - - for entry in entries: - lines.append(rst_section(namespace, 'sub-message', entry["name"])) - for fmt in entry["formats"]: - value =3D fmt["value"] - - lines.append(rst_bullet(bold(value))) - for attr in ['fixed-header', 'attribute-set']: - if attr in fmt: - lines.append(rst_fields(attr, - rst_ref(namespace, attr, fmt[a= ttr]), - 1)) - lines.append("\n") - - return "\n".join(lines) - - -def parse_yaml(obj: Dict[str, Any]) -> str: - """Format the whole YAML into a RST string""" - lines =3D [] - - # Main header - - family =3D obj['name'] - - lines.append(rst_header()) - lines.append(rst_label("netlink-" + family)) - - title =3D f"Family ``{family}`` netlink specification" - lines.append(rst_title(title)) - lines.append(rst_paragraph(".. contents:: :depth: 3\n")) - - if "doc" in obj: - lines.append(rst_subtitle("Summary")) - lines.append(rst_paragraph(obj["doc"], 0)) - - # Operations - if "operations" in obj: - lines.append(rst_subtitle("Operations")) - lines.append(parse_operations(obj["operations"]["list"], family)) - - # Multicast groups - if "mcast-groups" in obj: - lines.append(rst_subtitle("Multicast groups")) - lines.append(parse_mcast_group(obj["mcast-groups"]["list"])) - - # Definitions - if "definitions" in obj: - lines.append(rst_subtitle("Definitions")) - lines.append(parse_definitions(obj["definitions"], family)) - - # Attributes set - if "attribute-sets" in obj: - lines.append(rst_subtitle("Attribute sets")) - lines.append(parse_attr_sets(obj["attribute-sets"], family)) - - # Sub-messages - if "sub-messages" in obj: - lines.append(rst_subtitle("Sub-messages")) - lines.append(parse_sub_messages(obj["sub-messages"], family)) - - return "\n".join(lines) - - -# Main functions -# =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 +sys.path.append(pathlib.Path(__file__).resolve().parent.as_posix()) +from lib import YnlDocGenerator # pylint: disable=3DC0413 =20 def parse_arguments() -> argparse.Namespace: """Parse arguments from user""" @@ -392,15 +55,6 @@ def parse_arguments() -> argparse.Namespace: return args =20 =20 -def parse_yaml_file(filename: str) -> str: - """Transform the YAML specified by filename into an RST-formatted stri= ng""" - with open(filename, "r", encoding=3D"utf-8") as spec_file: - yaml_data =3D yaml.safe_load(spec_file) - content =3D parse_yaml(yaml_data) - - return content - - def write_to_rstfile(content: str, filename: str) -> None: """Write the generated content into an RST file""" logging.debug("Saving RST file to %s", filename) @@ -409,21 +63,22 @@ def write_to_rstfile(content: str, filename: str) -> N= one: rst_file.write(content) =20 =20 -def generate_main_index_rst(output: str) -> None: +def generate_main_index_rst(parser: YnlDocGenerator, output: str) -> None: """Generate the `networking_spec/index` content and write to the file"= "" lines =3D [] =20 - lines.append(rst_header()) - lines.append(rst_label("specs")) - lines.append(rst_title("Netlink Family Specifications")) - lines.append(rst_toctree(1)) + lines.append(parser.fmt.rst_header()) + lines.append(parser.fmt.rst_label("specs")) + lines.append(parser.fmt.rst_title("Netlink Family Specifications")) + lines.append(parser.fmt.rst_toctree(1)) =20 index_dir =3D os.path.dirname(output) logging.debug("Looking for .rst files in %s", index_dir) for filename in sorted(os.listdir(index_dir)): - if not filename.endswith(".rst") or filename =3D=3D "index.rst": + base, ext =3D os.path.splitext(filename) + if filename =3D=3D "index.rst" or ext not in [".rst", ".yaml"]: continue - lines.append(f" {filename.replace('.rst', '')}\n") + lines.append(f" {base}\n") =20 logging.debug("Writing an index file at %s", output) write_to_rstfile("".join(lines), output) @@ -434,10 +89,12 @@ def main() -> None: =20 args =3D parse_arguments() =20 + parser =3D YnlDocGenerator() + if args.input: logging.debug("Parsing %s", args.input) try: - content =3D parse_yaml_file(os.path.join(args.input)) + content =3D parser.parse_yaml_file(os.path.join(args.input)) except Exception as exception: logging.warning("Failed to parse %s.", args.input) logging.warning(exception) @@ -447,7 +104,7 @@ def main() -> None: =20 if args.index: # Generate the index RST file - generate_main_index_rst(args.output) + generate_main_index_rst(parser, args.output) =20 =20 if __name__ =3D=3D "__main__": --=20 2.49.0 From nobody Sun Dec 14 02:00:22 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 47172202F8E; Mon, 28 Jul 2025 16:02:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753718542; cv=none; b=AJZ9g6LCPXHcp2pNBV3UN3WksTM38+Z/CpZ9dh+Crddt9uKAGctcJcHeJleerGpSQCVZefVQnoixvOdJtW/wmx0Isz4VY+PS5qpNe06FnOzP+HymyCrhiWquiz6dxqibv5sGiNhx6wcN310C6F+v9aJzluWwg4kgpptQbkdHX/Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753718542; c=relaxed/simple; bh=Hv3gLw50JLGG5g7ccl776oKGKqF7lxHBB7tVJrn/eaQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BAyFGcUgRPfCei1NE47fUe4cQHRMsP53xbAqC6zKsEamk7GsRfTB2qyUmq4IecvqzZiFlpM8AarCRQiO3Eck54seaapXg9X2jsI2X/fyFKiba1TY0ya01DGEJAGTFhlWepIjeO6p6U5TpxyFs0duXaskPHwH1j8noAevXHU7SmM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Ty+Sqtq7; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Ty+Sqtq7" Received: by smtp.kernel.org (Postfix) with ESMTPSA id EA1E2C4CEF9; Mon, 28 Jul 2025 16:02:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1753718542; bh=Hv3gLw50JLGG5g7ccl776oKGKqF7lxHBB7tVJrn/eaQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Ty+Sqtq7VrudIdBdNN1rE0m3vCwtK+8o2gy/Qp8kgHRw9fjGY+arSIIO+Ex35OIFx 29rneKQcC2YYO5UlFAL1FmooxtFilvJ0H9LqQe/TykftAFhjecoY782TNzmBkaCmsO Vi0KmwbOChPh6AsTs7xODwzDFe0n6p28XF4bggk/Sxsa+8pO/XKRDBiJYB4z1atK/g IxhkYOJJoF4pHfkhC1HGdzPe8gt+m4pjHUwacd9CsULKEk6DkszvKrud2pqyhxOKmU C9uoSIvpK8yI+RBEXRVFXFelEyxbvowyPPZmB0H7O3x1G7LUXnYlM7wmN+To4H/fVN GVEpaPjsR2HTA== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1ugQIq-00000000Gcn-1Cdi; Mon, 28 Jul 2025 18:02:16 +0200 From: Mauro Carvalho Chehab To: "Message-ID :" , Linux Doc Mailing List Cc: Mauro Carvalho Chehab , "Akira Yokosawa" , "Breno Leitao" , "David S. Miller" , "Donald Hunter" , "Eric Dumazet" , "Ignacio Encinas Rubio" , "Jakub Kicinski" , "Jan Stancek" , "Jonathan Corbet" , "Marco Elver" , "Paolo Abeni" , "Randy Dunlap" , "Ruben Wauters" , "Shuah Khan" , "Simon Horman" , joel@joelfernandes.org, linux-kernel-mentees@lists.linux.dev, linux-kernel@vger.kernel.org, lkmm@lists.linux.dev, netdev@vger.kernel.org, peterz@infradead.org, stern@rowland.harvard.edu Subject: [PATCH v10 03/14] docs: netlink: index.rst: add a netlink index file Date: Mon, 28 Jul 2025 18:01:56 +0200 Message-ID: X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" Instead of generating the index file, use glob to automatically include all data from yaml. Signed-off-by: Mauro Carvalho Chehab Reviewed-by: Donald Hunter --- Documentation/netlink/specs/index.rst | 13 +++++++++++++ 1 file changed, 13 insertions(+) create mode 100644 Documentation/netlink/specs/index.rst diff --git a/Documentation/netlink/specs/index.rst b/Documentation/netlink/= specs/index.rst new file mode 100644 index 000000000000..7f7cf4a096f2 --- /dev/null +++ b/Documentation/netlink/specs/index.rst @@ -0,0 +1,13 @@ +.. SPDX-License-Identifier: GPL-2.0 + +.. _specs: + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D +Netlink Family Specifications +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D + +.. toctree:: + :maxdepth: 1 + :glob: + + * --=20 2.49.0 From nobody Sun Dec 14 02:00:22 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 76A332040AB; Mon, 28 Jul 2025 16:02:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753718542; cv=none; b=R0T1ImpE0T1ldR9o1Oq3OutxNOj5uiVjoj+jwjaZRUJqD5pyWzF44SCF/DEvA0WgzC3Ffy2Q9qyCQaFeNCLF28J+cZGvGlw8ip58WwBvawvXw7eNoAW53wb/ca5ByD8sLh+Vorrp5i/uyQPRJAKIekBWp5xZU4KsIWtFzS3Gwvs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753718542; c=relaxed/simple; bh=iZvN3Z3Z0rcVoAdWJ24GjnupP01y0n1PoT8Sy6J4zjM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hwT1CGdTqKIfRjdfWDo7ADx9ezndX0PScO/tfnIYRs4w+TXmFc/IymQKKK9ZkYlaFBpTYFcZoqPyN0oQa3GFgBOFpZSotW2yRZPc1+8Q40pLqI8ggWQa8SYPI1xUPB7+r3AXeWWOjORMCFYh8j8TAiSfsXMoOl81Oe973AFgrJQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=cF4e92rc; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="cF4e92rc" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D81EBC4CEEF; Mon, 28 Jul 2025 16:02:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1753718542; bh=iZvN3Z3Z0rcVoAdWJ24GjnupP01y0n1PoT8Sy6J4zjM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cF4e92rc2gPrDAmu8eiJ5Fa1pZujWTAvMZp4sPwicbtTz7h9HT59GdV0LKeENBzFC a+0d5kL+pkUT2v46CNCyIG0C7ZZckd0WhwdJLzzHEi8AcqFj2FiXYiYyzV/M8ZohIG imrbFd1Xzur1ypEBrUnDTrfA/hm8KiM3fDXj9HMMakHkZg5mmRKVFprwME3D1Zo1Oz uAEK3hUi7dfcvDhU/SEcEH6QAcD19EGcT/SJqxSV9A52Ujr94RtL6PcrWeiJhyMzBu gWGPum82DfHE7Y7BCIzQQqg8eeHd68C/2fZoLSRGbvj2eBj8MHdbNEQIGgMC1jxi+t x+8gR7H9sIQMQ== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1ugQIq-00000000Gcq-1JdN; Mon, 28 Jul 2025 18:02:16 +0200 From: Mauro Carvalho Chehab To: "Message-ID :" , Linux Doc Mailing List Cc: Mauro Carvalho Chehab , "Akira Yokosawa" , "Breno Leitao" , "David S. Miller" , "Donald Hunter" , "Eric Dumazet" , "Ignacio Encinas Rubio" , "Jakub Kicinski" , "Jan Stancek" , "Jonathan Corbet" , "Marco Elver" , "Paolo Abeni" , "Randy Dunlap" , "Ruben Wauters" , "Shuah Khan" , "Simon Horman" , joel@joelfernandes.org, linux-kernel-mentees@lists.linux.dev, linux-kernel@vger.kernel.org, lkmm@lists.linux.dev, netdev@vger.kernel.org, peterz@infradead.org, stern@rowland.harvard.edu Subject: [PATCH v10 04/14] tools: ynl_gen_rst.py: cleanup coding style Date: Mon, 28 Jul 2025 18:01:57 +0200 Message-ID: <9b0704037d6904d21097d5c4cf2fd7102dea361f.1753718185.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" Cleanup some coding style issues pointed by pylint and flake8. No functional changes. Signed-off-by: Mauro Carvalho Chehab Reviewed-by: Breno Leitao Reviewed-by: Donald Hunter --- tools/net/ynl/pyynl/lib/doc_generator.py | 72 ++++++++---------------- 1 file changed, 25 insertions(+), 47 deletions(-) diff --git a/tools/net/ynl/pyynl/lib/doc_generator.py b/tools/net/ynl/pyynl= /lib/doc_generator.py index 80e468086693..474b2b78c7bc 100644 --- a/tools/net/ynl/pyynl/lib/doc_generator.py +++ b/tools/net/ynl/pyynl/lib/doc_generator.py @@ -18,17 +18,12 @@ """ =20 from typing import Any, Dict, List -import os.path -import sys -import argparse -import logging import yaml =20 =20 -# =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -# RST Formatters -# =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D class RstFormatters: + """RST Formatters""" + SPACE_PER_LEVEL =3D 4 =20 @staticmethod @@ -36,81 +31,67 @@ class RstFormatters: """Return space to format""" return " " * (level * RstFormatters.SPACE_PER_LEVEL) =20 - @staticmethod def bold(text: str) -> str: """Format bold text""" return f"**{text}**" =20 - @staticmethod def inline(text: str) -> str: """Format inline text""" return f"``{text}``" =20 - @staticmethod def sanitize(text: str) -> str: """Remove newlines and multiple spaces""" # This is useful for some fields that are spread across multiple l= ines return str(text).replace("\n", " ").strip() =20 - def rst_fields(self, key: str, value: str, level: int =3D 0) -> str: """Return a RST formatted field""" return self.headroom(level) + f":{key}: {value}" =20 - def rst_definition(self, key: str, value: Any, level: int =3D 0) -> st= r: """Format a single rst definition""" return self.headroom(level) + key + "\n" + self.headroom(level + 1= ) + str(value) =20 - def rst_paragraph(self, paragraph: str, level: int =3D 0) -> str: """Return a formatted paragraph""" return self.headroom(level) + paragraph =20 - def rst_bullet(self, item: str, level: int =3D 0) -> str: """Return a formatted a bullet""" return self.headroom(level) + f"- {item}" =20 - @staticmethod def rst_subsection(title: str) -> str: """Add a sub-section to the document""" return f"{title}\n" + "-" * len(title) =20 - @staticmethod def rst_subsubsection(title: str) -> str: """Add a sub-sub-section to the document""" return f"{title}\n" + "~" * len(title) =20 - @staticmethod def rst_section(namespace: str, prefix: str, title: str) -> str: """Add a section to the document""" return f".. _{namespace}-{prefix}-{title}:\n\n{title}\n" + "=3D" *= len(title) =20 - @staticmethod def rst_subtitle(title: str) -> str: """Add a subtitle to the document""" return "\n" + "-" * len(title) + f"\n{title}\n" + "-" * len(title)= + "\n\n" =20 - @staticmethod def rst_title(title: str) -> str: """Add a title to the document""" return "=3D" * len(title) + f"\n{title}\n" + "=3D" * len(title) + = "\n\n" =20 - def rst_list_inline(self, list_: List[str], level: int =3D 0) -> str: """Format a list using inlines""" return self.headroom(level) + "[" + ", ".join(self.inline(i) for i= in list_) + "]" =20 - @staticmethod def rst_ref(namespace: str, prefix: str, name: str) -> str: """Add a hyperlink to the document""" @@ -122,7 +103,6 @@ class RstFormatters: prefix =3D mappings[prefix] return f":ref:`{namespace}-{prefix}-{name}`" =20 - def rst_header(self) -> str: """The headers for all the auto generated RST files""" lines =3D [] @@ -132,7 +112,6 @@ class RstFormatters: =20 return "\n".join(lines) =20 - @staticmethod def rst_toctree(maxdepth: int =3D 2) -> str: """Generate a toctree RST primitive""" @@ -143,16 +122,13 @@ class RstFormatters: =20 return "\n".join(lines) =20 - @staticmethod def rst_label(title: str) -> str: """Return a formatted label""" return f".. _{title}:\n\n" =20 -# =3D=3D=3D=3D=3D=3D=3D -# Parsers -# =3D=3D=3D=3D=3D=3D=3D class YnlDocGenerator: + """YAML Netlink specs Parser""" =20 fmt =3D RstFormatters() =20 @@ -164,7 +140,6 @@ class YnlDocGenerator: =20 return "\n".join(lines) =20 - def parse_do(self, do_dict: Dict[str, Any], level: int =3D 0) -> str: """Parse 'do' section and return a formatted string""" lines =3D [] @@ -177,16 +152,16 @@ class YnlDocGenerator: =20 return "\n".join(lines) =20 - def parse_do_attributes(self, attrs: Dict[str, Any], level: int =3D 0)= -> str: """Parse 'attributes' section""" if "attributes" not in attrs: return "" - lines =3D [self.fmt.rst_fields("attributes", self.fmt.rst_list_inl= ine(attrs["attributes"]), level + 1)] + lines =3D [self.fmt.rst_fields("attributes", + self.fmt.rst_list_inline(attrs["attri= butes"]), + level + 1)] =20 return "\n".join(lines) =20 - def parse_operations(self, operations: List[Dict[str, Any]], namespace= : str) -> str: """Parse operations block""" preprocessed =3D ["name", "doc", "title", "do", "dump", "flags"] @@ -194,7 +169,8 @@ class YnlDocGenerator: lines =3D [] =20 for operation in operations: - lines.append(self.fmt.rst_section(namespace, 'operation', oper= ation["name"])) + lines.append(self.fmt.rst_section(namespace, 'operation', + operation["name"])) lines.append(self.fmt.rst_paragraph(operation["doc"]) + "\n") =20 for key in operation.keys(): @@ -206,7 +182,8 @@ class YnlDocGenerator: value =3D self.fmt.rst_ref(namespace, key, value) lines.append(self.fmt.rst_fields(key, value, 0)) if 'flags' in operation: - lines.append(self.fmt.rst_fields('flags', self.fmt.rst_lis= t_inline(operation['flags']))) + lines.append(self.fmt.rst_fields('flags', + self.fmt.rst_list_inline(= operation['flags']))) =20 if "do" in operation: lines.append(self.fmt.rst_paragraph(":do:", 0)) @@ -220,7 +197,6 @@ class YnlDocGenerator: =20 return "\n".join(lines) =20 - def parse_entries(self, entries: List[Dict[str, Any]], level: int) -> = str: """Parse a list of entries""" ignored =3D ["pad"] @@ -235,17 +211,19 @@ class YnlDocGenerator: if type_: field_name +=3D f" ({self.fmt.inline(type_)})" lines.append( - self.fmt.rst_fields(field_name, self.fmt.sanitize(entr= y.get("doc", "")), level) + self.fmt.rst_fields(field_name, + self.fmt.sanitize(entry.get("doc",= "")), + level) ) elif isinstance(entry, list): lines.append(self.fmt.rst_list_inline(entry, level)) else: - lines.append(self.fmt.rst_bullet(self.fmt.inline(self.fmt.= sanitize(entry)), level)) + lines.append(self.fmt.rst_bullet(self.fmt.inline(self.fmt.= sanitize(entry)), + level)) =20 lines.append("\n") return "\n".join(lines) =20 - def parse_definitions(self, defs: Dict[str, Any], namespace: str) -> s= tr: """Parse definitions section""" preprocessed =3D ["name", "entries", "members"] @@ -270,7 +248,6 @@ class YnlDocGenerator: =20 return "\n".join(lines) =20 - def parse_attr_sets(self, entries: List[Dict[str, Any]], namespace: st= r) -> str: """Parse attribute from attribute-set""" preprocessed =3D ["name", "type"] @@ -279,7 +256,8 @@ class YnlDocGenerator: lines =3D [] =20 for entry in entries: - lines.append(self.fmt.rst_section(namespace, 'attribute-set', = entry["name"])) + lines.append(self.fmt.rst_section(namespace, 'attribute-set', + entry["name"])) for attr in entry["attributes"]: type_ =3D attr.get("type") attr_line =3D attr["name"] @@ -301,13 +279,13 @@ class YnlDocGenerator: =20 return "\n".join(lines) =20 - def parse_sub_messages(self, entries: List[Dict[str, Any]], namespace:= str) -> str: """Parse sub-message definitions""" lines =3D [] =20 for entry in entries: - lines.append(self.fmt.rst_section(namespace, 'sub-message', en= try["name"])) + lines.append(self.fmt.rst_section(namespace, 'sub-message', + entry["name"])) for fmt in entry["formats"]: value =3D fmt["value"] =20 @@ -315,13 +293,14 @@ class YnlDocGenerator: for attr in ['fixed-header', 'attribute-set']: if attr in fmt: lines.append(self.fmt.rst_fields(attr, - self.fmt.rst_ref(namespace= , attr, fmt[attr]), - 1)) + self.fmt.rst_ref(= namespace, + = attr, + = fmt[attr]), + 1)) lines.append("\n") =20 return "\n".join(lines) =20 - def parse_yaml(self, obj: Dict[str, Any]) -> str: """Format the whole YAML into a RST string""" lines =3D [] @@ -344,7 +323,8 @@ class YnlDocGenerator: # Operations if "operations" in obj: lines.append(self.fmt.rst_subtitle("Operations")) - lines.append(self.parse_operations(obj["operations"]["list"], = family)) + lines.append(self.parse_operations(obj["operations"]["list"], + family)) =20 # Multicast groups if "mcast-groups" in obj: @@ -368,11 +348,9 @@ class YnlDocGenerator: =20 return "\n".join(lines) =20 - # Main functions # =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 - def parse_yaml_file(self, filename: str) -> str: """Transform the YAML specified by filename into an RST-formatted = string""" with open(filename, "r", encoding=3D"utf-8") as spec_file: --=20 2.49.0 From nobody Sun Dec 14 02:00:22 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 96ADC273D7F; Mon, 28 Jul 2025 16:02:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753718542; cv=none; b=nbAXp9MqVZ+pd5Cd3QuFu/FP5flgiO/BmUHEBfgko77DtJYKccEclFoUMr8/j57hGb1pA0WVsLe7b2nszHX5CKz4sDNdv0nSsVP4qMXonad3p3P+bgeoPRGTnRztnsrP46DTJHW2pQXNKeO5VsK79hdW7Iv8wlC4UcIyALX3fSg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753718542; c=relaxed/simple; bh=XSw4mQEDkXYg1/MgVR7Ux51WqbxkUyoFtZ5Xkxz6xTs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=FOzy9RdYSV1GXIwE1E6R8s+fqregYOaGOhKmpkR7W8/qm4BDkTnozFbINab6b6cyRs//UZjtYVcPrzPikJYhS9QhWg29vqevcYqGU0oT76BofkHT61V6XMtZ1I2MJlJ0REX3CYMhZVVxxgfb0Sivf2NApseWq3ZmPu2/CskTyq4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Drmb86SH; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Drmb86SH" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1EA0EC116B1; Mon, 28 Jul 2025 16:02:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1753718542; bh=XSw4mQEDkXYg1/MgVR7Ux51WqbxkUyoFtZ5Xkxz6xTs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Drmb86SHisLz6/XaBYSkRUMw4q+Y4sagL2pNyxT88nO4h4j7+QgzO6S98YhmK9loE Cufy8D077P0hrukVf9IVmcaZN9HSt4yYCEjCLfOOGAmWUzYZ+6h6/g3gZycHmeF4wN jBltRkOU57pCbwecqTklQt/7Kp+zU83iwsr8n1Q7mWF65J1ejMqEfGZLw7CmDz8H0M 5b65tqSaqgW97oEfyFcyq79VO1Wg3+w1pKJcZ/CNwmze79IdOjeCQ7GR+7RwOfRXc+ Fbqo2xg+hLyeRAjVlfTFhN3DBUwDPSr9N4hlfrFOc0L9Sm1mrR4QbvzEd28RQumLr+ NYuvLEmGzCr3Q== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1ugQIq-00000000Gct-1QFd; Mon, 28 Jul 2025 18:02:16 +0200 From: Mauro Carvalho Chehab To: "Message-ID :" , Linux Doc Mailing List Cc: Mauro Carvalho Chehab , "Akira Yokosawa" , "Breno Leitao" , "David S. Miller" , "Donald Hunter" , "Eric Dumazet" , "Ignacio Encinas Rubio" , "Jakub Kicinski" , "Jan Stancek" , "Jonathan Corbet" , "Marco Elver" , "Paolo Abeni" , "Randy Dunlap" , "Ruben Wauters" , "Shuah Khan" , "Simon Horman" , joel@joelfernandes.org, linux-kernel-mentees@lists.linux.dev, linux-kernel@vger.kernel.org, lkmm@lists.linux.dev, netdev@vger.kernel.org, peterz@infradead.org, stern@rowland.harvard.edu Subject: [PATCH v10 05/14] docs: sphinx: add a parser for yaml files for Netlink specs Date: Mon, 28 Jul 2025 18:01:58 +0200 Message-ID: X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" Add a simple sphinx.Parser to handle yaml files and add the the code to handle Netlink specs. All other yaml files are ignored. The code was written in a way that parsing yaml for different subsystems and even for different parts of Netlink are easy. All it takes to have a different parser is to add an import line similar to: from doc_generator import YnlDocGenerator adding the corresponding parser somewhere at the extension: netlink_parser =3D YnlDocGenerator() And then add a logic inside parse() to handle different doc outputs, depending on the file location, similar to: if "/netlink/specs/" in fname: msg =3D self.netlink_parser.parse_yaml_file(fname) Signed-off-by: Mauro Carvalho Chehab Reviewed-by: Donald Hunter --- Documentation/sphinx/parser_yaml.py | 104 ++++++++++++++++++++++++++++ 1 file changed, 104 insertions(+) create mode 100755 Documentation/sphinx/parser_yaml.py diff --git a/Documentation/sphinx/parser_yaml.py b/Documentation/sphinx/par= ser_yaml.py new file mode 100755 index 000000000000..fa2e6da17617 --- /dev/null +++ b/Documentation/sphinx/parser_yaml.py @@ -0,0 +1,104 @@ +# SPDX-License-Identifier: GPL-2.0 +# Copyright 2025 Mauro Carvalho Chehab + +""" +Sphinx extension for processing YAML files +""" + +import os +import re +import sys + +from pprint import pformat + +from docutils.parsers.rst import Parser as RSTParser +from docutils.statemachine import ViewList + +from sphinx.util import logging +from sphinx.parsers import Parser + +srctree =3D os.path.abspath(os.environ["srctree"]) +sys.path.insert(0, os.path.join(srctree, "tools/net/ynl/pyynl/lib")) + +from doc_generator import YnlDocGenerator # pylint: disable=3DC0413 + +logger =3D logging.getLogger(__name__) + +class YamlParser(Parser): + """ + Kernel parser for YAML files. + + This is a simple sphinx.Parser to handle yaml files inside the + Kernel tree that will be part of the built documentation. + + The actual parser function is not contained here: the code was + written in a way that parsing yaml for different subsystems + can be done from a single dispatcher. + + All it takes to have parse YAML patches is to have an import line: + + from some_parser_code import NewYamlGenerator + + To this module. Then add an instance of the parser with: + + new_parser =3D NewYamlGenerator() + + and add a logic inside parse() to handle it based on the path, + like this: + + if "/foo" in fname: + msg =3D self.new_parser.parse_yaml_file(fname) + """ + + supported =3D ('yaml', ) + + netlink_parser =3D YnlDocGenerator() + + def rst_parse(self, inputstring, document, msg): + """ + Receives a ReST content that was previously converted by the + YAML parser, adding it to the document tree. + """ + + self.setup_parse(inputstring, document) + + result =3D ViewList() + + try: + # Parse message with RSTParser + for i, line in enumerate(msg.split('\n')): + result.append(line, document.current_source, i) + + rst_parser =3D RSTParser() + rst_parser.parse('\n'.join(result), document) + + except Exception as e: + document.reporter.error("YAML parsing error: %s" % pformat(e)) + + self.finish_parse() + + # Overrides docutils.parsers.Parser. See sphinx.parsers.RSTParser + def parse(self, inputstring, document): + """Check if a YAML is meant to be parsed.""" + + fname =3D document.current_source + + # Handle netlink yaml specs + if "/netlink/specs/" in fname: + msg =3D self.netlink_parser.parse_yaml_file(fname) + self.rst_parse(inputstring, document, msg) + + # All other yaml files are ignored + +def setup(app): + """Setup function for the Sphinx extension.""" + + # Add YAML parser + app.add_source_parser(YamlParser) + app.add_source_suffix('.yaml', 'yaml') + + return { + 'version': '1.0', + 'parallel_read_safe': True, + 'parallel_write_safe': True, + } --=20 2.49.0 From nobody Sun Dec 14 02:00:22 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8DC0B273D7C; Mon, 28 Jul 2025 16:02:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753718542; cv=none; b=iiZDjhnjGhmmc39ajqToY3YL5iyRTp/KWPvIKQESn41aAg4Qy/+K0fxyVNWNUuTdiy6KOjLVxFFkdXg1xMqShJ8C0pCbWGyeMvktgRfFERj56fXP9OZ4Mr/GbAX6IsVteYLe7W8sJNporNDeXTJJ2cZCIKODcOkxPh0ntH5qgoc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753718542; c=relaxed/simple; bh=CA8rENJCqrPXF4f4frEOOWVuEMYfDrH4SLtxePG/My8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Z3yjBx3bcxuxSyikNIIwrJkuty9kb1Dwe33PIwE2atHgnV8/dVGmyUuUjU1NRKL/+Yy6W3deSaG9v/nYAblMPxvGMefAjE4aNMU8/mYVATmGRvs5cWyiymMZmq7rL1CFeEdrPzzsAYKYqWZlXIP4wYrebdnD8twmNKR2cgUz4c4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=QVtETEYT; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="QVtETEYT" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 21148C116D0; Mon, 28 Jul 2025 16:02:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1753718542; bh=CA8rENJCqrPXF4f4frEOOWVuEMYfDrH4SLtxePG/My8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QVtETEYTFqLYBIljuChujb1J58Zw8yQbjVgsgEWAVOy4b20CDT7F+a3/Ffx5jOWqM IYBZl9NfTRe+sKSq1X6dSTB5D5xqXCM9o0saX26xhtv4gL1vzL7kn+viD8BAHW16y1 1nw0SJ+tjRZ+K1aaRnGDBYYk6z2eSI3Tw+k3c8NR4COqKVsfT9QOV1zYETRqSpCP94 7q7bBsD7KLbie6pb2yFXeMSzSRVbAc3haeaIWnWbAA1T0A/Dtt+Spita90+QGxG9Kj aBnGIWf9J7AJuDsxEaZvbegeK0kbdJ2wF4AYcjQb4kOB0EMtIJ+RdjdgaQ5V5Ne6l9 o1+eqR8zC25gA== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1ugQIq-00000000Gcw-1Y3S; Mon, 28 Jul 2025 18:02:16 +0200 From: Mauro Carvalho Chehab To: "Message-ID :" , Linux Doc Mailing List Cc: Mauro Carvalho Chehab , "Akira Yokosawa" , "Breno Leitao" , "David S. Miller" , "Donald Hunter" , "Eric Dumazet" , "Ignacio Encinas Rubio" , "Jakub Kicinski" , "Jan Stancek" , "Jonathan Corbet" , "Marco Elver" , "Paolo Abeni" , "Randy Dunlap" , "Ruben Wauters" , "Shuah Khan" , "Simon Horman" , joel@joelfernandes.org, linux-kernel-mentees@lists.linux.dev, linux-kernel@vger.kernel.org, lkmm@lists.linux.dev, netdev@vger.kernel.org, peterz@infradead.org, stern@rowland.harvard.edu Subject: [PATCH v10 06/14] docs: use parser_yaml extension to handle Netlink specs Date: Mon, 28 Jul 2025 18:01:59 +0200 Message-ID: <4c97889f0674b69f584dedc543a879d227ef5de0.1753718185.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" Instead of manually calling ynl_gen_rst.py, use a Sphinx extension. This way, no .rst files would be written to the Kernel source directories. We are using here a toctree with :glob: property. This way, there is no need to touch the netlink/specs/index.rst file every time a new Netlink spec is added/renamed/removed. Signed-off-by: Mauro Carvalho Chehab Reviewed-by: Donald Hunter --- Documentation/Makefile | 17 ---------------- Documentation/conf.py | 20 ++++++++++++++----- Documentation/networking/index.rst | 2 +- .../networking/netlink_spec/readme.txt | 4 ---- 4 files changed, 16 insertions(+), 27 deletions(-) delete mode 100644 Documentation/networking/netlink_spec/readme.txt diff --git a/Documentation/Makefile b/Documentation/Makefile index b98477df5ddf..820f07e0afe6 100644 --- a/Documentation/Makefile +++ b/Documentation/Makefile @@ -104,22 +104,6 @@ quiet_cmd_sphinx =3D SPHINX $@ --> file://$(abspath $= (BUILDDIR)/$3/$4) cp $(if $(patsubst /%,,$(DOCS_CSS)),$(abspath $(srctree)/$(DOCS_CSS)),$(= DOCS_CSS)) $(BUILDDIR)/$3/_static/; \ fi =20 -YNL_INDEX:=3D$(srctree)/Documentation/networking/netlink_spec/index.rst -YNL_RST_DIR:=3D$(srctree)/Documentation/networking/netlink_spec -YNL_YAML_DIR:=3D$(srctree)/Documentation/netlink/specs -YNL_TOOL:=3D$(srctree)/tools/net/ynl/pyynl/ynl_gen_rst.py - -YNL_RST_FILES_TMP :=3D $(patsubst %.yaml,%.rst,$(wildcard $(YNL_YAML_DIR)/= *.yaml)) -YNL_RST_FILES :=3D $(patsubst $(YNL_YAML_DIR)%,$(YNL_RST_DIR)%, $(YNL_RST_= FILES_TMP)) - -$(YNL_INDEX): $(YNL_RST_FILES) - $(Q)$(YNL_TOOL) -o $@ -x - -$(YNL_RST_DIR)/%.rst: $(YNL_YAML_DIR)/%.yaml $(YNL_TOOL) - $(Q)$(YNL_TOOL) -i $< -o $@ - -htmldocs texinfodocs latexdocs epubdocs xmldocs: $(YNL_INDEX) - htmldocs: @$(srctree)/scripts/sphinx-pre-install --version-check @+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,html,$(var),,$(var))) @@ -186,7 +170,6 @@ refcheckdocs: $(Q)cd $(srctree);scripts/documentation-file-ref-check =20 cleandocs: - $(Q)rm -f $(YNL_INDEX) $(YNL_RST_FILES) $(Q)rm -rf $(BUILDDIR) $(Q)$(MAKE) BUILDDIR=3D$(abspath $(BUILDDIR)) $(build)=3DDocumentation/us= erspace-api/media clean =20 diff --git a/Documentation/conf.py b/Documentation/conf.py index 700516238d3f..f9828f3862f9 100644 --- a/Documentation/conf.py +++ b/Documentation/conf.py @@ -42,6 +42,15 @@ exclude_patterns =3D [] dyn_include_patterns =3D [] dyn_exclude_patterns =3D ["output"] =20 +# Currently, only netlink/specs has a parser for yaml. +# Prefer using include patterns if available, as it is faster +if has_include_patterns: + dyn_include_patterns.append("netlink/specs/*.yaml") +else: + dyn_exclude_patterns.append("netlink/*.yaml") + dyn_exclude_patterns.append("devicetree/bindings/**.yaml") + dyn_exclude_patterns.append("core-api/kho/bindings/**.yaml") + # Properly handle include/exclude patterns # ---------------------------------------- =20 @@ -102,12 +111,12 @@ extensions =3D [ "kernel_include", "kfigure", "maintainers_include", + "parser_yaml", "rstFlatTable", "sphinx.ext.autosectionlabel", "sphinx.ext.ifconfig", "translations", ] - # Since Sphinx version 3, the C function parser is more pedantic with rega= rds # to type checking. Due to that, having macros at c:function cause problem= s. # Those needed to be escaped by using c_id_attributes[] array @@ -204,10 +213,11 @@ else: # Add any paths that contain templates here, relative to this directory. templates_path =3D ["sphinx/templates"] =20 -# The suffix(es) of source filenames. -# You can specify multiple suffix as a list of string: -# source_suffix =3D ['.rst', '.md'] -source_suffix =3D '.rst' +# The suffixes of source filenames that will be automatically parsed +source_suffix =3D { + ".rst": "restructuredtext", + ".yaml": "yaml", +} =20 # The encoding of source files. # source_encoding =3D 'utf-8-sig' diff --git a/Documentation/networking/index.rst b/Documentation/networking/= index.rst index ac90b82f3ce9..b7a4969e9bc9 100644 --- a/Documentation/networking/index.rst +++ b/Documentation/networking/index.rst @@ -57,7 +57,7 @@ Contents: filter generic-hdlc generic_netlink - netlink_spec/index + ../netlink/specs/index gen_stats gtp ila diff --git a/Documentation/networking/netlink_spec/readme.txt b/Documentati= on/networking/netlink_spec/readme.txt deleted file mode 100644 index 030b44aca4e6..000000000000 --- a/Documentation/networking/netlink_spec/readme.txt +++ /dev/null @@ -1,4 +0,0 @@ -SPDX-License-Identifier: GPL-2.0 - -This file is populated during the build of the documentation (htmldocs) by= the -tools/net/ynl/pyynl/ynl_gen_rst.py script. --=20 2.49.0 From nobody Sun Dec 14 02:00:22 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DD0B227466F; Mon, 28 Jul 2025 16:02:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753718543; cv=none; b=qjIVlX0XQnL8D6y8yzgUjg9ciTQFvtE372viMxmDafn6peiuA0n3fDDAgJaqYfTx1iv3TuHQEctEFdR8DjYdyaqyGFqEGVgZaeeAxKDkcgfPcE61nXJiD4b33tkI0WChmBJCGCzHXit6ercFPkrpSwydAEG+vpyvGa4gpApxqGA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753718543; c=relaxed/simple; bh=gCUujhjF05O4bN0j1mBzTOE2gqunIOFmC86Oe+w0BHo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WCgZs1LcmDE6pYGAcwEVaURJBdzZ+EDynrHY0XytTvVpnOAE9IIeagxAa+RbY2R6sgzLk9Zhx6y4uLp5NN550Rl6+Xuswp2YjG5sk2sG1pWc/imtSMJOXxF5qextMT1ZN1JuWj0ENP+jl6PhypWqmRLKoIN0Ff1rIPr/Enp3N0w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=MhyUot7d; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="MhyUot7d" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 288D8C19421; Mon, 28 Jul 2025 16:02:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1753718542; bh=gCUujhjF05O4bN0j1mBzTOE2gqunIOFmC86Oe+w0BHo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MhyUot7diHCj3qYbkq7B/N+cAWFftX02PcIh1z4wUmKdBmpPjG84d2W8JWsXEovXT /EddXGkmVpD+4JSBkblik2RjwSL4k1RUq11DLRxFKVKqgslhZpKjOPb0JmiSr2o0DL WXvZ+ycdfqbtw/YUnJInsEH41vfURuM7r8VtEpB647nPUI+Ut0hTVPrEtsMC8wBeU7 xLGG4DZJ5SHyiYgBSfbfUvlyG7PQK5vM8yegV+BZ9SQxBhrqWuozamdOmjjro0Wn6Q 7K7tw73Z9fjY1x2GtqbEJ8cmSI5ZBV6U30xIscIOWtj6shFR6z61/nbzh1a5QYsTnc +qLBeA5CISELQ== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1ugQIq-00000000Gcz-1eY4; Mon, 28 Jul 2025 18:02:16 +0200 From: Mauro Carvalho Chehab To: "Message-ID :" , Linux Doc Mailing List Cc: Mauro Carvalho Chehab , "Akira Yokosawa" , "Breno Leitao" , "David S. Miller" , "Donald Hunter" , "Eric Dumazet" , "Ignacio Encinas Rubio" , "Jakub Kicinski" , "Jan Stancek" , "Jonathan Corbet" , "Marco Elver" , "Paolo Abeni" , "Randy Dunlap" , "Ruben Wauters" , "Shuah Khan" , "Simon Horman" , joel@joelfernandes.org, linux-kernel-mentees@lists.linux.dev, linux-kernel@vger.kernel.org, lkmm@lists.linux.dev, netdev@vger.kernel.org, peterz@infradead.org, stern@rowland.harvard.edu Subject: [PATCH v10 07/14] docs: uapi: netlink: update netlink specs link Date: Mon, 28 Jul 2025 18:02:00 +0200 Message-ID: X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" With the recent parser_yaml extension, and the removal of the auto-generated ReST source files, the location of netlink specs changed. Update uAPI accordingly. Signed-off-by: Mauro Carvalho Chehab Reviewed-by: Donald Hunter --- Documentation/userspace-api/netlink/index.rst | 2 +- Documentation/userspace-api/netlink/specs.rst | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Documentation/userspace-api/netlink/index.rst b/Documentation/= userspace-api/netlink/index.rst index c1b6765cc963..83ae25066591 100644 --- a/Documentation/userspace-api/netlink/index.rst +++ b/Documentation/userspace-api/netlink/index.rst @@ -18,4 +18,4 @@ Netlink documentation for users. =20 See also: - :ref:`Documentation/core-api/netlink.rst ` - - :ref:`Documentation/networking/netlink_spec/index.rst ` + - :ref:`Documentation/netlink/specs/index.rst ` diff --git a/Documentation/userspace-api/netlink/specs.rst b/Documentation/= userspace-api/netlink/specs.rst index 1b50d97d8d7c..debb4bfca5c4 100644 --- a/Documentation/userspace-api/netlink/specs.rst +++ b/Documentation/userspace-api/netlink/specs.rst @@ -15,7 +15,7 @@ kernel headers directly. Internally kernel uses the YAML specs to generate: =20 - the C uAPI header - - documentation of the protocol as a ReST file - see :ref:`Documentation/= networking/netlink_spec/index.rst ` + - documentation of the protocol as a ReST file - see :ref:`Documentation/= netlink/specs/index.rst ` - policy tables for input attribute validation - operation tables =20 --=20 2.49.0 From nobody Sun Dec 14 02:00:22 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D557A27465B; Mon, 28 Jul 2025 16:02:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753718543; cv=none; b=BchIJpoA/G0m9B8vuNI8e82tlhckkJAm94+L5ExK8SRRY2yM+4JP/VlW0UvqH8x4xB2W52vi75ORA+9FL/Nb/C5Xs9r/NraXYyD99USu0Wnobk11ut9srV9yImns0XWMuSL0HZhmjqMtJXRjCJng+W5LHCflHEaMlgk/JJBN/ME= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753718543; c=relaxed/simple; bh=jowhmr+KipOArxfk6mh3f69ZIZNViQ4HOjhtbsGep+s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=eHA9ZPtSWcM4XhuN1n0RaKLLWm+EZVy9M++R1yYorbcWEKv7US4XBypaTodqovVQlNVSVMFSJhkAXLDDM0XY6Z266CCi5yOn0ASi+lHdXh19vY/bSj6MNDl51zHqsjm2Zd2x/5o2OUv3thpItM/by0AtfJdLTcKhAOQ7Vb20htM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Y/LjokCZ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Y/LjokCZ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6A807C4CEFC; Mon, 28 Jul 2025 16:02:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1753718542; bh=jowhmr+KipOArxfk6mh3f69ZIZNViQ4HOjhtbsGep+s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Y/LjokCZoaE0bTjy1GzZOIxDi/El7KW7GD1BZOOPvLVTmquFM7ghBX82fm9UxsSHB WxNo7Ub4lyC19v5QrUkDqtAOAO9DWFPSpLkCLOrYIGKsxdpzUfvWFjVr0r3bp6J5vR MulQfzHyc2/aVUdkLRPkHpICanVh0MylAiPo8srsmeYq04nCV+OfRw4BomQL7GNIyN jtuR3HcFkMYrf+sSWH3o1KBUw5wQ4mXbVcf6B9dHVH8xv8n1qiyGS7wes9a/DptaWH GBDTNHA3Hdls71vTXs29fHFd6XCPxM7MtCsm/gu2xptvrGegaC4brgjtvTaZp4ATsO Eq5A+9zV61XLA== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1ugQIq-00000000Gd2-1l9X; Mon, 28 Jul 2025 18:02:16 +0200 From: Mauro Carvalho Chehab To: "Message-ID :" , Linux Doc Mailing List Cc: Mauro Carvalho Chehab , "Akira Yokosawa" , "Breno Leitao" , "David S. Miller" , "Donald Hunter" , "Eric Dumazet" , "Ignacio Encinas Rubio" , "Jakub Kicinski" , "Jan Stancek" , "Jonathan Corbet" , "Marco Elver" , "Paolo Abeni" , "Randy Dunlap" , "Ruben Wauters" , "Shuah Khan" , "Simon Horman" , joel@joelfernandes.org, linux-kernel-mentees@lists.linux.dev, linux-kernel@vger.kernel.org, lkmm@lists.linux.dev, netdev@vger.kernel.org, peterz@infradead.org, stern@rowland.harvard.edu Subject: [PATCH v10 08/14] tools: ynl_gen_rst.py: drop support for generating index files Date: Mon, 28 Jul 2025 18:02:01 +0200 Message-ID: <051ddbdd5b0c77126bc391c97014ae6b18bc4b82.1753718185.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" As we're now using an index file with a glob, there's no need to generate index files anymore. Signed-off-by: Mauro Carvalho Chehab Reviewed-by: Donald Hunter --- tools/net/ynl/pyynl/ynl_gen_rst.py | 28 ---------------------------- 1 file changed, 28 deletions(-) diff --git a/tools/net/ynl/pyynl/ynl_gen_rst.py b/tools/net/ynl/pyynl/ynl_g= en_rst.py index 010315fad498..90ae19aac89d 100755 --- a/tools/net/ynl/pyynl/ynl_gen_rst.py +++ b/tools/net/ynl/pyynl/ynl_gen_rst.py @@ -31,9 +31,6 @@ def parse_arguments() -> argparse.Namespace: =20 # Index and input are mutually exclusive group =3D parser.add_mutually_exclusive_group() - group.add_argument( - "-x", "--index", action=3D"store_true", help=3D"Generate the index= page" - ) group.add_argument("-i", "--input", help=3D"YAML file name") =20 args =3D parser.parse_args() @@ -63,27 +60,6 @@ def write_to_rstfile(content: str, filename: str) -> Non= e: rst_file.write(content) =20 =20 -def generate_main_index_rst(parser: YnlDocGenerator, output: str) -> None: - """Generate the `networking_spec/index` content and write to the file"= "" - lines =3D [] - - lines.append(parser.fmt.rst_header()) - lines.append(parser.fmt.rst_label("specs")) - lines.append(parser.fmt.rst_title("Netlink Family Specifications")) - lines.append(parser.fmt.rst_toctree(1)) - - index_dir =3D os.path.dirname(output) - logging.debug("Looking for .rst files in %s", index_dir) - for filename in sorted(os.listdir(index_dir)): - base, ext =3D os.path.splitext(filename) - if filename =3D=3D "index.rst" or ext not in [".rst", ".yaml"]: - continue - lines.append(f" {base}\n") - - logging.debug("Writing an index file at %s", output) - write_to_rstfile("".join(lines), output) - - def main() -> None: """Main function that reads the YAML files and generates the RST files= """ =20 @@ -102,10 +78,6 @@ def main() -> None: =20 write_to_rstfile(content, args.output) =20 - if args.index: - # Generate the index RST file - generate_main_index_rst(parser, args.output) - =20 if __name__ =3D=3D "__main__": main() --=20 2.49.0 From nobody Sun Dec 14 02:00:22 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BFE332741BD; Mon, 28 Jul 2025 16:02:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753718542; cv=none; b=Ng5JD85joUdncCxWDwWhmg40A7ch7dRtpiaGbWNQqZ7OAq8tBV7+5M9+6W6flqAjGZEZA1p90e/dUcjPnFfsQkQjdrbcBjCVsTrXBSAiPnDa/OynDL5iJSoELLPv0njqJXq2GyWj8M8vWT1bePVjMJFimB4bSxve04HGzolQi6Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753718542; c=relaxed/simple; bh=ogBoG80VZdRt95wsoC2snRc4eLHl6tjexBrTDr8vuXE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=a3lLPE2ygM3t/dNA9c/GRDn4pQW4jdGgm8Ss6s8YKu3ITjjzTROpJmOAe/QwKyWxZFPWC3gt0RqdTZKOFjumydRiyEIs3/CGs02jxSUvMI64y8OAd93j2+P00oiDJHOCarMzdh94x8W2pWcAQ/HT23SAjH/Jaza/r4Le/nTm+aA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=RT2bjzc+; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="RT2bjzc+" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 24C32C116C6; Mon, 28 Jul 2025 16:02:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1753718542; bh=ogBoG80VZdRt95wsoC2snRc4eLHl6tjexBrTDr8vuXE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RT2bjzc+oeTp5nVMl3M4RFauOKtPZ7kmyZJ93oJ4DUv+m+vSG7Y6Zbn5JqkvIZlB9 8DBNEXeAyvN+L/xERGkmUW0PROYgfTBxSzb0CVU+XUn7DzW3ZgNYrVKt7HGalyEYuF Y34ucS6FWlqV7vECKXnxlTQz613DaG6Zabv5APodcmiOB9ayIgtNf7YPwoUZWas02e Sbb19jcfDMt4MZMt3/IH61gv6AoqPF6m9457wPmMwHCMCIBsOcH4B2F00pUDxHDG0M NmBgaZSo9tztnk6uGRZQbICx63FjYGUnNQvKdsd0VsJFRk8XLgi2+tdlK52OLSh9Qb pu6JVnjRE2Hmw== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1ugQIq-00000000Gd5-1rq7; Mon, 28 Jul 2025 18:02:16 +0200 From: Mauro Carvalho Chehab To: "Message-ID :" , Linux Doc Mailing List Cc: Mauro Carvalho Chehab , "Akira Yokosawa" , "Breno Leitao" , "David S. Miller" , "Donald Hunter" , "Eric Dumazet" , "Ignacio Encinas Rubio" , "Jakub Kicinski" , "Jan Stancek" , "Jonathan Corbet" , "Marco Elver" , "Paolo Abeni" , "Randy Dunlap" , "Ruben Wauters" , "Shuah Khan" , "Simon Horman" , joel@joelfernandes.org, linux-kernel-mentees@lists.linux.dev, linux-kernel@vger.kernel.org, lkmm@lists.linux.dev, netdev@vger.kernel.org, peterz@infradead.org, stern@rowland.harvard.edu Subject: [PATCH v10 09/14] docs: netlink: remove obsolete .gitignore from unused directory Date: Mon, 28 Jul 2025 18:02:02 +0200 Message-ID: <8abdef54661349acc7d0240bcd1bb23c2f318548.1753718185.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" The previous code was generating source rst files under Documentation/networking/netlink_spec/. With the Sphinx YAML parser, this is now gone. So, stop ignoring *.rst files inside netlink specs directory. Signed-off-by: Mauro Carvalho Chehab Reviewed-by: Donald Hunter --- Documentation/networking/netlink_spec/.gitignore | 1 - 1 file changed, 1 deletion(-) delete mode 100644 Documentation/networking/netlink_spec/.gitignore diff --git a/Documentation/networking/netlink_spec/.gitignore b/Documentati= on/networking/netlink_spec/.gitignore deleted file mode 100644 index 30d85567b592..000000000000 --- a/Documentation/networking/netlink_spec/.gitignore +++ /dev/null @@ -1 +0,0 @@ -*.rst --=20 2.49.0 From nobody Sun Dec 14 02:00:22 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DDA91274671; Mon, 28 Jul 2025 16:02:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753718543; cv=none; b=qww0nfCgDiXNqN9RwJxNqh0EqDo6WOvn9+gWnNKX6RJYh+InNIRYRiix6ny2iVdl90jzNAsuhIqWMs7iiXI912iiLbWNhWjXQtnssSVbULShlItZ2gKx4iTUZqVAiFoJPoTY446zT70Ck7uvIlMQ5295AAXvM9msU8i/IQt8rmw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753718543; c=relaxed/simple; bh=I9xfKt4o8uiH29LP7HMJc0AeFwdIlfRvdpCS3v7bSi0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=eEGh5ppZu/qpmaVtE+92Fi7yCL7WT/8Ax78Te2Y8PvfDdVTytCAvRxZI3CkYAorUFEPQTJiMCW5IWRHCM2YPrYB/uWpYmpw3oxiTxcICXvVA470VBTtzgy6GSS++opq1o5t07v5UKsH1o+4OYhdjkO4K9hhhm6NNF6fOmtGNP1M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=pbeGIWVn; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="pbeGIWVn" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2D90FC16AAE; Mon, 28 Jul 2025 16:02:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1753718542; bh=I9xfKt4o8uiH29LP7HMJc0AeFwdIlfRvdpCS3v7bSi0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pbeGIWVnlZWehVcKLjOJZu2Bty8X8Ic9W+JMytrFnoScOd5VHg1tvQlATG8MFKALr b0IjAeXVGxmiJMt9eSONgrDWxLT0Y8Qso9Kjp5TaoK8b680xn3CubAtxl3XWhgKEIQ HZgS9YEMaLmhVHuuN4yAFgfO4g0BAJc3Cwhm3j0jfU4Zu3icB0mtWeKZJ+w00zKBPv 8MJAgouNTqTS5a3s73GX15/Vc/+Unj85ojWsVEJ2CQIduqUXd6jkt7Lcl2/FA4Pt1n dVU9ik1c897bqlac8IMulzWC7Xqf4uOla0oXwB9KgZLM8FtQMFMxH4u1ERMonQ3l5L PQJ2V7KRtyyMA== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1ugQIq-00000000Gd8-1xzg; Mon, 28 Jul 2025 18:02:16 +0200 From: Mauro Carvalho Chehab To: "Message-ID :" , Linux Doc Mailing List Cc: Mauro Carvalho Chehab , "Akira Yokosawa" , "Breno Leitao" , "David S. Miller" , "Donald Hunter" , "Eric Dumazet" , "Ignacio Encinas Rubio" , "Jakub Kicinski" , "Jan Stancek" , "Jonathan Corbet" , "Marco Elver" , "Paolo Abeni" , "Randy Dunlap" , "Ruben Wauters" , "Shuah Khan" , "Simon Horman" , joel@joelfernandes.org, linux-kernel-mentees@lists.linux.dev, linux-kernel@vger.kernel.org, lkmm@lists.linux.dev, netdev@vger.kernel.org, peterz@infradead.org, stern@rowland.harvard.edu Subject: [PATCH v10 10/14] MAINTAINERS: add netlink_yml_parser.py to linux-doc Date: Mon, 28 Jul 2025 18:02:03 +0200 Message-ID: X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" The documentation build depends on the parsing code at ynl_gen_rst.py. Ensure that changes to it will be c/c to linux-doc ML and maintainers by adding an entry for it. This way, if a change there would affect the build, or the minimal version required for Python, doc developers may know in advance. Signed-off-by: Mauro Carvalho Chehab Reviewed-by: Breno Leitao Reviewed-by: Donald Hunter --- MAINTAINERS | 1 + 1 file changed, 1 insertion(+) diff --git a/MAINTAINERS b/MAINTAINERS index b02127967322..f509a16054ec 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -7202,6 +7202,7 @@ F: scripts/get_abi.py F: scripts/kernel-doc* F: scripts/lib/abi/* F: scripts/lib/kdoc/* +F: tools/net/ynl/pyynl/lib/doc_generator.py F: scripts/sphinx-pre-install X: Documentation/ABI/ X: Documentation/admin-guide/media/ --=20 2.49.0 From nobody Sun Dec 14 02:00:22 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8F468273D7E; Mon, 28 Jul 2025 16:02:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753718542; cv=none; b=pJ/qLwhdA9To5G8zBvj7ULQrY/9ovJoXJBW7gEf5stidoHNiyb8pfkCQPvDn4vPKmb8KjHb85YfTAJNseaUlwEoUKeNUzG17YeCQJ6VDJkJNropfBRsekzjai8Av2uqxpik4m63/2RDPcNcDD7ift/lsm1xoCu00TPrEV0gkO/I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753718542; c=relaxed/simple; bh=deh3Qwq57w+TnIRPLss0GAT2BTvn4TViPnV+RhZDsuU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Cm0UZe/qCfPVcGagyk56l2bm0BkE+RqofeeVh97orB64IBYGAsHxwU8XIokWy1pHQXHAn7nMG/P0NVvceqbynnP4/fXX3xCxkPoP4Tjgma081CSy739mjve5rkeqxFoolCa8hYufCn0/b6yM1dBcvvHTlEAuTpT/cLf2TMzAOBk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Es92puf7; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Es92puf7" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 11B0DC4CEFB; Mon, 28 Jul 2025 16:02:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1753718542; bh=deh3Qwq57w+TnIRPLss0GAT2BTvn4TViPnV+RhZDsuU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Es92puf7cu8sxUCISF1zXpjJyy8zn8IR1uC6kiFV1bINXA27e5DZoH5+50aaPZLAn qEvh1zNBpYMMrU7VHETV6szwY07NYE9IwSwzitYnRSy6Xo9J+Qk3MVcH6+/9TfebDC ZqFDmDKhyXE9av0OaxhKSC/RsO1zRVuKD9Uh4IYmPZP1A3fsKbITxdIQT8mq7lyQuw f6jesAC1O7BmItg46rq30sbcf1KKolCoTLajpjSFjoZPw5pvKmY0BTFMfm1QJuKXtT BRcp06d0hS66R4BubPEgg/t41ZoTMPQf3sb2lQAp2WF/pubW3HiVLjzQj1R5rxAcbB AAtIF5Wlh0yBA== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1ugQIq-00000000GdB-24S3; Mon, 28 Jul 2025 18:02:16 +0200 From: Mauro Carvalho Chehab To: "Message-ID :" , Linux Doc Mailing List Cc: Mauro Carvalho Chehab , "Akira Yokosawa" , "Breno Leitao" , "David S. Miller" , "Donald Hunter" , "Eric Dumazet" , "Ignacio Encinas Rubio" , "Jakub Kicinski" , "Jan Stancek" , "Jonathan Corbet" , "Marco Elver" , "Paolo Abeni" , "Randy Dunlap" , "Ruben Wauters" , "Shuah Khan" , "Simon Horman" , joel@joelfernandes.org, linux-kernel-mentees@lists.linux.dev, linux-kernel@vger.kernel.org, lkmm@lists.linux.dev, netdev@vger.kernel.org, peterz@infradead.org, stern@rowland.harvard.edu Subject: [PATCH v10 11/14] tools: netlink_yml_parser.py: add line numbers to parsed data Date: Mon, 28 Jul 2025 18:02:04 +0200 Message-ID: <7b358e8c7684f0ac23b65245205d1d3bd776e47d.1753718185.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" When something goes wrong, we want Sphinx error to point to the right line number from the original source, not from the processed ReST data. Signed-off-by: Mauro Carvalho Chehab --- tools/net/ynl/pyynl/lib/doc_generator.py | 34 ++++++++++++++++++++++-- 1 file changed, 32 insertions(+), 2 deletions(-) diff --git a/tools/net/ynl/pyynl/lib/doc_generator.py b/tools/net/ynl/pyynl= /lib/doc_generator.py index 474b2b78c7bc..658759a527a6 100644 --- a/tools/net/ynl/pyynl/lib/doc_generator.py +++ b/tools/net/ynl/pyynl/lib/doc_generator.py @@ -20,6 +20,16 @@ from typing import Any, Dict, List import yaml =20 +LINE_STR =3D '__lineno__' + +class NumberedSafeLoader(yaml.SafeLoader): # pylint: disable= =3DR0901 + """Override the SafeLoader class to add line number to parsed data""" + + def construct_mapping(self, node, *args, **kwargs): + mapping =3D super().construct_mapping(node, *args, **kwargs) + mapping[LINE_STR] =3D node.start_mark.line + + return mapping =20 class RstFormatters: """RST Formatters""" @@ -127,6 +137,11 @@ class RstFormatters: """Return a formatted label""" return f".. _{title}:\n\n" =20 + @staticmethod + def rst_lineno(lineno: int) -> str: + """Return a lineno comment""" + return f".. LINENO {lineno}\n" + class YnlDocGenerator: """YAML Netlink specs Parser""" =20 @@ -144,6 +159,9 @@ class YnlDocGenerator: """Parse 'do' section and return a formatted string""" lines =3D [] for key in do_dict.keys(): + if key =3D=3D LINE_STR: + lines.append(self.fmt.rst_lineno(do_dict[key])) + continue lines.append(self.fmt.rst_paragraph(self.fmt.bold(key), level = + 1)) if key in ['request', 'reply']: lines.append(self.parse_do_attributes(do_dict[key], level = + 1) + "\n") @@ -174,6 +192,10 @@ class YnlDocGenerator: lines.append(self.fmt.rst_paragraph(operation["doc"]) + "\n") =20 for key in operation.keys(): + if key =3D=3D LINE_STR: + lines.append(self.fmt.rst_lineno(operation[key])) + continue + if key in preprocessed: # Skip the special fields continue @@ -233,6 +255,9 @@ class YnlDocGenerator: for definition in defs: lines.append(self.fmt.rst_section(namespace, 'definition', def= inition["name"])) for k in definition.keys(): + if k =3D=3D LINE_STR: + lines.append(self.fmt.rst_lineno(definition[k])) + continue if k in preprocessed + ignored: continue lines.append(self.fmt.rst_fields(k, self.fmt.sanitize(defi= nition[k]), 0)) @@ -268,6 +293,9 @@ class YnlDocGenerator: lines.append(self.fmt.rst_subsubsection(attr_line)) =20 for k in attr.keys(): + if k =3D=3D LINE_STR: + lines.append(self.fmt.rst_lineno(attr[k])) + continue if k in preprocessed + ignored: continue if k in linkable: @@ -306,6 +334,8 @@ class YnlDocGenerator: lines =3D [] =20 # Main header + lineno =3D obj.get('__lineno__', 0) + lines.append(self.fmt.rst_lineno(lineno)) =20 family =3D obj['name'] =20 @@ -354,7 +384,7 @@ class YnlDocGenerator: def parse_yaml_file(self, filename: str) -> str: """Transform the YAML specified by filename into an RST-formatted = string""" with open(filename, "r", encoding=3D"utf-8") as spec_file: - yaml_data =3D yaml.safe_load(spec_file) - content =3D self.parse_yaml(yaml_data) + numbered_yaml =3D yaml.load(spec_file, Loader=3DNumberedSafeLo= ader) + content =3D self.parse_yaml(numbered_yaml) =20 return content --=20 2.49.0 From nobody Sun Dec 14 02:00:22 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DB8EA27466A; Mon, 28 Jul 2025 16:02:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753718543; cv=none; b=pI5wmgkPvvDCs9q6SyzH7fXNF3U0uklutnkBvhcA0LvRN6gqZLPZlmVsmbnqhnY2f7kTy8AmoO8YeXEUGVumzaF+9MeT2LpQ1bRJJK0PfjXIBDVSkAECg/6LeHuve/qgxjaLA3P5dmIB9J9WHce9CTBJvmAB/ua3HHH5WSGYZ+M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753718543; c=relaxed/simple; bh=otVjsBKFUHYlcqJjnUpXdq/CVdL2g8H+2Ln8zBoihTA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=RQkV7tlxfWp9N888CnllH6CgwdT6WB3K7qpXoLfMs4Ooiv/6LHHLqof3z/rIyLWjAiazjURNdRKakmCi/WDo7aJ1EfKXr32LeFCGp1TbjCxnoIuOGtkRio2O1YTkWJtK0H5GL8KFQQlK7JPCpRp566cMysVo9MKY2v//9zL9rzE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=hviABLWC; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="hviABLWC" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3C575C2BC9E; Mon, 28 Jul 2025 16:02:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1753718542; bh=otVjsBKFUHYlcqJjnUpXdq/CVdL2g8H+2Ln8zBoihTA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hviABLWC3B52o465dGrJRSUARhJjIIcXjrDVvjfZ9P7qdrnp8t/+baZ0QPKmAEDFr RADNjM8I866j6YAahmRBkX+ZCgc6jqPh03WbwZbkNjrnDOQg8FV948eJf/gsom88wQ LTOrrvR8UfyH0XIODexiVGhv8wJGUTnRvKfLfIEI4eXseGdyXWI/4FmgM4SOeSx5Mp JqXZVnF8rN/9ufxg6q1qhyqFCyMe24khFdrw49s56BhcUqIFUwHmPHIMW4K5m9SCB9 dfq6G7sms29QlNY/GRwt1FIg98J6p74cmI9FkFGkcaXxCenqYCTZOazYUmmGR/z04w GI+oNKvug0hDg== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1ugQIq-00000000GdE-2C4A; Mon, 28 Jul 2025 18:02:16 +0200 From: Mauro Carvalho Chehab To: "Message-ID :" , Linux Doc Mailing List Cc: Mauro Carvalho Chehab , "Akira Yokosawa" , "Breno Leitao" , "David S. Miller" , "Donald Hunter" , "Eric Dumazet" , "Ignacio Encinas Rubio" , "Jakub Kicinski" , "Jan Stancek" , "Jonathan Corbet" , "Marco Elver" , "Paolo Abeni" , "Randy Dunlap" , "Ruben Wauters" , "Shuah Khan" , "Simon Horman" , joel@joelfernandes.org, linux-kernel-mentees@lists.linux.dev, linux-kernel@vger.kernel.org, lkmm@lists.linux.dev, netdev@vger.kernel.org, peterz@infradead.org, stern@rowland.harvard.edu Subject: [PATCH v10 12/14] docs: parser_yaml.py: add support for line numbers from the parser Date: Mon, 28 Jul 2025 18:02:05 +0200 Message-ID: X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" Instead of printing line numbers from the temp converted ReST file, get them from the original source. Signed-off-by: Mauro Carvalho Chehab --- Documentation/sphinx/parser_yaml.py | 12 ++++++++++-- tools/net/ynl/pyynl/lib/doc_generator.py | 16 ++++++++++++---- 2 files changed, 22 insertions(+), 6 deletions(-) diff --git a/Documentation/sphinx/parser_yaml.py b/Documentation/sphinx/par= ser_yaml.py index fa2e6da17617..8288e2ff7c7c 100755 --- a/Documentation/sphinx/parser_yaml.py +++ b/Documentation/sphinx/parser_yaml.py @@ -54,6 +54,8 @@ class YamlParser(Parser): =20 netlink_parser =3D YnlDocGenerator() =20 + re_lineno =3D re.compile(r"\.\. LINENO ([0-9]+)$") + def rst_parse(self, inputstring, document, msg): """ Receives a ReST content that was previously converted by the @@ -66,8 +68,14 @@ class YamlParser(Parser): =20 try: # Parse message with RSTParser - for i, line in enumerate(msg.split('\n')): - result.append(line, document.current_source, i) + lineoffset =3D 0; + for line in msg.split('\n'): + match =3D self.re_lineno.match(line) + if match: + lineoffset =3D int(match.group(1)) + continue + + result.append(line, document.current_source, lineoffset) =20 rst_parser =3D RSTParser() rst_parser.parse('\n'.join(result), document) diff --git a/tools/net/ynl/pyynl/lib/doc_generator.py b/tools/net/ynl/pyynl= /lib/doc_generator.py index 658759a527a6..403abf1a2eda 100644 --- a/tools/net/ynl/pyynl/lib/doc_generator.py +++ b/tools/net/ynl/pyynl/lib/doc_generator.py @@ -158,9 +158,11 @@ class YnlDocGenerator: def parse_do(self, do_dict: Dict[str, Any], level: int =3D 0) -> str: """Parse 'do' section and return a formatted string""" lines =3D [] + if LINE_STR in do_dict: + lines.append(self.fmt.rst_lineno(do_dict[LINE_STR])) + for key in do_dict.keys(): if key =3D=3D LINE_STR: - lines.append(self.fmt.rst_lineno(do_dict[key])) continue lines.append(self.fmt.rst_paragraph(self.fmt.bold(key), level = + 1)) if key in ['request', 'reply']: @@ -187,13 +189,15 @@ class YnlDocGenerator: lines =3D [] =20 for operation in operations: + if LINE_STR in operation: + lines.append(self.fmt.rst_lineno(operation[LINE_STR])) + lines.append(self.fmt.rst_section(namespace, 'operation', operation["name"])) lines.append(self.fmt.rst_paragraph(operation["doc"]) + "\n") =20 for key in operation.keys(): if key =3D=3D LINE_STR: - lines.append(self.fmt.rst_lineno(operation[key])) continue =20 if key in preprocessed: @@ -253,10 +257,12 @@ class YnlDocGenerator: lines =3D [] =20 for definition in defs: + if LINE_STR in definition: + lines.append(self.fmt.rst_lineno(definition[LINE_STR])) + lines.append(self.fmt.rst_section(namespace, 'definition', def= inition["name"])) for k in definition.keys(): if k =3D=3D LINE_STR: - lines.append(self.fmt.rst_lineno(definition[k])) continue if k in preprocessed + ignored: continue @@ -284,6 +290,9 @@ class YnlDocGenerator: lines.append(self.fmt.rst_section(namespace, 'attribute-set', entry["name"])) for attr in entry["attributes"]: + if LINE_STR in attr: + lines.append(self.fmt.rst_lineno(attr[LINE_STR])) + type_ =3D attr.get("type") attr_line =3D attr["name"] if type_: @@ -294,7 +303,6 @@ class YnlDocGenerator: =20 for k in attr.keys(): if k =3D=3D LINE_STR: - lines.append(self.fmt.rst_lineno(attr[k])) continue if k in preprocessed + ignored: continue --=20 2.49.0 From nobody Sun Dec 14 02:00:22 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C17192741C2; Mon, 28 Jul 2025 16:02:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753718542; cv=none; b=Jj8ocLNtAeJurhFXDF3CXHskg1ETeGYQQPZbLkAHvw3TaB7i1vcDs1aaM0o4nZi5Ztsj+Kx7FDXrbrDvMzG/gLZ4GoBYBDBvRRnogPKEb2LQkyAAXhmdUZzS7re0iQgUKYalgp5HD9yLp6h6VZXwbJ3O3at018JkKsKc5dMKZIc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753718542; c=relaxed/simple; bh=/8GKUg/hE6EZzmvBGeqtvN5XqiYkoQRfu9Nd+64B7DM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TTfucKgjp/swqIuxjgXHtLvewWtA3VL0GYngopJOiZq04XVyjlwqbll0nts8bXl0twNxX0ZifVCEg1acFk4UpT+eEzrxTih/aeAG9+eTtWRDw4R+2hbKTn/JDx12AxhJZpe0/N6/wq2nEXN1Dppi+iPqUKUe3aJ/Mrg9u8dWNCI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=dXJSLu6L; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="dXJSLu6L" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3865DC2BC87; Mon, 28 Jul 2025 16:02:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1753718542; bh=/8GKUg/hE6EZzmvBGeqtvN5XqiYkoQRfu9Nd+64B7DM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dXJSLu6LcGij5t1mTm16tztOBw7o78gJBAacpUCeCkX6tUKYT4GjlZbIzWHYIpUAA nybwD/1FjADUS3DODns7P1MwMwuNDLtfFUtEh7Pnb4x/7fuSuALk2TGQm2NeUQpcWw Luh6NXdK1ZOMYxEO+7ysOe5/31dh2Uqg6KIjKSQ8iLh86trwXhKn04WxbN0mddq3Ki abs3u2qVI4gmcWQtbOgxvAgvYWOxZHOsW4eAJSfTrBOObqABB+T856ZOHXKEpr1Vk1 rpPXEhmijg6lyDz4ovQUpueDQwWDPpKBIxoQ8itWmURbtTHmEmtciwSAQ2OWG0PUF4 +aMJypVaoo1wQ== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1ugQIq-00000000GdH-2IKi; Mon, 28 Jul 2025 18:02:16 +0200 From: Mauro Carvalho Chehab To: "Message-ID :" , Linux Doc Mailing List Cc: Mauro Carvalho Chehab , "Akira Yokosawa" , "Breno Leitao" , "David S. Miller" , "Donald Hunter" , "Eric Dumazet" , "Ignacio Encinas Rubio" , "Jakub Kicinski" , "Jan Stancek" , "Jonathan Corbet" , "Marco Elver" , "Paolo Abeni" , "Randy Dunlap" , "Ruben Wauters" , "Shuah Khan" , "Simon Horman" , joel@joelfernandes.org, linux-kernel-mentees@lists.linux.dev, linux-kernel@vger.kernel.org, lkmm@lists.linux.dev, netdev@vger.kernel.org, peterz@infradead.org, stern@rowland.harvard.edu Subject: [PATCH v10 13/14] docs: parser_yaml.py: fix backward compatibility with old docutils Date: Mon, 28 Jul 2025 18:02:06 +0200 Message-ID: <9715e41c91302e2b8a9baf349903f3fcff726295.1753718185.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" As reported by Akira, older docutils versions are not compatible with the way some Sphinx versions send tab_width. Add a code to address it. Reported-by: Akira Yokosawa Closes: https://lore.kernel.org/linux-doc/598b2cb7-2fd7-4388-96ba-2ddf0ab55= d2a@gmail.com/ Tested-by: Akira Yokosawa Signed-off-by: Mauro Carvalho Chehab --- Documentation/sphinx/parser_yaml.py | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/Documentation/sphinx/parser_yaml.py b/Documentation/sphinx/par= ser_yaml.py index 8288e2ff7c7c..1602b31f448e 100755 --- a/Documentation/sphinx/parser_yaml.py +++ b/Documentation/sphinx/parser_yaml.py @@ -77,6 +77,10 @@ class YamlParser(Parser): =20 result.append(line, document.current_source, lineoffset) =20 + # Fix backward compatibility with docutils < 0.17.1 + if "tab_width" not in vars(document.settings): + document.settings.tab_width =3D 8 + rst_parser =3D RSTParser() rst_parser.parse('\n'.join(result), document) =20 --=20 2.49.0 From nobody Sun Dec 14 02:00:22 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1B792274B38; Mon, 28 Jul 2025 16:02:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753718543; cv=none; b=F0esV9kE9Cx9yv0HQmZOnntIRyOWhsqRGfLBz3yq6vyWb2o18MitJfrUnRlFtUnrb44juAN7WyaIYqm+mfhX8G9zkdHdl+OXdy3yf60hVFrBvsj+VA/xN/dsnXbScFBgnFSDed+HmZZRfG5ip121w96ij/Tq5Osirjvj8v1AKNk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753718543; c=relaxed/simple; bh=Re34NdFIFBEjM60ZOokQBSoSZa53NQDJZGofuj8tscY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=jXWKxw7GzPE3+WQCow0teMXm4uFYweB81o4miEvo6yoU32KAAKl9cQ+0t65FmhTxfXEY0x+4QPHn8EfUWVBDStfZolJs/rtM3Kz8iORyJpjPsN8rYDIR6EaZ2p1NjeJBwbObfxwM6zpzMZGZKfy8PZaYYyivUMc3M12OaYQ3zlo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=G9WTJHLh; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="G9WTJHLh" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 73773C113D0; Mon, 28 Jul 2025 16:02:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1753718542; bh=Re34NdFIFBEjM60ZOokQBSoSZa53NQDJZGofuj8tscY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=G9WTJHLhvpI0HtXYEFfFu5EMSp8YbHOPg/6SAELkZ/NBvbshoF0lnSEUXAJxlCiiw zWB3hl78PbGnmveFYqIYmfsJ/08ybTVyAwNjo0JoVw2SsFApYell+BNRf/cQn/+JF5 bu8R9BMJMeBzQbWwJfnkoWQONyMEBXX9KH8dhVDGoin1KAkYAXqbuFADweJyACwb8t Fp6JOyo56SDrWCQ55JgAcLZeDMbWdQE7ao4ygMqjHoFdiBdQ7fZvmE8dI4UiIR8szz pHBjAtNyzZIzsTvJu8/nM7N177MrOV7TrfaKAn1iqYI/18nyYm2mTE8K8hG/70atWr ISLUt9Yu7jkDg== Received: from mchehab by mail.kernel.org with local (Exim 4.98.2) (envelope-from ) id 1ugQIq-00000000GdK-2OZs; Mon, 28 Jul 2025 18:02:16 +0200 From: Mauro Carvalho Chehab To: "Message-ID :" , Linux Doc Mailing List Cc: Mauro Carvalho Chehab , "Akira Yokosawa" , "Breno Leitao" , "David S. Miller" , "Donald Hunter" , "Eric Dumazet" , "Ignacio Encinas Rubio" , "Jakub Kicinski" , "Jan Stancek" , "Jonathan Corbet" , "Marco Elver" , "Paolo Abeni" , "Randy Dunlap" , "Ruben Wauters" , "Shuah Khan" , "Simon Horman" , joel@joelfernandes.org, linux-kernel-mentees@lists.linux.dev, linux-kernel@vger.kernel.org, lkmm@lists.linux.dev, netdev@vger.kernel.org, peterz@infradead.org, stern@rowland.harvard.edu Subject: [PATCH v10 14/14] sphinx: parser_yaml.py: fix line numbers information Date: Mon, 28 Jul 2025 18:02:07 +0200 Message-ID: X-Mailer: git-send-email 2.49.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" As reported by Donald, this code: rst_parser =3D RSTParser() rst_parser.parse('\n'.join(result), document) breaks line parsing. As an alternative, I tested a variant of it: rst_parser.parse(result, document) but still line number was not preserved. As Donald noted, standard Parser classes don't have a direct mechanism to preserve line numbers from ViewList(). So, instead, let's use a mechanism similar to what we do already at kerneldoc.py: call the statemachine mechanism directly there. I double-checked when states and statemachine were introduced: both were back in 2002. I also tested doc build with docutils 0.16 and 0.21.2. It worked with both, so it seems to be stable enough for our needs. Reported-by: Donald Hunter Closes: https://lore.kernel.org/linux-doc/m24ivk78ng.fsf@gmail.com/T/#u Signed-off-by: Mauro Carvalho Chehab --- Documentation/sphinx/parser_yaml.py | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/Documentation/sphinx/parser_yaml.py b/Documentation/sphinx/par= ser_yaml.py index 1602b31f448e..634d84a202fc 100755 --- a/Documentation/sphinx/parser_yaml.py +++ b/Documentation/sphinx/parser_yaml.py @@ -11,7 +11,9 @@ import sys =20 from pprint import pformat =20 +from docutils import statemachine from docutils.parsers.rst import Parser as RSTParser +from docutils.parsers.rst import states from docutils.statemachine import ViewList =20 from sphinx.util import logging @@ -56,6 +58,8 @@ class YamlParser(Parser): =20 re_lineno =3D re.compile(r"\.\. LINENO ([0-9]+)$") =20 + tab_width =3D 8 + def rst_parse(self, inputstring, document, msg): """ Receives a ReST content that was previously converted by the @@ -66,10 +70,18 @@ class YamlParser(Parser): =20 result =3D ViewList() =20 + self.statemachine =3D states.RSTStateMachine(state_classes=3Dstate= s.state_classes, + initial_state=3D'Body', + debug=3Ddocument.report= er.debug_flag) + try: # Parse message with RSTParser lineoffset =3D 0; - for line in msg.split('\n'): + + lines =3D statemachine.string2lines(msg, self.tab_width, + convert_whitespace=3DTrue) + + for line in lines: match =3D self.re_lineno.match(line) if match: lineoffset =3D int(match.group(1)) @@ -77,12 +89,7 @@ class YamlParser(Parser): =20 result.append(line, document.current_source, lineoffset) =20 - # Fix backward compatibility with docutils < 0.17.1 - if "tab_width" not in vars(document.settings): - document.settings.tab_width =3D 8 - - rst_parser =3D RSTParser() - rst_parser.parse('\n'.join(result), document) + self.statemachine.run(result, document) =20 except Exception as e: document.reporter.error("YAML parsing error: %s" % pformat(e)) --=20 2.49.0