From nobody Sat Feb 7 14:01:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1D1501C82F4; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; cv=none; b=XKzuzH44PdV1cW2Qtul4619iflqk4Lw6FVL30vbCOiswlbjyvErodnn7PDS71MtlvnzdRwG46Hbw7vz92LaGOVE7Igfsc7Xc9TEmUdg44x6Th+y4Z13eMC0MJrAYrAJlyJFvxLkIrCwQqU/kJddEYvHm4yzZo1GScCfH1eqCVkw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; c=relaxed/simple; bh=aeuqk5Ez6XBK1bcBiwdwu3D8K0LeqXVto+XLcOhJeWA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Pxb3khz+0ld4cNb7/f0/cUNbBzOx9+2RO+PExBTSCCnYDOMzjXrGyK+V6iXahixXHg88Ca9en+IBocOovYRfUcRrpi5Zx7nh8W7naMg39bJrgtccguB7xib0Ubx0ByeZESEp5fe5sSTYNv+f/ZEbtMu7kwUr8te2whQyIMzJ2lk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=J5iNvIEi; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="J5iNvIEi" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E1077C4CEEB; Wed, 19 Feb 2025 08:32:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739953971; bh=aeuqk5Ez6XBK1bcBiwdwu3D8K0LeqXVto+XLcOhJeWA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=J5iNvIEiipPo7GxZiDEnaXU1XLfO/KSoV40wNgDuzdBkGvyBgTXmcBvD1D9H4vhwn eed7soqaUQvDVpTJHCn9zzFV3x9eozPWmr6Ce2c1tIgg1hGxS4b501DYJbCIhFszbg +yZL0LT2qrquT/zsDmWZMD2Avv6noTkW8lv/TS5ALEXuNMr/4KiIYNqZdu/qjRIvS3 i/7AU4GfPTpynn3KfXkfcvkOKsCHKfdUCdb/COfmR691PghSIJu661FtiirymaYc4w 6sP3GdJLqoOsu4orpy2+rskNkvj5N75D5TEMCvxAu9aN9ky1foNW0Fj2B7M0nPCj7q HZJQsmy2JyBXg== Received: from mchehab by mail.kernel.org with local (Exim 4.98) (envelope-from ) id 1tkfVh-0000000Gv4L-4122; Wed, 19 Feb 2025 09:32:49 +0100 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , "Mauro Carvalho Chehab" , Arnd Bergmann , linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 01/27] include/asm-generic/io.h: fix kerneldoc markup Date: Wed, 19 Feb 2025 09:32:17 +0100 Message-ID: <53fe92cda720035a5c72bc95a88a3d965bc9b1fc.1739952783.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" Kerneldoc requires a "-" after the name of a function for it to be recognized as a function. Add it. Fix those kernel-doc warnings: include/asm-generic/io.h:1215: warning: This comment starts with '/**', but= isn't a kernel-doc comment. Refer Documentation/doc-guide/kernel-doc.rst * memset_io Set a range of I/O memory to a constant value include/asm-generic/io.h:1227: warning: This comment starts with '/**', but= isn't a kernel-doc comment. Refer Documentation/doc-guide/kernel-doc.rst * memcpy_fromio Copy a block of data from I/O memory include/asm-generic/io.h:1239: warning: This comment starts with '/**', but= isn't a kernel-doc comment. Refer Documentation/doc-guide/kernel-doc.rst * memcpy_toio Copy a block of data into I/O memory Signed-off-by: Mauro Carvalho Chehab Acked-by: Arnd Bergmann --- include/asm-generic/io.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/include/asm-generic/io.h b/include/asm-generic/io.h index a5cbbf3e26ec..3c61c29ff6ab 100644 --- a/include/asm-generic/io.h +++ b/include/asm-generic/io.h @@ -1212,7 +1212,7 @@ static inline void unxlate_dev_mem_ptr(phys_addr_t ph= ys, void *addr) =20 #ifndef memset_io /** - * memset_io Set a range of I/O memory to a constant value + * memset_io - Set a range of I/O memory to a constant value * @addr: The beginning of the I/O-memory range to set * @val: The value to set the memory to * @count: The number of bytes to set @@ -1224,7 +1224,7 @@ void memset_io(volatile void __iomem *addr, int val, = size_t count); =20 #ifndef memcpy_fromio /** - * memcpy_fromio Copy a block of data from I/O memory + * memcpy_fromio - Copy a block of data from I/O memory * @dst: The (RAM) destination for the copy * @src: The (I/O memory) source for the data * @count: The number of bytes to copy @@ -1236,7 +1236,7 @@ void memcpy_fromio(void *dst, const volatile void __i= omem *src, size_t count); =20 #ifndef memcpy_toio /** - * memcpy_toio Copy a block of data into I/O memory + * memcpy_toio - Copy a block of data into I/O memory * @dst: The (I/O memory) destination for the copy * @src: The (RAM) source for the data * @count: The number of bytes to copy --=20 2.48.1 From nobody Sat Feb 7 14:01:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1D09E1C7019; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; cv=none; b=Q8+9FQOde+v5UYF60DD8iJZlapiulsy5Y+c7Nlv71ZrznmJHhLPR40vsmPlT/h+VVJHbMe9QVuhMNEm6yZ/VViY+E4efzTaGm4p8aCPAlfp1LoC2ENP2ZywZ74zhjk/b9DxHK78N5C9TX7VSeuBmPWOmNxUC9w7gQJyV0dajyjQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; c=relaxed/simple; bh=dpp4qyIMyF9Zjj8hBit/kiPWhv8Tsh4yLBwdy/oLV7M=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=s+Fa5Pomj16sFvCt/y+mibAHL8uNhhW6MZhzoP+rj5sTrmMh1JXMF/ouxhNPHoPvc4kGRZfki/tKjsu2iLbzTlMz35MTZzXw5VwD7LJ+nWJoWpyx9H0eD0kzRIq4vtHdINHGF8S4Kv0LyvVO81UggSzfBIIiFhInI5nlnvspqqw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=mNYbvHPH; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="mNYbvHPH" Received: by smtp.kernel.org (Postfix) with ESMTPSA id DB8BAC4CEE7; Wed, 19 Feb 2025 08:32:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739953971; bh=dpp4qyIMyF9Zjj8hBit/kiPWhv8Tsh4yLBwdy/oLV7M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mNYbvHPHc/9E7jcpK6rMPaBalq+I85teunOusdpmMnnMZBjd3XTE3MLxXbwGn+Khs vifc8SvjP9uNGUSUIZ+/ZHOW9ilPYd5JPnJMLijP9GLQ2o6b7j5VnJaWx89ijc/45R Tf+3heHqLze0H88SxDeIyF/lKCct7/dDsN7FwwqFJLwD/hU9cWTlv5OH4I+vO7UT0G hTtGz6ifQM9/+pVtYFOwRjghkdEhPSa2aUmLa+PymNJtpFaURwMQ/qEBE2IZQXHXLM zsVmT4ReagerAFr6JRgWBZb3LsmICXuEnvFNKwyvUb2WOd+8Ie3ToFAzMtyxoxkf4A 19GctKkeKkqJQ== Received: from mchehab by mail.kernel.org with local (Exim 4.98) (envelope-from ) id 1tkfVh-0000000Gv4P-48Az; Wed, 19 Feb 2025 09:32:49 +0100 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , "Mauro Carvalho Chehab" , Bingbu Cao , Greg Kroah-Hartman , Sakari Ailus , Tianshu Qiu , linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linux-staging@lists.linux.dev Subject: [PATCH 02/27] drivers: media: intel-ipu3.h: fix identation on a kernel-doc markup Date: Wed, 19 Feb 2025 09:32:18 +0100 Message-ID: <8cad70e8f536c1a963c6b827347df1f7052fc5e5.1739952783.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" The "Rule" description is part of y_calc parameter. Having a line starting at the beginning makes it part of the function description instead, which is not the original intent. Signed-off-by: Mauro Carvalho Chehab --- drivers/staging/media/ipu3/include/uapi/intel-ipu3.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/staging/media/ipu3/include/uapi/intel-ipu3.h b/drivers= /staging/media/ipu3/include/uapi/intel-ipu3.h index 4aa2797f5e3c..8b85524beb59 100644 --- a/drivers/staging/media/ipu3/include/uapi/intel-ipu3.h +++ b/drivers/staging/media/ipu3/include/uapi/intel-ipu3.h @@ -322,7 +322,8 @@ struct ipu3_uapi_ae_config { * 0: positive, 1: negative, default 0. * @y_calc: Pre-processing that converts Bayer quad to RGB+Y values to be * used for building histogram. Range [0, 32], default 8. - * Rule: + * + * Rule: * y_gen_rate_gr + y_gen_rate_r + y_gen_rate_b + y_gen_rate_gb =3D 32 * A single Y is calculated based on sum of Gr/R/B/Gb based on * their contribution ratio. --=20 2.48.1 From nobody Sat Feb 7 14:01:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1CFF31BE23E; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; cv=none; b=XEdypR2vBQ92Zw7E+hOBLGo2jiWpkbC5f6jRFcDIA8qHeF3VEqMwvJqzllDxTayNnAR/wAeiJoDV71pZjs2stml+sJ8/XzSTr2ksztwyxhVACLTe5bqG/6sKOitYM1wfhunnLmdPd021wMKvF0gL5tD57IYafS462s1R/Z5XcNY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; c=relaxed/simple; bh=EuSV6D3szJ0VW+HjV2NwWusmhdj/JnS4ByP03OXscaU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=VZ6J+cY7GXRsWNKhzXb2/hsjl6H33oqSnQR77fb7ITE5KdSSFtIm2I8ZPNQqD3yYeV8x77Cv4xOh8uUiWuLh0XhaaEqQgTlx+tKKAnQA/HEpam9hEE44+7h6QYiIt6u0f9vng+uC8JXZR6ILLJ9tDImD4EScPsN9CuFGp1wkRec= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=f3dqcQX7; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="f3dqcQX7" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D4DCEC4CEE6; Wed, 19 Feb 2025 08:32:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739953971; bh=EuSV6D3szJ0VW+HjV2NwWusmhdj/JnS4ByP03OXscaU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=f3dqcQX7HKpLOq9UuAsVZvG+1e7cu8jhCTyw9qStK3Wy1gxr24aRyzVIKPYlcY5ZM aESzNHJ7xm6eZhH0VvbF6V9wXUuD467jQ2s4s4uaK+iOac1u8UU6vXefyVuDZhSUqT oq9YYi25VP5AdaD/8KbdEe95Zq6OPpDfRS8yACo5bmv6SDv4pr4mIrMflST9xNAdak Rmt7ooaGXd6dqXpa59SftHFKYcpc/Itd/gILJVTgv8ah9VmNiPS33QSsLphBrkUhff OMKxot/N3Zfnm+idHoXRLOuGSr9Z/PXRkf0Z5P3C7P68GD+71rTivCCxhv8Fdi9iff M2S1l/tQQTfJQ== Received: from mchehab by mail.kernel.org with local (Exim 4.98) (envelope-from ) id 1tkfVi-0000000Gv4T-02dM; Wed, 19 Feb 2025 09:32:50 +0100 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , "Mauro Carvalho Chehab" , Takashi Sakamoto , linux-kernel@vger.kernel.org, linux1394-devel@lists.sourceforge.net Subject: [PATCH 03/27] drivers: firewire: firewire-cdev.h: fix identation on a kernel-doc markup Date: Wed, 19 Feb 2025 09:32:19 +0100 Message-ID: <56d88f897214cbfc4593b4bb4b2a04d0168865d7.1739952783.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" The description of @tstamp parameter has one line that starts at the beginning. This moves such line to the description, which is not the intent here. Signed-off-by: Mauro Carvalho Chehab --- include/uapi/linux/firewire-cdev.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/include/uapi/linux/firewire-cdev.h b/include/uapi/linux/firewi= re-cdev.h index 1f2c9469f921..05e3aa8fa8bc 100644 --- a/include/uapi/linux/firewire-cdev.h +++ b/include/uapi/linux/firewire-cdev.h @@ -449,7 +449,8 @@ struct fw_cdev_event_phy_packet { * which the packet arrived. For %FW_CDEV_EVENT_PHY_PACKET_SENT2 and non-= ping packet, * the time stamp of isochronous cycle at which the packet was sent. For = ping packet, * the tick count for round-trip time measured by 1394 OHCI controller. - * The time stamp of isochronous cycle at which either the response was se= nt for + * + * The time stamp of isochronous cycle at which either the response was s= ent for * %FW_CDEV_EVENT_PHY_PACKET_SENT2 or the request arrived for * %FW_CDEV_EVENT_PHY_PACKET_RECEIVED2. * @data: Incoming data --=20 2.48.1 From nobody Sat Feb 7 14:01:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1D0F11C7B62; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; cv=none; b=fNa+oiVNgjD1clXvXd0VY7zalf2HLVtvm0YzFNBOQorytkndW9iWZlvA65MI7PNtWEUes4W0stwndtveLBoZZL8z+XpxfMYOyAcCh/rQaGHy/LXX6ekwdy3oKBZKhyGVjaBD+dcVA6Rn6StNDYSLF0HIBPOaWmkDntEiJeEolY4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; c=relaxed/simple; bh=UcUtD1ZVE/t/e49mIRjHjGIlObJeJfXBstuSw82d2G4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Qm0J3zxm8CXtZcpiDeE0TzBbACmOp7Wrulnl7hqe95ubn81PDKMIZLRx2bG23zI12vdKYOD+foHa4MDnT1WC83T5D3jSLyITzev7OgzNkrrZZ41TwJcYTFI9Vh6nT3VN3291usXQLM3JI0UDtxMnaF6Tb2d38Y5/FJ65pT53voo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=H6LRgT6s; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="H6LRgT6s" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D5A16C4CEE8; Wed, 19 Feb 2025 08:32:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739953971; bh=UcUtD1ZVE/t/e49mIRjHjGIlObJeJfXBstuSw82d2G4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=H6LRgT6skiHuwnztX0mroM6CF36RVrS/RsNANn8HW+hgqZFCQE6xqRlBntmTN+LqY qp428kYMxsr6jKytvO1VGV3h3RX6yxQOQO7ZKuCpNG7HA12g/PzC89PUOOa+XLuBf7 ZW2w9FxXDYZ/jh4ACkkOQ6HwnJsIoypXyq8Afnqgy5S2aqqQ/jjHHz8YKdncrSw5IG AgiNr3ELSehF3Ykl313Bm3Y8xLwoE446AuILHSx0tyoS+yG9pvr0HcVL3vDltaaEBl xzcVNEo+r1oNptQdZ7DHKDWjtYzcysl9k2xR1jtX3jNxGGWB8pBYQDe/8E9akz1KPX wAhJ+sDa5b3Ag== Received: from mchehab by mail.kernel.org with local (Exim 4.98) (envelope-from ) id 1tkfVi-0000000Gv4X-097P; Wed, 19 Feb 2025 09:32:50 +0100 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , "Mauro Carvalho Chehab" , linux-kernel@vger.kernel.org Subject: [PATCH 04/27] docs: driver-api/infiniband.rst: fix Kerneldoc markup Date: Wed, 19 Feb 2025 09:32:20 +0100 Message-ID: <033bcbcc6edaadda6dae59a66530e1603ae6948a.1739952783.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" kerneldoc.py extension doesn't handle a "\" character at the end of the line: it will just merge it to the cmd line, producing this command: scripts/kernel-doc -rst -enable-lineno -function iscsi_iser_pdu_alloc -fun= ction iser_initialize_task_headers -function \ -function iscsi_iser_task_in= it -function iscsi_iser_mtask_xmit -function iscsi_iser_task_xmit -function= \ -function iscsi_iser_cleanup_task -function iscsi_iser_check_protection = -function \ -function iscsi_iser_conn_create -function iscsi_iser_conn_bind= -function \ -function iscsi_iser_conn_start -function iscsi_iser_conn_stop= -function \ -function iscsi_iser_session_destroy -function iscsi_iser_sess= ion_create -function \ -function iscsi_iser_set_param -function iscsi_iser_= ep_connect -function iscsi_iser_ep_poll -function \ -function iscsi_iser_ep= _disconnect ./drivers/infiniband/ulp/iser/iscsi_iser.c which may not work as expected. Signed-off-by: Mauro Carvalho Chehab --- Documentation/driver-api/infiniband.rst | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/Documentation/driver-api/infiniband.rst b/Documentation/driver= -api/infiniband.rst index 30e142ccbee9..10d8be9e74fe 100644 --- a/Documentation/driver-api/infiniband.rst +++ b/Documentation/driver-api/infiniband.rst @@ -77,14 +77,14 @@ iSCSI Extensions for RDMA (iSER) :internal: =20 .. kernel-doc:: drivers/infiniband/ulp/iser/iscsi_iser.c - :functions: iscsi_iser_pdu_alloc iser_initialize_task_headers \ - iscsi_iser_task_init iscsi_iser_mtask_xmit iscsi_iser_task_xmit \ - iscsi_iser_cleanup_task iscsi_iser_check_protection \ - iscsi_iser_conn_create iscsi_iser_conn_bind \ - iscsi_iser_conn_start iscsi_iser_conn_stop \ - iscsi_iser_session_destroy iscsi_iser_session_create \ - iscsi_iser_set_param iscsi_iser_ep_connect iscsi_iser_ep_poll \ - iscsi_iser_ep_disconnect + :functions: iscsi_iser_pdu_alloc iser_initialize_task_headers + iscsi_iser_task_init iscsi_iser_mtask_xmit iscsi_iser_task_= xmit + iscsi_iser_cleanup_task iscsi_iser_check_protection + iscsi_iser_conn_create iscsi_iser_conn_bind + iscsi_iser_conn_start iscsi_iser_conn_stop + iscsi_iser_session_destroy iscsi_iser_session_create + iscsi_iser_set_param iscsi_iser_ep_connect iscsi_iser_ep_po= ll + iscsi_iser_ep_disconnect =20 .. kernel-doc:: drivers/infiniband/ulp/iser/iser_initiator.c :internal: --=20 2.48.1 From nobody Sat Feb 7 14:01:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 645D31D416E; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; cv=none; b=gT8ksfC0u/TatgN2NRm2wlhDOmYddt5Eqi9tLC/cc/BymcZEkbWC62v99oYjLuchdvp9c11azU//L9ItV4dlmk2DECYxN3C56I4EMTK0Vmcdq/uLT96isySX2SaABiQdyifmJ9M+g79My6ameBNfqK5aoSVFXDk2B0zcquVjNKw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; c=relaxed/simple; bh=yvf7sytYLZ4Yo2xKDIwz8bS24pWVxbZ/2mOY1uefT38=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=u14y8BHaBvWV70opz2fHp3qlT2a4cw0RZYeIk4+tmiuqkl+y9Er5tX3j1YBLC9msenfwrqT1j5DHS8SqcURWFLNXIeQk/iEev5cHfBF6w1ES5AOWeMjEHBTpBvSBA4rjaIqDnzkLRCpZTHNCZkiXe83xBzHj74StOXaR1Qzey/w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=H1P46Odl; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="H1P46Odl" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E3C10C4CEED; Wed, 19 Feb 2025 08:32:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739953972; bh=yvf7sytYLZ4Yo2xKDIwz8bS24pWVxbZ/2mOY1uefT38=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=H1P46OdldTj+xPr9+6Os6i9blKx/Kq1f3xm8D6hiXLxAecHgY+vuumVmhTBwzxJhc JV8XpFDkxAEyAVdQFd0wozw5FNg3ii2udEiuu9wJX2YKmasGoIzFkxgOhN7GzAHYUW 8AuscBWfZfK6Do8Ro85rhSBSNWdlIuRs6orhmxfvWAVuXN8b2XyFfdKH090NvBlRLe aEkod/dnj3ASH0NYZ9z1Dv57otJKBT6oRouQ83yCS2Y5Rtyd53nuj6J46FV9hSPMUe J1OlBtzI9eIs+Crh/N+dGuM0piyBqHtAXl+3leRs7/ye0YRkxhxPI7YsliyzdcdkF8 8GdL2uFeXL7Ew== Received: from mchehab by mail.kernel.org with local (Exim 4.98) (envelope-from ) id 1tkfVi-0000000Gv4b-0FWK; Wed, 19 Feb 2025 09:32:50 +0100 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , "Mauro Carvalho Chehab" , linux-kernel@vger.kernel.org Subject: [PATCH 05/27] scripts/kernel-doc: don't add not needed new lines Date: Wed, 19 Feb 2025 09:32:21 +0100 Message-ID: X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" This helps comparing kernel-doc output with the new .py version of it. Signed-off-by: Mauro Carvalho Chehab --- scripts/kernel-doc | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/scripts/kernel-doc b/scripts/kernel-doc index 2c77b914d017..d59552e1a31d 100755 --- a/scripts/kernel-doc +++ b/scripts/kernel-doc @@ -760,6 +760,10 @@ sub output_highlight_rst { if ($block) { $output .=3D highlight_block($block); } + + $output =3D~ s/^\n+//g; + $output =3D~ s/\n+$//g; + foreach $line (split "\n", $output) { print $lineprefix . $line . "\n"; } --=20 2.48.1 From nobody Sat Feb 7 14:01:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5D3FE1CEAC3; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; cv=none; b=mQbUjMxZAmYw7sQ70mMv6gwWHpD9L0YsTF8h06jPr5lR1IZjaHZD6Y7WdMPU51I3gZyKOOqR77WKMwqOGHTzGBme1J9tjsCmU9Z5M2WJJIA5sbaVowZEHFXAEcgcmpMK5508/CRJN7dHdovGudaPwy7ajRFRZ7qIuw2pXqN+sqA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; c=relaxed/simple; bh=o+1xc6xKU/lADwcvw9syISNuAVlcBEcy5nbY0N3w4kc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=m3ePcNYcy9AXhnM+PvXEQEbwuSBo6MCfiKBu7+4e3vuZgNzvgmVNWE3nKJZ6cweXKZDdPaxqzuN2iyFbfV5fg5wD6jM5B34PVWQxujB7SlGKEyx/ut4bDSAwW8HeJtWslckJ5+43pXYOvgP7I+f0rJGTzBKJrQ/39zpbaecRc4w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=BW87f3rO; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="BW87f3rO" Received: by smtp.kernel.org (Postfix) with ESMTPSA id EE73DC4CEEC; Wed, 19 Feb 2025 08:32:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739953972; bh=o+1xc6xKU/lADwcvw9syISNuAVlcBEcy5nbY0N3w4kc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BW87f3rOoOH+5hXco+hAjc/XgauazXzSPXWPdYnovT3YGvtx/gXgB8S5H5dGnAbU7 Y5Hi60/+8kjez3kfnd87VWbbJNOpiZ2EkZrawW/BEHbO0pWG60t3O9ifPb+gtDmZ1D xAz4godOY6VqhLL6XT8va15kXSkUyO3qhmaIt6fMAlG4Pohc4gSDUFvpsV9EgC7eb0 P6+v6bNQtjM20nkKFLT9qcMyXaxDU8uUqYbMKQg0kUkpHsRU26Gi6lG8t03/mK7vDi 4LuiLKinCID3oNr33BSTuXIKT53CB5GCNgS8hGgkqed+AkNoxM8zqSM3oTISSIJgL7 r1WnfC/wIMFqg== Received: from mchehab by mail.kernel.org with local (Exim 4.98) (envelope-from ) id 1tkfVi-0000000Gv4f-0M0d; Wed, 19 Feb 2025 09:32:50 +0100 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , "Mauro Carvalho Chehab" , linux-kernel@vger.kernel.org Subject: [PATCH 06/27] scripts/kernel-doc: drop dead code for Wcontents_before_sections Date: Wed, 19 Feb 2025 09:32:22 +0100 Message-ID: X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" There is a warning about contents before sections, which doesn't work, since in_doc_sect variable is always true at the point it is checked. Drop the dead code. Signed-off-by: Mauro Carvalho Chehab --- scripts/kernel-doc | 11 ----------- 1 file changed, 11 deletions(-) diff --git a/scripts/kernel-doc b/scripts/kernel-doc index d59552e1a31d..af6cf408b96d 100755 --- a/scripts/kernel-doc +++ b/scripts/kernel-doc @@ -137,7 +137,6 @@ my $verbose =3D 0; my $Werror =3D 0; my $Wreturn =3D 0; my $Wshort_desc =3D 0; -my $Wcontents_before_sections =3D 0; my $output_mode =3D "rst"; my $output_preformatted =3D 0; my $no_doc_sections =3D 0; @@ -223,7 +222,6 @@ use constant { STATE_INLINE =3D> 7, # gathering doc outside main block }; my $state; -my $in_doc_sect; my $leading_space; =20 # Inline documentation state @@ -332,12 +330,9 @@ while ($ARGV[0] =3D~ m/^--?(.*)/) { $Wreturn =3D 1; } elsif ($cmd eq "Wshort-desc" or $cmd eq "Wshort-description") { $Wshort_desc =3D 1; - } elsif ($cmd eq "Wcontents-before-sections") { - $Wcontents_before_sections =3D 1; } elsif ($cmd eq "Wall") { $Wreturn =3D 1; $Wshort_desc =3D 1; - $Wcontents_before_sections =3D 1; } elsif (($cmd eq "h") || ($cmd eq "help")) { pod2usage(-exitval =3D> 0, -verbose =3D> 2); } elsif ($cmd eq 'no-doc-sections') { @@ -1963,7 +1958,6 @@ sub process_export_file($) { sub process_normal() { if (/$doc_start/o) { $state =3D STATE_NAME; # next line is always the function n= ame - $in_doc_sect =3D 0; $declaration_start_line =3D $. + 1; } } @@ -2068,7 +2062,6 @@ sub process_body($$) { } =20 if (/$doc_sect/i) { # case insensitive for supported section names - $in_doc_sect =3D 1; $newsection =3D $1; $newcontents =3D $2; =20 @@ -2085,14 +2078,10 @@ sub process_body($$) { } =20 if (($contents ne "") && ($contents ne "\n")) { - if (!$in_doc_sect && $Wcontents_before_sections) { - emit_warning("${file}:$.", "contents before sections\n"); - } dump_section($file, $section, $contents); $section =3D $section_default; } =20 - $in_doc_sect =3D 1; $state =3D STATE_BODY; $contents =3D $newcontents; $new_start_line =3D $.; --=20 2.48.1 From nobody Sat Feb 7 14:01:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6325B1D14FF; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; cv=none; b=tYzshy14wTrNwDVqZknj4wpvP0HmhmR8upRXerPF2kbWJx1liclYNSi2toZG60CHr5jwC4pipiSEwU3l6w4/uG16smnnk/RxSuRyPRFIF38sryv7PPIEZdAxP8NR3e2zC2ymPM6+2/Oc3MDfeds3YX7+MWkeyl7VBAYZp6tGPz8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; c=relaxed/simple; bh=pT2AurFk7T5HHxyb+zGbm0v00i4Pdq52KmG0zBAistg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Fxn/zPcFxhCEjaFX8C2Ga+TH24vFXHgSQSaeh/ICyek8HJx7kPlGoJN2YKynw6LxZHQ24eLT0+7Au37YVoueaO7yRqQV13e4SGU0N5rc727Qu/m8DG1/H14elF0tkukMX76bI88ve98oEJtI//Kr5AKivcMYhBJC46bcswnV5q8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Y2F9GMBc; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Y2F9GMBc" Received: by smtp.kernel.org (Postfix) with ESMTPSA id F2A69C4CEF1; Wed, 19 Feb 2025 08:32:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739953972; bh=pT2AurFk7T5HHxyb+zGbm0v00i4Pdq52KmG0zBAistg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Y2F9GMBcCheO8BiQlvKnIJ1rvxtjLAHCEGJnVAst4BwdM3+xx6Ts2jo0OT6YfdKxm AwYTjMhTmODYeb1h6oS23wXDaYyB81be7ZPivtSnz3iFFf54qS5Q9e3SdxKQLUcu4x xDFoj8x7sonE9xKkvFvwPCKlOXafI0iNE86BB2bkXs/dsDCBMxUy/0rcw7fy6J2Byz EfqK9zLuXlVdDtihhhBNDOBCg/czYHcaVLfPdP/a5NzPbS/JZJLR9mCm2Z9NcXEEdl pwIfxpw/qDqMOGGtzvScRClVzK6TQFqPGhCGCD8v8FPfDhTMeXaMiBGDuPpAMRv+FQ BBHLIZRtAFIGQ== Received: from mchehab by mail.kernel.org with local (Exim 4.98) (envelope-from ) id 1tkfVi-0000000Gv4j-0SY5; Wed, 19 Feb 2025 09:32:50 +0100 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , "Mauro Carvalho Chehab" , linux-kernel@vger.kernel.org Subject: [PATCH 07/27] scripts/kernel-doc: rename it to scripts/kernel-doc.pl Date: Wed, 19 Feb 2025 09:32:23 +0100 Message-ID: X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" In preparation for deprecating scripts/kernel-doc in favor of a new version written in Perl, rename it to scripts/kernel-doc.pl. Signed-off-by: Mauro Carvalho Chehab --- scripts/{kernel-doc =3D> kernel-doc.pl} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename scripts/{kernel-doc =3D> kernel-doc.pl} (100%) diff --git a/scripts/kernel-doc b/scripts/kernel-doc.pl similarity index 100% rename from scripts/kernel-doc rename to scripts/kernel-doc.pl --=20 2.48.1 From nobody Sat Feb 7 14:01:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A554C1DB92E; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; cv=none; b=Dj84q07Rcft3jXZHmjx8E7QKPPDLw9cua9HlL0BK0Ai6uRVKm30B8FeOQgvlxVvajkRpQ4DwZFV4ywmZVrcpSc6xBnSYdSMpGMpCOoXZkrXkCu7eN4Cl0KW9XPuJ4BpaQpP3HxlpqaRCtyGd/T4vZUhgwJotdpdJhCBqj+gt8Hk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; c=relaxed/simple; bh=ojXYm2ybn18UU1+Rz9pniQpO3g763W9s/DzLh/PxuxA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=I7Is3Nei/V7whNXEGlLDtg0u04u/SsUZyG/gI4b3wtMYo1NVyQxlu5tknChSW1TUNbd1a5nSNc3gLKQaw7JZRIDq9RZr8cENyyEDlBEGNbXde64E2LQR08VktfuarTkOFhVhXP2Nk9MH7s2jgbw66zbCswf6Zu4PIe9qYdkW8aI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=pKw86eO6; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="pKw86eO6" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2F737C4CEFF; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739953972; bh=ojXYm2ybn18UU1+Rz9pniQpO3g763W9s/DzLh/PxuxA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pKw86eO60YukQMLJ7wic2bN+mODdww9MFBlG8Yg8Nw9PkaYt92lGwLikxI8xzRtpI 4EjkwzS6+Li2GZC6nh7zI7R1ZE1Ozo2gfv1B+F1ZLpyybotFhae/VqkedE3ct7h3BK E4GCcdhvOyuqHVDAO4sO3oR3J/oAU9LVd92x9pURtt24ngBSJWaEWL9dl6+U9yShJR ZkrhL04HrmPbiNmAZZvwmGHPHEo0fUNQbpcPMYhlLhl8SR+JTN1B/qyrdqYgMaSPjx uToyWuNoCXAt5v+M0obdkmUc8hUvHz3LPzwp4nSOYYAh6UsLASkSodISURI8CUqnzF EN5cSE888CJYA== Received: from mchehab by mail.kernel.org with local (Exim 4.98) (envelope-from ) id 1tkfVi-0000000Gv4n-0ZdU; Wed, 19 Feb 2025 09:32:50 +0100 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , "Mauro Carvalho Chehab" , linux-kernel@vger.kernel.org Subject: [PATCH 08/27] scripts/kernel-doc: add a symlink to the Perl version of kernel-doc Date: Wed, 19 Feb 2025 09:32:24 +0100 Message-ID: <21857431e7cb0b53e21259e7dd999ce50920e0d3.1739952783.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" Preserve kernel-doc name, associating with the curent version in Perl. Signed-off-by: Mauro Carvalho Chehab --- scripts/kernel-doc | 1 + 1 file changed, 1 insertion(+) create mode 120000 scripts/kernel-doc diff --git a/scripts/kernel-doc b/scripts/kernel-doc new file mode 120000 index 000000000000..f175155c1e66 --- /dev/null +++ b/scripts/kernel-doc @@ -0,0 +1 @@ +kernel-doc.pl \ No newline at end of file --=20 2.48.1 From nobody Sat Feb 7 14:01:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 03A5F1DE3AF; Wed, 19 Feb 2025 08:32:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953973; cv=none; b=TpabDgh5a7mZPdk63ARWFqjzAbrxfQXISrn7ylX7jIAEWycKC87fxgqXlWYg0Dwxy1L3itoEVr8CszE/vW5GM24nvSaDojOZ/jgDoY1vuZ9pk17jSmGCCxhTHNKuLcjeyYMOQlorvlQfXg3I2Mw4QieW7ShKJaehu583oI/1RSs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953973; c=relaxed/simple; bh=Pl6oSKiWCpoo2+dE9Fw6KFNver/u1CwO5ptehFE4p+w=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ocOxYBVfzRYrEcKIMteLV85Ei41gGtg6bBz1y1UDEeY9xDdV3qv175Yz9nDjPP+6rLdALK0+6hvTFe+dW4iXkEa9rar725w3tPc8tT0PicN4AARTNIowpdDiOSZQnYcPIqDPsx/6ao6j8BGjOkdMgSbCz+s8VoGBLU/Zj6V6lGw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=QfUejCC6; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="QfUejCC6" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 27C23C4CEF9; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739953972; bh=Pl6oSKiWCpoo2+dE9Fw6KFNver/u1CwO5ptehFE4p+w=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QfUejCC6JP1Alci9NMkSzNjLjB4SjPbEjmBFQPsWpE4VQ1dYs/vSKbB/J5ISoBPlt nPSE44TgvTdAM7IYbmNbOJX8itYEoZr74WJZ1Sq6IIC1DSveSIKgxXsxpp5XkVy8ro aTWR//pT3WAVJYxlZzI611dnbYyE1XYqgftWekK1qE8ojNCE1fcRYgnOf6yXhGSsRI NWGgJFDmi0fAmSQ/lIlZlvB2c78Cix4tD0Ne6Cwkb8uH/sjs2pfQVr/Mt7A3dTulzE DY7ErBXkfs1YwnRUpeqictIAccp3bPSt88ooAqTzNxITO6nEgSy2NNvPFkrPUwIx+J 0eZUAjTs0g2Vg== Received: from mchehab by mail.kernel.org with local (Exim 4.98) (envelope-from ) id 1tkfVi-0000000Gv4s-0hW5; Wed, 19 Feb 2025 09:32:50 +0100 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , "Gustavo A. R. Silva" , "Mauro Carvalho Chehab" , Kees Cook , linux-hardening@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 09/27] scripts/kernel-doc.py: add a Python parser Date: Wed, 19 Feb 2025 09:32:25 +0100 Message-ID: X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" Maintaining kernel-doc has been a challenge, as there aren't many perl developers among maintainers. Also, the logic there is too complex. Having lots of global variables and using pure functions doesn't help. Rewrite the script in Python, placing most global variables inside classes. This should help maintaining the script in long term. It also allows a better integration with kernel-doc Sphinx extension in the future. I opted to keep this version as close as possible to what we have already in Perl. There are some differences though: 1. There is one regular expression that required a rewrite: /\bSTRUCT_GROUP(\(((?:(?>[^)(]+)|(?1))*)\))[^;]*;/ As this one uses two features that aren't available by the native Python regular expression module (re): - recursive patterns: ?1 - atomic grouping (?>...) Rewrite it to use a much simpler regular expression: /\bSTRUCT_GROUP\(([^\)]+)\)[^;]*;/ Extra care should be taken when validating this script, as such replacement might cause some regressions. 2. The filters are now applied only during output generation. In particular, "nosymbol" argument is only handled there. It means that, if the same file is processed twice for different symbols, the warnings will be duplicated. I opted to use this behavior as it allows the Sphinx extension to read the file(s) only once, and apply the filtering only when producing the ReST output. This hopefully will help to speed up doc generation 3. This version can handle multiple files and multiple directories. So, if one just wants to produce a big output with everything inside a file, this could be done with $ time ./scripts/kernel-doc.py -man . 2>/dev/null >new real 0m54.592s user 0m53.345s sys 0m0.997s 4. I tried to replicate as much as possible the same arguments from kernel-doc, with about the same behavior, for the command line parameters starting with a single dash (-parameter). I also added one letter aliases for each parameter, and a --parameter (sometimes with a better name). 5. There are some sutile nuances between how Perl handles certain regular expressions. In special, the qr operatior, which compiles a regular expression also works as a non-capturing group. It means that some regexes like this one: my $type1 =3D qr{[\w\s]+}; needs to be mapped as: type1 =3D r'(?:[\w\s]+)?' Signed-off-by: Mauro Carvalho Chehab --- TODO: - on this RFC, the man output doesn't match yet the same output of kernel-doc. The ReST output matches, except for some whitespaces and suppressed empty sectionsl - this version lacks support for -W parameters: it will just output all warnings. - all classes are at the same file. I want to split the classes on multiple files for the final version, but, during development time, it is easier to have everything on a single file, but I plan to split classes on different files to help maintaining the script. Signed-off-by: Mauro Carvalho Chehab --- scripts/kernel-doc.py | 2757 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 2757 insertions(+) create mode 100755 scripts/kernel-doc.py diff --git a/scripts/kernel-doc.py b/scripts/kernel-doc.py new file mode 100755 index 000000000000..5cf5ed63f215 --- /dev/null +++ b/scripts/kernel-doc.py @@ -0,0 +1,2757 @@ +#!/usr/bin/env python3 +# pylint: disable=3DR0902,R0903,R0904,R0911,R0912,R0913,R0914,R0915,R0917,= R1702 +# pylint: disable=3DC0302,C0103,C0301 +# pylint: disable=3DC0116,C0115,W0511,W0613 +# Copyright(c) 2025: Mauro Carvalho Chehab . +# SPDX-License-Identifier: GPL-2.0 + +# TODO: implement warning filtering + +""" +kernel_doc +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Print formatted kernel documentation to stdout + +Read C language source or header FILEs, extract embedded +documentation comments, and print formatted documentation +to standard output. + +The documentation comments are identified by the "/**" +opening comment mark. + +See Documentation/doc-guide/kernel-doc.rst for the +documentation comment syntax. +""" + +import argparse +import logging +import os +import re +import sys + +from datetime import datetime +from pprint import pformat + +from dateutil import tz + +# Local cache for regular expressions +re_cache =3D {} + + +class Re: + """ + Helper class to simplify regex declaration and usage, + + It calls re.compile for a given pattern. It also allows adding + regular expressions and define sub at class init time. + + Regular expressions can be cached via an argument, helping to speedup + searches. + """ + + def _add_regex(self, string, flags): + if string in re_cache: + self.regex =3D re_cache[string] + else: + self.regex =3D re.compile(string, flags=3Dflags) + + if self.cache: + re_cache[string] =3D self.regex + + def __init__(self, string, cache=3DTrue, flags=3D0): + self.cache =3D cache + self.last_match =3D None + + self._add_regex(string, flags) + + def __str__(self): + return self.regex.pattern + + def __add__(self, other): + return Re(str(self) + str(other), cache=3Dself.cache or other.cach= e, + flags=3Dself.regex.flags | other.regex.flags) + + def match(self, string): + self.last_match =3D self.regex.match(string) + return self.last_match + + def search(self, string): + self.last_match =3D self.regex.search(string) + return self.last_match + + def findall(self, string): + return self.regex.findall(string) + + def split(self, string): + return self.regex.split(string) + + def sub(self, sub, string, count=3D0): + return self.regex.sub(sub, string, count=3Dcount) + + def group(self, num): + return self.last_match.group(num) + +# +# Regular expressions used to parse kernel-doc markups at KernelDoc class. +# +# Let's declare them in lowercase outside any class to make easier to +# convert from the python script. +# +# As those are evaluated at the beginning, no need to cache them +# + + +# Allow whitespace at end of comment start. +doc_start =3D Re(r'^/\*\*\s*$', cache=3DFalse) + +doc_end =3D Re(r'\*/', cache=3DFalse) +doc_com =3D Re(r'\s*\*\s*', cache=3DFalse) +doc_com_body =3D Re(r'\s*\* ?', cache=3DFalse) +doc_decl =3D doc_com + Re(r'(\w+)', cache=3DFalse) + +# @params and a strictly limited set of supported section names +# Specifically: +# Match @word: +# @...: +# @{section-name}: +# while trying to not match literal block starts like "example::" +# +doc_sect =3D doc_com + \ + Re(r'\s*(\@[.\w]+|\@\.\.\.|description|context|returns?|notes?= |examples?)\s*:([^:].*)?$', + flags=3Dre.I, cache=3DFalse) + +doc_content =3D doc_com_body + Re(r'(.*)', cache=3DFalse) +doc_block =3D doc_com + Re(r'DOC:\s*(.*)?', cache=3DFalse) +doc_inline_start =3D Re(r'^\s*/\*\*\s*$', cache=3DFalse) +doc_inline_sect =3D Re(r'\s*\*\s*(@\s*[\w][\w\.]*\s*):(.*)', cache=3DFalse) +doc_inline_end =3D Re(r'^\s*\*/\s*$', cache=3DFalse) +doc_inline_oneline =3D Re(r'^\s*/\*\*\s*(@[\w\s]+):\s*(.*)\s*\*/\s*$', cac= he=3DFalse) +function_pointer =3D Re(r"([^\(]*\(\*)\s*\)\s*\(([^\)]*)\)", cache=3DFalse) +attribute =3D Re(r"__attribute__\s*\(\([a-z0-9,_\*\s\(\)]*\)\)", + flags=3Dre.I | re.S, cache=3DFalse) + +# match expressions used to find embedded type information +type_constant =3D Re(r"\b``([^\`]+)``\b", cache=3DFalse) +type_constant2 =3D Re(r"\%([-_*\w]+)", cache=3DFalse) +type_func =3D Re(r"(\w+)\(\)", cache=3DFalse) +type_param =3D Re(r"\@(\w*((\.\w+)|(->\w+))*(\.\.\.)?)", cache=3DFalse) +type_param_ref =3D Re(r"([\!~\*]?)\@(\w*((\.\w+)|(->\w+))*(\.\.\.)?)", cac= he=3DFalse) + +# Special RST handling for func ptr params +type_fp_param =3D Re(r"\@(\w+)\(\)", cache=3DFalse) + +# Special RST handling for structs with func ptr params +type_fp_param2 =3D Re(r"\@(\w+->\S+)\(\)", cache=3DFalse) + +type_env =3D Re(r"(\$\w+)", cache=3DFalse) +type_enum =3D Re(r"\&(enum\s*([_\w]+))", cache=3DFalse) +type_struct =3D Re(r"\&(struct\s*([_\w]+))", cache=3DFalse) +type_typedef =3D Re(r"\&(typedef\s*([_\w]+))", cache=3DFalse) +type_union =3D Re(r"\&(union\s*([_\w]+))", cache=3DFalse) +type_member =3D Re(r"\&([_\w]+)(\.|->)([_\w]+)", cache=3DFalse) +type_fallback =3D Re(r"\&([_\w]+)", cache=3DFalse) +type_member_func =3D type_member + Re(r"\(\)", cache=3DFalse) + +export_symbol =3D Re(r'^\s*EXPORT_SYMBOL(_GPL)?\s*\(\s*(\w+)\s*\)\s*', cac= he=3DFalse) +export_symbol_ns =3D Re(r'^\s*EXPORT_SYMBOL_NS(_GPL)?\s*\(\s*(\w+)\s*,\s*"= \S+"\)\s*', cache=3DFalse) + +class KernelDoc: + # Parser states + STATE_NORMAL =3D 0 # normal code + STATE_NAME =3D 1 # looking for function name + STATE_BODY_MAYBE =3D 2 # body - or maybe more description + STATE_BODY =3D 3 # the body of the comment + STATE_BODY_WITH_BLANK_LINE =3D 4 # the body which has a blank line + STATE_PROTO =3D 5 # scanning prototype + STATE_DOCBLOCK =3D 6 # documentation block + STATE_INLINE =3D 7 # gathering doc outside main block + + st_name =3D [ + "NORMAL", + "NAME", + "BODY_MAYBE", + "BODY", + "BODY_WITH_BLANK_LINE", + "PROTO", + "DOCBLOCK", + "INLINE", + ] + + # Inline documentation state + STATE_INLINE_NA =3D 0 # not applicable ($state !=3D STATE_INLINE) + STATE_INLINE_NAME =3D 1 # looking for member name (@foo:) + STATE_INLINE_TEXT =3D 2 # looking for member documentation + STATE_INLINE_END =3D 3 # done + STATE_INLINE_ERROR =3D 4 # error - Comment without header was found. + # Spit a warning as it's not + # proper kernel-doc and ignore the rest. + + st_inline_name =3D [ + "", + "_NAME", + "_TEXT", + "_END", + "_ERROR", + ] + + # Section names + + section_default =3D "Description" # default section + section_intro =3D "Introduction" + section_context =3D "Context" + section_return =3D "Return" + + undescribed =3D "-- undescribed --" + + def __init__(self, config, fname): + """Initialize internal variables""" + + self.fname =3D fname + self.config =3D config + + # Initial state for the state machines + self.state =3D self.STATE_NORMAL + self.inline_doc_state =3D self.STATE_INLINE_NA + + # Store entry currently being processed + self.entry =3D None + + # Place all potential outputs into an array + self.entries =3D [] + + def show_warnings(self, dtype, declaration_name): + # TODO: implement it + + return True + + # TODO: rename to emit_message + def emit_warning(self, ln, msg, warning=3DTrue): + """Emit a message""" + + if warning: + self.config.log.warning("%s:%d %s", self.fname, ln, msg) + else: + self.config.log.info("%s:%d %s", self.fname, ln, msg) + + def dump_section(self, start_new=3DTrue): + """ + Dumps section contents to arrays/hashes intended for that purpose. + """ + + name =3D self.entry.section + contents =3D self.entry.contents + + if type_param.match(name): + name =3D type_param.group(1) + + self.entry.parameterdescs[name] =3D contents + self.entry.parameterdesc_start_lines[name] =3D self.entry.new_= start_line + + self.entry.sectcheck +=3D name + " " + self.entry.new_start_line =3D 0 + + elif name =3D=3D "@...": + name =3D "..." + self.entry.parameterdescs[name] =3D contents + self.entry.sectcheck +=3D name + " " + self.entry.parameterdesc_start_lines[name] =3D self.entry.new_= start_line + self.entry.new_start_line =3D 0 + + else: + if name in self.entry.sections and self.entry.sections[name] != =3D "": + # Only warn on user-specified duplicate section names + if name !=3D self.section_default: + self.emit_warning(self.entry.new_start_line, + f"duplicate section name '{name}'\n") + self.entry.sections[name] +=3D contents + else: + self.entry.sections[name] =3D contents + self.entry.sectionlist.append(name) + self.entry.section_start_lines[name] =3D self.entry.new_st= art_line + self.entry.new_start_line =3D 0 + +# self.config.log.debug("Section: %s : %s", name, pformat(vars(self= .entry))) + + if start_new: + self.entry.section =3D self.section_default + self.entry.contents =3D "" + + # TODO: rename it to store_declaration + def output_declaration(self, dtype, name, **args): + """ + Stores the entry into an entry array. + + The actual output and output filters will be handled elsewhere + """ + + # The implementation here is different than the original kernel-do= c: + # instead of checking for output filters or actually output anythi= ng, + # it just stores the declaration content at self.entries, as the + # output will happen on a separate class. + # + # For now, we're keeping the same name of the function just to make + # easier to compare the source code of both scripts + + if "declaration_start_line" not in args: + args["declaration_start_line"] =3D self.entry.declaration_star= t_line + + args["type"] =3D dtype + + self.entries.append((name, args)) + + self.config.log.debug("Output: %s:%s =3D %s", dtype, name, pformat= (args)) + + def reset_state(self, ln): + """ + Ancillary routine to create a new entry. It initializes all + variables used by the state machine. + """ + + self.entry =3D argparse.Namespace + + self.entry.contents =3D "" + self.entry.function =3D "" + self.entry.sectcheck =3D "" + self.entry.struct_actual =3D "" + self.entry.prototype =3D "" + + self.entry.parameterlist =3D [] + self.entry.parameterdescs =3D {} + self.entry.parametertypes =3D {} + self.entry.parameterdesc_start_lines =3D {} + + self.entry.section_start_lines =3D {} + self.entry.sectionlist =3D [] + self.entry.sections =3D {} + + self.entry.anon_struct_union =3D False + + self.entry.leading_space =3D None + + # State flags + self.state =3D self.STATE_NORMAL + self.inline_doc_state =3D self.STATE_INLINE_NA + self.entry.brcount =3D 0 + + self.entry.in_doc_sect =3D False + self.entry.declaration_start_line =3D ln + + def push_parameter(self, ln, decl_type, param, dtype, + org_arg, declaration_name): + if self.entry.anon_struct_union and dtype =3D=3D "" and param =3D= =3D "}": + return # Ignore the ending }; from anonymous struct/union + + self.entry.anon_struct_union =3D False + + param =3D Re(r'[\[\)].*').sub('', param, count=3D1) + + if dtype =3D=3D "" and param.endswith("..."): + if Re(r'\w\.\.\.$').search(param): + # For named variable parameters of the form `x...`, + # remove the dots + param =3D param[:-3] + else: + # Handles unnamed variable parameters + param =3D "..." + + if param not in self.entry.parameterdescs or \ + not self.entry.parameterdescs[param]: + + self.entry.parameterdescs[param] =3D "variable arguments" + + elif dtype =3D=3D "" and (not param or param =3D=3D "void"): + param =3D "void" + self.entry.parameterdescs[param] =3D "no arguments" + + elif dtype =3D=3D "" and param in ["struct", "union"]: + # Handle unnamed (anonymous) union or struct + dtype =3D param + param =3D "{unnamed_" + param + "}" + self.entry.parameterdescs[param] =3D "anonymous\n" + self.entry.anon_struct_union =3D True + + # Handle cache group enforcing variables: they do not need + # to be described in header files + elif "__cacheline_group" in param: + # Ignore __cacheline_group_begin and __cacheline_group_end + return + + # Warn if parameter has no description + # (but ignore ones starting with # as these are not parameters + # but inline preprocessor statements) + if param not in self.entry.parameterdescs and not param.startswith= ("#"): + self.entry.parameterdescs[param] =3D self.undescribed + + if self.show_warnings(dtype, declaration_name) and "." not in = param: + if decl_type =3D=3D 'function': + dname =3D f"{decl_type} parameter" + else: + dname =3D f"{decl_type} member" + + self.emit_warning(ln, + f"{dname} '{param}' not described in '{d= eclaration_name}'") + + # Strip spaces from param so that it is one continuous string on + # parameterlist. This fixes a problem where check_sections() + # cannot find a parameter like "addr[6 + 2]" because it actually + # appears as "addr[6", "+", "2]" on the parameter list. + # However, it's better to maintain the param string unchanged for + # output, so just weaken the string compare in check_sections() + # to ignore "[blah" in a parameter string. + + self.entry.parameterlist.append(param) + org_arg =3D Re(r'\s\s+').sub(' ', org_arg, count=3D1) + self.entry.parametertypes[param] =3D org_arg + + def save_struct_actual(self, actual): + """ + Strip all spaces from the actual param so that it looks like + one string item. + """ + + actual =3D Re(r'\s*').sub("", actual, count=3D1) + + self.entry.struct_actual +=3D actual + " " + + def create_parameter_list(self, ln, decl_type, args, splitter, declara= tion_name): + + # temporarily replace all commas inside function pointer definition + arg_expr =3D Re(r'(\([^\),]+),') + while arg_expr.search(args): + args =3D arg_expr.sub(r"\1#", args) + + for arg in args.split(splitter): + # Strip comments + arg =3D Re(r'\/\*.*\*\/').sub('', arg) + + # Ignore argument attributes + arg =3D Re(r'\sPOS0?\s').sub(' ', arg) + + # Strip leading/trailing spaces + arg =3D arg.strip() + arg =3D Re(r'\s+').sub(' ', arg, count=3D1) + + if arg.startswith('#'): + # Treat preprocessor directive as a typeless variable just= to fill + # corresponding data structures "correctly". Catch it late= r in + # output_* subs. + + # Treat preprocessor directive as a typeless variable + self.push_parameter(ln, decl_type, arg, "", + "", declaration_name) + + elif Re(r'\(.+\)\s*\(').search(arg): + # Pointer-to-function + + arg =3D arg.replace('#', ',') + + r =3D Re(r'[^\(]+\(\*?\s*([\w\[\]\.]*)\s*\)') + if r.match(arg): + param =3D r.group(1) + else: + self.emit_warning(ln, f"Invalid param: {arg}") + param =3D arg + + dtype =3D Re(r'([^\(]+\(\*?)\s*' + re.escape(param)).sub(r= '\1', arg) + self.save_struct_actual(param) + self.push_parameter(ln, decl_type, param, dtype, + arg, declaration_name) + + elif Re(r'\(.+\)\s*\[').search(arg): + # Array-of-pointers + + arg =3D arg.replace('#', ',') + r =3D Re(r'[^\(]+\(\s*\*\s*([\w\[\]\.]*?)\s*(\s*\[\s*[\w]+= \s*\]\s*)*\)') + if r.match(arg): + param =3D r.group(1) + else: + self.emit_warning(ln, f"Invalid param: {arg}") + param =3D arg + + dtype =3D Re(r'([^\(]+\(\*?)\s*' + re.escape(param)).sub(r= '\1', arg) + + self.save_struct_actual(param) + self.push_parameter(ln, decl_type, param, dtype, + arg, declaration_name) + + elif arg: + arg =3D Re(r'\s*:\s*').sub(":", arg) + arg =3D Re(r'\s*\[').sub('[', arg) + + args =3D Re(r'\s*,\s*').split(arg) + if args[0] and '*' in args[0]: + args[0] =3D re.sub(r'(\*+)\s*', r' \1', args[0]) + + first_arg =3D [] + r =3D Re(r'^(.*\s+)(.*?\[.*\].*)$') + if args[0] and r.match(args[0]): + args.pop(0) + first_arg.extend(r.group(1)) + first_arg.append(r.group(2)) + else: + first_arg =3D Re(r'\s+').split(args.pop(0)) + + args.insert(0, first_arg.pop()) + dtype =3D ' '.join(first_arg) + + for param in args: + if Re(r'^(\*+)\s*(.*)').match(param): + r =3D Re(r'^(\*+)\s*(.*)') + if not r.match(param): + self.emit_warning(ln, f"Invalid param: {param}= ") + continue + + param =3D r.group(1) + + self.save_struct_actual(r.group(2)) + self.push_parameter(ln, decl_type, r.group(2), + f"{dtype} {r.group(1)}", + arg, declaration_name) + + elif Re(r'(.*?):(\w+)').search(param): + r =3D Re(r'(.*?):(\w+)') + if not r.match(param): + self.emit_warning(ln, f"Invalid param: {param}= ") + continue + + if dtype !=3D "": # Skip unnamed bit-fields + self.save_struct_actual(r.group(1)) + self.push_parameter(ln, decl_type, r.group(1), + f"{dtype}:{r.group(2)}", + arg, declaration_name) + else: + self.save_struct_actual(param) + self.push_parameter(ln, decl_type, param, dtype, + arg, declaration_name) + + def check_sections(self, ln, decl_name, decl_type, sectcheck, prmschec= k): + sects =3D sectcheck.split() + prms =3D prmscheck.split() + err =3D False + + for sx in range(len(sects)): # pylint: disable=3D= C0200 + err =3D True + for px in range(len(prms)): # pylint: disable=3D= C0200 + prm_clean =3D prms[px] + prm_clean =3D Re(r'\[.*\]').sub('', prm_clean) + prm_clean =3D attribute.sub('', prm_clean) + + # ignore array size in a parameter string; + # however, the original param string may contain + # spaces, e.g.: addr[6 + 2] + # and this appears in @prms as "addr[6" since the + # parameter list is split at spaces; + # hence just ignore "[..." for the sections check; + prm_clean =3D Re(r'\[.*').sub('', prm_clean) + + if prm_clean =3D=3D sects[sx]: + err =3D False + break + + if err: + if decl_type =3D=3D 'function': + dname =3D f"{decl_type} parameter" + else: + dname =3D f"{decl_type} member" + + self.emit_warning(ln, + f"Excess {dname} '{sects[sx]}' descripti= on in '{decl_name}'") + + def check_return_section(self, ln, declaration_name, return_type): + + if not self.config.wreturn: + return + + # Ignore an empty return type (It's a macro) + # Ignore functions with a "void" return type (but not "void *") + if not return_type or Re(r'void\s*\w*\s*$').search(return_type): + return + + if not self.entry.sections.get("Return", None): + self.emit_warning(ln, + f"No description found for return value of '= {declaration_name}'") + + def dump_struct(self, ln, proto): + """ + Store an entry for an struct or union + """ + + type_pattern =3D r'(struct|union)' + + qualifiers =3D [ + "__attribute__", + "__packed", + "__aligned", + "____cacheline_aligned_in_smp", + "____cacheline_aligned", + ] + + definition_body =3D r'\{(.*)\}\s*' + "(?:" + '|'.join(qualifiers) = + ")?" + struct_members =3D Re(type_pattern + r'([^\{\};]+)(\{)([^\{\}]*)(\= })([^\{\}\;]*)(\;)') + + # Extract struct/union definition + members =3D None + declaration_name =3D None + decl_type =3D None + + r =3D Re(type_pattern + r'\s+(\w+)\s*' + definition_body) + if r.search(proto): + decl_type =3D r.group(1) + declaration_name =3D r.group(2) + members =3D r.group(3) + else: + r =3D Re(r'typedef\s+' + type_pattern + r'\s*' + definition_bo= dy + r'\s*(\w+)\s*;') + + if r.search(proto): + decl_type =3D r.group(1) + declaration_name =3D r.group(3) + members =3D r.group(2) + + if not members: + self.emit_warning(ln, f"{proto} error: Cannot parse struct or = union!") + self.config.errors +=3D 1 + return + + if self.entry.identifier !=3D declaration_name: + self.emit_warning(ln, + f"expecting prototype for {decl_type} {self.= entry.identifier}. Prototype was for {decl_type} {declaration_name} instead= \n") + return + + args_pattern =3Dr'([^,)]+)' + + sub_prefixes =3D [ + (Re(r'\/\*\s*private:.*?\/\*\s*public:.*?\*\/', re.S | re.I), = ''), + (Re(r'\/\*\s*private:.*', re.S| re.I), ''), + + # Strip comments + (Re(r'\/\*.*?\*\/', re.S), ''), + + # Strip attributes + (attribute, ' '), + (Re(r'\s*__aligned\s*\([^;]*\)', re.S), ' '), + (Re(r'\s*__counted_by\s*\([^;]*\)', re.S), ' '), + (Re(r'\s*__counted_by_(le|be)\s*\([^;]*\)', re.S), ' '), + (Re(r'\s*__packed\s*', re.S), ' '), + (Re(r'\s*CRYPTO_MINALIGN_ATTR', re.S), ' '), + (Re(r'\s*____cacheline_aligned_in_smp', re.S), ' '), + (Re(r'\s*____cacheline_aligned', re.S), ' '), + + # Unwrap struct_group() based on this definition: + # __struct_group(TAG, NAME, ATTRS, MEMBERS...) + # which has variants like: struct_group(NAME, MEMBERS...) + + (Re(r'\bstruct_group\s*\(([^,]*,)', re.S), r'STRUCT_GROUP('), + (Re(r'\bstruct_group_attr\s*\(([^,]*,){2}', re.S), r'STRUCT_G= ROUP('), + (Re(r'\bstruct_group_tagged\s*\(([^,]*),([^,]*),', re.S), r's= truct \1 \2; STRUCT_GROUP('), + (Re(r'\b__struct_group\s*\(([^,]*,){3}', re.S), r'STRUCT_GROU= P('), + + # This is incompatible with Python re, as it uses: + # recursive patterns ((?1)) and atomic grouping ((?>...)): + # '\bSTRUCT_GROUP(\(((?:(?>[^)(]+)|(?1))*)\))[^;]*;' + # Let's see if this works instead: + (Re(r'\bSTRUCT_GROUP\(([^\)]+)\)[^;]*;', re.S), r'\1'), + + # Replace macros + (Re(r'__ETHTOOL_DECLARE_LINK_MODE_MASK\s*\(([^\)]+)\)', re.S),= r'DECLARE_BITMAP(\1, __ETHTOOL_LINK_MODE_MASK_NBITS)'), + (Re(r'DECLARE_PHY_INTERFACE_MASK\s*\(([^\)]+)\)', re.S), r'DE= CLARE_BITMAP(\1, PHY_INTERFACE_MODE_MAX)'), + (Re(r'DECLARE_BITMAP\s*\(' + args_pattern + r',\s*' + args_pat= tern + r'\)', re.S), r'unsigned long \1[BITS_TO_LONGS(\2)]'), + (Re(r'DECLARE_HASHTABLE\s*\(' + args_pattern + r',\s*' + args_= pattern + r'\)', re.S), r'unsigned long \1[1 << ((\2) - 1)]'), + (Re(r'DECLARE_KFIFO\s*\(' + args_pattern + r',\s*' + args_patt= ern + r',\s*' + args_pattern + r'\)', re.S), r'\2 *\1'), + (Re(r'DECLARE_KFIFO_PTR\s*\(' + args_pattern + r',\s*' + args_= pattern + r'\)', re.S), r'\2 *\1'), + (Re(r'(?:__)?DECLARE_FLEX_ARRAY\s*\(' + args_pattern + r',\s*'= + args_pattern + r'\)', re.S), r'\1 \2[]'), + (Re(r'DEFINE_DMA_UNMAP_ADDR\s*\(' + args_pattern + r'\)', re.S= ), r'dma_addr_t \1'), + (Re(r'DEFINE_DMA_UNMAP_LEN\s*\(' + args_pattern + r'\)', re.S)= , r'__u32 \1'), + ] + + for search, sub in sub_prefixes: + members =3D search.sub(sub, members) + + # Keeps the original declaration as-is + declaration =3D members + + # Split nested struct/union elements + # + # This loop was simpler at the original kernel-doc perl version, as + # while ($members =3D~ m/$struct_members/) { ... } + # reads 'members' string on each interaction. + # + # Python behavior is different: it parses 'members' only once, + # creating a list of tuples from the first interaction. + # + # On other words, this won't get nested structs. + # + # So, we need to have an extra loop on Python to override such + # re limitation. + + while True: + tuples =3D struct_members.findall(members) + if not tuples: + break + + for t in tuples: + newmember =3D "" + maintype =3D t[0] + s_ids =3D t[5] + content =3D t[3] + + oldmember =3D "".join(t) + + for s_id in s_ids.split(','): + s_id =3D s_id.strip() + + newmember +=3D f"{maintype} {s_id}; " + s_id =3D Re(r'[:\[].*').sub('', s_id) + s_id =3D Re(r'^\s*\**(\S+)\s*').sub(r'\1', s_id) + + for arg in content.split(';'): + arg =3D arg.strip() + + if not arg: + continue + + r =3D Re(r'^([^\(]+\(\*?\s*)([\w\.]*)(\s*\).*)') + if r.match(arg): + # Pointer-to-function + dtype =3D r.group(1) + name =3D r.group(2) + extra =3D r.group(3) + + if not name: + continue + + if not s_id: + # Anonymous struct/union + newmember +=3D f"{dtype}{name}{extra}; " + else: + newmember +=3D f"{dtype}{s_id}.{name}{extr= a}; " + + else: + arg =3D arg.strip() + # Handle bitmaps + arg =3D Re(r':\s*\d+\s*').sub('', arg) + + # Handle arrays + arg =3D Re(r'\[.*\]').sub('', arg) + + # Handle multiple IDs + arg =3D Re(r'\s*,\s*').sub(',', arg) + + + r =3D Re(r'(.*)\s+([\S+,]+)') + + if r.search(arg): + dtype =3D r.group(1) + names =3D r.group(2) + else: + newmember +=3D f"{arg}; " + continue + + for name in names.split(','): + name =3D Re(r'^\s*\**(\S+)\s*').sub(r'\1',= name).strip() + + if not name: + continue + + if not s_id: + # Anonymous struct/union + newmember +=3D f"{dtype} {name}; " + else: + newmember +=3D f"{dtype} {s_id}.{name}= ; " + + members =3D members.replace(oldmember, newmember) + + # Ignore other nested elements, like enums + members =3D re.sub(r'(\{[^\{\}]*\})', '', members) + + self.create_parameter_list(ln, decl_type, members, ';', + declaration_name) + self.check_sections(ln, declaration_name, decl_type, + self.entry.sectcheck, self.entry.struct_actual) + + # Adjust declaration for better display + declaration =3D Re(r'([\{;])').sub(r'\1\n', declaration) + declaration =3D Re(r'\}\s+;').sub('};', declaration) + + # Better handle inlined enums + while True: + r =3D Re(r'(enum\s+\{[^\}]+),([^\n])') + if not r.search(declaration): + break + + declaration =3D r.sub(r'\1,\n\2', declaration) + + def_args =3D declaration.split('\n') + level =3D 1 + declaration =3D "" + for clause in def_args: + + clause =3D clause.strip() + clause =3D Re(r'\s+').sub(' ', clause, count=3D1) + + if not clause: + continue + + if '}' in clause and level > 1: + level -=3D 1 + + if not Re(r'^\s*#').match(clause): + declaration +=3D "\t" * level + + declaration +=3D "\t" + clause + "\n" + if "{" in clause and "}" not in clause: + level +=3D 1 + + self.output_declaration(decl_type, declaration_name, + struct=3Ddeclaration_name, + module=3Dself.entry.modulename, + definition=3Ddeclaration, + parameterlist=3Dself.entry.parameterlist, + parameterdescs=3Dself.entry.parameterdescs, + parametertypes=3Dself.entry.parametertypes, + sectionlist=3Dself.entry.sectionlist, + sections=3Dself.entry.sections, + purpose=3Dself.entry.declaration_purpose) + + def dump_enum(self, ln, proto): + + # Ignore members marked private + proto =3D Re(r'\/\*\s*private:.*?\/\*\s*public:.*?\*\/', flags=3Dr= e.S).sub('', proto) + proto =3D Re(r'\/\*\s*private:.*}', flags=3Dre.S).sub('}', proto) + + # Strip comments + proto =3D Re(r'\/\*.*?\*\/', flags=3Dre.S).sub('', proto) + + # Strip #define macros inside enums + proto =3D Re(r'#\s*((define|ifdef|if)\s+|endif)[^;]*;', flags=3Dre= .S).sub('', proto) + + members =3D None + declaration_name =3D None + + r =3D Re(r'typedef\s+enum\s*\{(.*)\}\s*(\w*)\s*;') + if r.search(proto): + declaration_name =3D r.group(2) + members =3D r.group(1).rstrip() + else: + r =3D Re(r'enum\s+(\w*)\s*\{(.*)\}') + if r.match(proto): + declaration_name =3D r.group(1) + members =3D r.group(2).rstrip() + + if not members: + self.emit_warning(ln, f"{proto}: error: Cannot parse enum!") + self.config.errors +=3D 1 + return + + if self.entry.identifier !=3D declaration_name: + if self.entry.identifier =3D=3D "": + self.emit_warning(ln, + f"{proto}: wrong kernel-doc identifier o= n prototype") + else: + self.emit_warning(ln, + f"expecting prototype for enum {self.ent= ry.identifier}. Prototype was for enum {declaration_name} instead") + return + + if not declaration_name: + declaration_name =3D "(anonymous)" + + member_set =3D set() + + members =3D Re(r'\([^;]*?[\)]').sub('', members) + + for arg in members.split(','): + if not arg: + continue + arg =3D Re(r'^\s*(\w+).*').sub(r'\1', arg) + self.entry.parameterlist.append(arg) + if arg not in self.entry.parameterdescs: + self.entry.parameterdescs[arg] =3D self.undescribed + if self.show_warnings("enum", declaration_name): + self.emit_warning(ln, + f"Enum value '{arg}' not described i= n enum '{declaration_name}'") + member_set.add(arg) + + for k in self.entry.parameterdescs: + if k not in member_set: + if self.show_warnings("enum", declaration_name): + self.emit_warning(ln, + f"Excess enum value '%{k}' descripti= on in '{declaration_name}'") + + self.output_declaration('enum', declaration_name, + enum=3Ddeclaration_name, + module=3Dself.config.modulename, + parameterlist=3Dself.entry.parameterlist, + parameterdescs=3Dself.entry.parameterdescs, + sectionlist=3Dself.entry.sectionlist, + sections=3Dself.entry.sections, + purpose=3Dself.entry.declaration_purpose) + + def dump_declaration(self, ln, prototype): + if self.entry.decl_type =3D=3D "enum": + self.dump_enum(ln, prototype) + return + + if self.entry.decl_type =3D=3D "typedef": + self.dump_typedef(ln, prototype) + return + + if self.entry.decl_type in ["union", "struct"]: + self.dump_struct(ln, prototype) + return + + # TODO: handle other types + self.output_declaration(self.entry.decl_type, prototype, + entry=3Dself.entry) + + def dump_function(self, ln, prototype): + + func_macro =3D False + return_type =3D '' + decl_type =3D 'function' + + # Prefixes that would be removed + sub_prefixes =3D [ + (r"^static +", "", 0), + (r"^extern +", "", 0), + (r"^asmlinkage +", "", 0), + (r"^inline +", "", 0), + (r"^__inline__ +", "", 0), + (r"^__inline +", "", 0), + (r"^__always_inline +", "", 0), + (r"^noinline +", "", 0), + (r"^__FORTIFY_INLINE +", "", 0), + (r"__init +", "", 0), + (r"__init_or_module +", "", 0), + (r"__deprecated +", "", 0), + (r"__flatten +", "", 0), + (r"__meminit +", "", 0), + (r"__must_check +", "", 0), + (r"__weak +", "", 0), + (r"__sched +", "", 0), + (r"_noprof", "", 0), + (r"__printf\s*\(\s*\d*\s*,\s*\d*\s*\) +", "", 0), + (r"__(?:re)?alloc_size\s*\(\s*\d+\s*(?:,\s*\d+\s*)?\) +", "", = 0), + (r"__diagnose_as\s*\(\s*\S+\s*(?:,\s*\d+\s*)*\) +", "", 0), + (r"DECL_BUCKET_PARAMS\s*\(\s*(\S+)\s*,\s*(\S+)\s*\)", r"\1, \2= ", 0), + (r"__attribute_const__ +", "", 0), + + # It seems that Python support for re.X is broken: + # At least for me (Python 3.13), this didn't work +# (r""" +# __attribute__\s*\(\( +# (?: +# [\w\s]+ # attribute name +# (?:\([^)]*\))? # attribute arguments +# \s*,? # optional comma at the end +# )+ +# \)\)\s+ +# """, "", re.X), + + # So, remove whitespaces and comments from it + (r"__attribute__\s*\(\((?:[\w\s]+(?:\([^)]*\))?\s*,?)+\)\)\s+"= , "", 0), + ] + + for search, sub, flags in sub_prefixes: + prototype =3D Re(search, flags).sub(sub, prototype) + + # Macros are a special case, as they change the prototype format + new_proto =3D Re(r"^#\s*define\s+").sub("", prototype) + if new_proto !=3D prototype: + is_define_proto =3D True + prototype =3D new_proto + else: + is_define_proto =3D False + + # Yes, this truly is vile. We are looking for: + # 1. Return type (may be nothing if we're looking at a macro) + # 2. Function name + # 3. Function parameters. + # + # All the while we have to watch out for function pointer paramete= rs + # (which IIRC is what the two sections are for), C types (these + # regexps don't even start to express all the possibilities), and + # so on. + # + # If you mess with these regexps, it's a good idea to check that + # the following functions' documentation still comes out right: + # - parport_register_device (function pointer parameters) + # - atomic_set (macro) + # - pci_match_device, __copy_to_user (long return type) + + name =3D r'[a-zA-Z0-9_~:]+' + prototype_end1 =3D r'[^\(]*' + prototype_end2 =3D r'[^\{]*' + prototype_end =3D fr'\(({prototype_end1}|{prototype_end2})\)' + + # Besides compiling, Perl qr{[\w\s]+} works as a non-capturing gro= up. + # So, this needs to be mapped in Python with (?:...)? or (?:...)+ + + type1 =3D r'(?:[\w\s]+)?' + type2 =3D r'(?:[\w\s]+\*+)+' + + found =3D False + + if is_define_proto: + r =3D Re(r'^()(' + name + r')\s+') + + if r.search(prototype): + return_type =3D '' + declaration_name =3D r.group(2) + func_macro =3D True + + found =3D True + + if not found: + patterns =3D [ + rf'^()({name})\s*{prototype_end}', + rf'^({type1})\s+({name})\s*{prototype_end}', + rf'^({type2})\s*({name})\s*{prototype_end}', + ] + + for p in patterns: + r =3D Re(p) + + if r.match(prototype): + + return_type =3D r.group(1) + declaration_name =3D r.group(2) + args =3D r.group(3) + + self.create_parameter_list(ln, decl_type, args, ',', + declaration_name) + + found =3D True + break + if not found: + self.emit_warning(ln, + f"cannot understand function prototype: '{pr= ototype}'") + return + + if self.entry.identifier !=3D declaration_name: + self.emit_warning(ln, + f"expecting prototype for {self.entry.identi= fier}(). Prototype was for {declaration_name}() instead") + return + + prms =3D " ".join(self.entry.parameterlist) + self.check_sections(ln, declaration_name, "function", + self.entry.sectcheck, prms) + + self.check_return_section(ln, declaration_name, return_type) + + if 'typedef' in return_type: + self.output_declaration(decl_type, declaration_name, + function=3Ddeclaration_name, + typedef=3DTrue, + module=3Dself.config.modulename, + functiontype=3Dreturn_type, + parameterlist=3Dself.entry.parameterlist, + parameterdescs=3Dself.entry.parameterdescs, + parametertypes=3Dself.entry.parametertypes, + sectionlist=3Dself.entry.sectionlist, + sections=3Dself.entry.sections, + purpose=3Dself.entry.declaration_purpose, + func_macro=3Dfunc_macro) + else: + self.output_declaration(decl_type, declaration_name, + function=3Ddeclaration_name, + typedef=3DFalse, + module=3Dself.config.modulename, + functiontype=3Dreturn_type, + parameterlist=3Dself.entry.parameterlist, + parameterdescs=3Dself.entry.parameterdescs, + parametertypes=3Dself.entry.parametertypes, + sectionlist=3Dself.entry.sectionlist, + sections=3Dself.entry.sections, + purpose=3Dself.entry.declaration_purpose, + func_macro=3Dfunc_macro) + + def dump_typedef(self, ln, proto): + typedef_type =3D r'((?:\s+[\w\*]+\b){1,8})\s*' + typedef_ident =3D r'\*?\s*(\w\S+)\s*' + typedef_args =3D r'\s*\((.*)\);' + + typedef1 =3D Re(r'typedef' + typedef_type + r'\(' + typedef_ident = + r'\)' + typedef_args) + typedef2 =3D Re(r'typedef' + typedef_type + typedef_ident + typede= f_args) + + # Strip comments + proto =3D Re(r'/\*.*?\*/', flags=3Dre.S).sub('', proto) + + # Parse function typedef prototypes + for r in [typedef1, typedef2]: + if not r.match(proto): + continue + + return_type =3D r.group(1).strip() + declaration_name =3D r.group(2) + args =3D r.group(3) + + if self.entry.identifier !=3D declaration_name: + self.emit_warning(ln, + f"expecting prototype for typedef {self.= entry.identifier}. Prototype was for typedef {declaration_name} instead\n") + return + + decl_type =3D 'function' + self.create_parameter_list(ln, decl_type, args, ',', declarati= on_name) + + self.output_declaration(decl_type, declaration_name, + function=3Ddeclaration_name, + typedef=3DTrue, + module=3Dself.entry.modulename, + functiontype=3Dreturn_type, + parameterlist=3Dself.entry.parameterlist, + parameterdescs=3Dself.entry.parameterdescs, + parametertypes=3Dself.entry.parametertypes, + sectionlist=3Dself.entry.sectionlist, + sections=3Dself.entry.sections, + purpose=3Dself.entry.declaration_purpose) + return + + # Handle nested parentheses or brackets + r =3D Re(r'(\(*.\)\s*|\[*.\]\s*);$') + while r.search(proto): + proto =3D r.sub('', proto) + + # Parse simple typedefs + r =3D Re(r'typedef.*\s+(\w+)\s*;') + if r.match(proto): + declaration_name =3D r.group(1) + + if self.entry.identifier !=3D declaration_name: + self.emit_warning(ln, f"expecting prototype for typedef {s= elf.entry.identifier}. Prototype was for typedef {declaration_name} instead= \n") + return + + self.output_declaration('typedef', declaration_name, + typedef=3Ddeclaration_name, + module=3Dself.entry.modulename, + sectionlist=3Dself.entry.sectionlist, + sections=3Dself.entry.sections, + purpose=3Dself.entry.declaration_purpose) + return + + self.emit_warning(ln, "error: Cannot parse typedef!") + self.config.errors +=3D 1 + + @staticmethod + def process_export(function_table, line): + """ + process EXPORT_SYMBOL* tags + + This method is called both internally and externally, so, it + doesn't use self. + """ + + if export_symbol.search(line): + symbol =3D export_symbol.group(2) + function_table.add(symbol) + + if export_symbol_ns.search(line): + symbol =3D export_symbol_ns.group(2) + function_table.add(symbol) + + def process_normal(self, ln, line): + """ + STATE_NORMAL: looking for the /** to begin everything. + """ + + if not doc_start.match(line): + return + + # start a new entry + self.reset_state(ln + 1) + self.entry.in_doc_sect =3D False + + # next line is always the function name + self.state =3D self.STATE_NAME + + def process_name(self, ln, line): + """ + STATE_NAME: Looking for the "name - description" line + """ + + if doc_block.search(line): + self.entry.new_start_line =3D ln + + if not doc_block.group(1): + self.entry.section =3D self.section_intro + else: + self.entry.section =3D doc_block.group(1) + + self.state =3D self.STATE_DOCBLOCK + return + + if doc_decl.search(line): + self.entry.identifier =3D doc_decl.group(1) + self.entry.is_kernel_comment =3D False + + decl_start =3D str(doc_com) # comment block asterisk + fn_type =3D r"(?:\w+\s*\*\s*)?" # type (for non-functions) + parenthesis =3D r"(?:\(\w*\))?" # optional parenthesis on fu= nction + decl_end =3D r"(?:[-:].*)" # end of the name part + + # test for pointer declaration type, foo * bar() - desc + r =3D Re(fr"^{decl_start}([\w\s]+?){parenthesis}?\s*{decl_end}= ?$") + if r.search(line): + self.entry.identifier =3D r.group(1) + + # Test for data declaration + r =3D Re(r"^\s*\*?\s*(struct|union|enum|typedef)\b\s*(\w*)") + if r.search(line): + self.entry.decl_type =3D r.group(1) + self.entry.identifier =3D r.group(2) + self.entry.is_kernel_comment =3D True + else: + # Look for foo() or static void foo() - description; + # or misspelt identifier + + r1 =3D Re(fr"^{decl_start}{fn_type}(\w+)\s*{parenthesis}\s= *{decl_end}?$") + r2 =3D Re(fr"^{decl_start}{fn_type}(\w+[^-:]*){parenthesis= }\s*{decl_end}$") + + for r in [r1, r2]: + if r.search(line): + self.entry.identifier =3D r.group(1) + self.entry.decl_type =3D "function" + + r =3D Re(r"define\s+") + self.entry.identifier =3D r.sub("", self.entry.ide= ntifier) + self.entry.is_kernel_comment =3D True + break + + self.entry.identifier =3D self.entry.identifier.strip(" ") + + self.state =3D self.STATE_BODY + + # if there's no @param blocks need to set up default section h= ere + self.entry.section =3D self.section_default + self.entry.new_start_line =3D ln + 1 + + r =3D Re("[-:](.*)") + if r.search(line): + # strip leading/trailing/multiple spaces + self.entry.descr =3D r.group(1).strip(" ") + + r =3D Re(r"\s+") + self.entry.descr =3D r.sub(" ", self.entry.descr) + self.entry.declaration_purpose =3D self.entry.descr + self.state =3D self.STATE_BODY_MAYBE + else: + self.entry.declaration_purpose =3D "" + + if not self.entry.is_kernel_comment: + self.emit_warning(ln, + f"This comment starts with '/**', but is= n't a kernel-doc comment. Refer Documentation/doc-guide/kernel-doc.rst\n{li= ne}") + self.state =3D self.STATE_NORMAL + + if not self.entry.declaration_purpose and self.config.wshort_d= esc: + self.emit_warning(ln, + f"missing initial short description on l= ine:\n{line}") + + if not self.entry.identifier and self.entry.decl_type !=3D "en= um": + self.emit_warning(ln, + f"wrong kernel-doc identifier on line:\n= {line}") + self.state =3D self.STATE_NORMAL + + if self.config.verbose: + self.emit_warning(ln, + f"Scanning doc for {self.entry.decl_type= } {self.entry.identifier}", + warning=3DFalse) + + return + + # Failed to find an identifier. Emit a warning + self.emit_warning(ln, f"Cannot find identifier on line:\n{line}") + + def process_body(self, ln, line): + """ + STATE_BODY and STATE_BODY_MAYBE: the bulk of a kerneldoc comment. + """ + + if self.state =3D=3D self.STATE_BODY_WITH_BLANK_LINE: + r =3D Re(r"\s*\*\s?\S") + if r.match(line): + self.dump_section() + self.entry.section =3D self.section_default + self.entry.new_start_line =3D line + self.entry.contents =3D "" + + if doc_sect.search(line): + self.entry.in_doc_sect =3D True + newsection =3D doc_sect.group(1) + + if newsection.lower() in ["description", "context"]: + newsection =3D newsection.title() + + # Special case: @return is a section, not a param description + if newsection.lower() in ["@return", "@returns", + "return", "returns"]: + newsection =3D "Return" + + # Perl kernel-doc has a check here for contents before section= s. + # the logic there is always false, as in_doc_sect variable is + # always true. So, just don't implement Wcontents_before_secti= ons + + # .title() + newcontents =3D doc_sect.group(2) + if not newcontents: + newcontents =3D "" + + if self.entry.contents.strip("\n"): + self.dump_section() + + self.entry.new_start_line =3D ln + self.entry.section =3D newsection + self.entry.leading_space =3D None + + self.entry.contents =3D newcontents.lstrip() + if self.entry.contents: + self.entry.contents +=3D "\n" + + self.state =3D self.STATE_BODY + return + + if doc_end.search(line): + if self.entry.contents.strip("\n"): + self.dump_section() + + # Look for doc_com + + doc_end: + r =3D Re(r'\s*\*\s*[a-zA-Z_0-9:\.]+\*/') + if r.match(line): + self.emit_warning(ln, f"suspicious ending line: {line}") + + self.entry.prototype =3D "" + self.entry.new_start_line =3D ln + 1 + + self.state =3D self.STATE_PROTO + return + + if doc_content.search(line): + cont =3D doc_content.group(1) + + if cont =3D=3D "": + if self.entry.section =3D=3D self.section_context: + self.dump_section() + + self.entry.new_start_line =3D ln + self.state =3D self.STATE_BODY + else: + if self.entry.section !=3D self.section_default: + self.state =3D self.STATE_BODY_WITH_BLANK_LINE + else: + self.state =3D self.STATE_BODY + + self.entry.contents +=3D "\n" + + elif self.state =3D=3D self.STATE_BODY_MAYBE: + + # Continued declaration purpose + self.entry.declaration_purpose =3D self.entry.declaration_= purpose.rstrip() + self.entry.declaration_purpose +=3D " " + cont + + r =3D Re(r"\s+") + self.entry.declaration_purpose =3D r.sub(' ', + self.entry.declarat= ion_purpose) + + else: + if self.entry.section.startswith('@') or \ + self.entry.section =3D=3D self.section_context: + if self.entry.leading_space is None: + r =3D Re(r'^(\s+)') + if r.match(cont): + self.entry.leading_space =3D len(r.group(1)) + else: + self.entry.leading_space =3D 0 + + # Double-check if leading space are realy spaces + pos =3D 0 + for i in range(0, self.entry.leading_space): + if cont[i] !=3D " ": + break + pos +=3D 1 + + cont =3D cont[pos:] + + # NEW LOGIC: + # In case it is different, update it + if self.entry.leading_space !=3D pos: + self.entry.leading_space =3D pos + + self.entry.contents +=3D cont + "\n" + return + + # Unknown line, ignore + self.emit_warning(ln, f"bad line: {line}") + + def process_inline(self, ln, line): + """STATE_INLINE: docbook comments within a prototype.""" + + if self.inline_doc_state =3D=3D self.STATE_INLINE_NAME and \ + doc_inline_sect.search(line): + self.entry.section =3D doc_inline_sect.group(1) + self.entry.new_start_line =3D ln + + self.entry.contents =3D doc_inline_sect.group(2).lstrip() + if self.entry.contents !=3D "": + self.entry.contents +=3D "\n" + + self.inline_doc_state =3D self.STATE_INLINE_TEXT + # Documentation block end */ + return + + if doc_inline_end.search(line): + if self.entry.contents not in ["", "\n"]: + self.dump_section() + + self.state =3D self.STATE_PROTO + self.inline_doc_state =3D self.STATE_INLINE_NA + return + + if doc_content.search(line): + if self.inline_doc_state =3D=3D self.STATE_INLINE_TEXT: + self.entry.contents +=3D doc_content.group(1) + "\n" + if not self.entry.contents.strip(" ").rstrip("\n"): + self.entry.contents =3D "" + + elif self.inline_doc_state =3D=3D self.STATE_INLINE_NAME: + self.emit_warning(ln, + f"Incorrect use of kernel-doc format: {l= ine}") + + self.inline_doc_state =3D self.STATE_INLINE_ERROR + + def syscall_munge(self, ln, proto): + """ + Handle syscall definitions + """ + + is_void =3D False + + # Strip newlines/CR's + proto =3D re.sub(r'[\r\n]+', ' ', proto) + + # Check if it's a SYSCALL_DEFINE0 + if 'SYSCALL_DEFINE0' in proto: + is_void =3D True + + # Replace SYSCALL_DEFINE with correct return type & function name + proto =3D Re(r'SYSCALL_DEFINE.*\(').sub('long sys_', proto) + + r =3D Re(r'long\s+(sys_.*?),') + if r.search(proto): + proto =3D proto.replace(',', '(', count=3D1) + elif is_void: + proto =3D proto.replace(')', '(void)', count=3D1) + + # Now delete all of the odd-numbered commas in the proto + # so that argument types & names don't have a comma between them + count =3D 0 + length =3D len(proto) + + if is_void: + length =3D 0 # skip the loop if is_void + + for ix in range(length): + if proto[ix] =3D=3D ',': + count +=3D 1 + if count % 2 =3D=3D 1: + proto =3D proto[:ix] + ' ' + proto[ix+1:] + + return proto + + def tracepoint_munge(self, ln, proto): + """ + Handle tracepoint definitions + """ + + tracepointname =3D None + tracepointargs =3D None + + # Match tracepoint name based on different patterns + r =3D Re(r'TRACE_EVENT\((.*?),') + if r.search(proto): + tracepointname =3D r.group(1) + + r =3D Re(r'DEFINE_SINGLE_EVENT\((.*?),') + if r.search(proto): + tracepointname =3D r.group(1) + + r =3D Re(r'DEFINE_EVENT\((.*?),(.*?),') + if r.search(proto): + tracepointname =3D r.group(2) + + if tracepointname: + tracepointname =3D tracepointname.lstrip() + + r =3D Re(r'TP_PROTO\((.*?)\)') + if r.search(proto): + tracepointargs =3D r.group(1) + + if not tracepointname or not tracepointargs: + self.emit_warning(ln, + f"Unrecognized tracepoint format:\n{proto}\n= ") + else: + proto =3D f"static inline void trace_{tracepointname}({tracepo= intargs})" + self.entry.identifier =3D f"trace_{self.entry.identifier}" + + return proto + + def process_proto_function(self, ln, line): + """Ancillary routine to process a function prototype""" + + # strip C99-style comments to end of line + r =3D Re(r"\/\/.*$", re.S) + line =3D r.sub('', line) + + if Re(r'\s*#\s*define').match(line): + self.entry.prototype =3D line + elif line.startswith('#'): + # Strip other macros like #ifdef/#ifndef/#endif/... + pass + else: + r =3D Re(r'([^\{]*)') + if r.match(line): + self.entry.prototype +=3D r.group(1) + " " + + if '{' in line or ';' in line or Re(r'\s*#\s*define').match(line): + # strip comments + r =3D Re(r'/\*.*?\*/') + self.entry.prototype =3D r.sub('', self.entry.prototype) + + # strip newlines/cr's + r =3D Re(r'[\r\n]+') + self.entry.prototype =3D r.sub(' ', self.entry.prototype) + + # strip leading spaces + r =3D Re(r'^\s+') + self.entry.prototype =3D r.sub('', self.entry.prototype) + + # Handle self.entry.prototypes for function pointers like: + # int (*pcs_config)(struct foo) + + r =3D Re(r'^(\S+\s+)\(\s*\*(\S+)\)') + self.entry.prototype =3D r.sub(r'\1\2', self.entry.prototype) + + if 'SYSCALL_DEFINE' in self.entry.prototype: + self.entry.prototype =3D self.syscall_munge(ln, + self.entry.proto= type) + + r =3D Re(r'TRACE_EVENT|DEFINE_EVENT|DEFINE_SINGLE_EVENT') + if r.search(self.entry.prototype): + self.entry.prototype =3D self.tracepoint_munge(ln, + self.entry.pr= ototype) + + self.dump_function(ln, self.entry.prototype) + self.reset_state(ln) + + def process_proto_type(self, ln, line): + """Ancillary routine to process a type""" + + # Strip newlines/cr's. + line =3D Re(r'[\r\n]+', re.S).sub(' ', line) + + # Strip leading spaces + line =3D Re(r'^\s+', re.S).sub('', line) + + # Strip trailing spaces + line =3D Re(r'\s+$', re.S).sub('', line) + + # Strip C99-style comments to the end of the line + line =3D Re(r"\/\/.*$", re.S).sub('', line) + + # To distinguish preprocessor directive from regular declaration l= ater. + if line.startswith('#'): + line +=3D ";" + + r =3D Re(r'([^\{\};]*)([\{\};])(.*)') + while True: + if r.search(line): + if self.entry.prototype: + self.entry.prototype +=3D " " + self.entry.prototype +=3D r.group(1) + r.group(2) + + self.entry.brcount +=3D r.group(2).count('{') + self.entry.brcount -=3D r.group(2).count('}') + + self.entry.brcount =3D max(self.entry.brcount, 0) + + if r.group(2) =3D=3D ';' and self.entry.brcount =3D=3D 0: + self.dump_declaration(ln, self.entry.prototype) + self.reset_state(ln) + break + + line =3D r.group(3) + else: + self.entry.prototype +=3D line + break + + def process_proto(self, ln, line): + """STATE_PROTO: reading a function/whatever prototype.""" + + if doc_inline_oneline.search(line): + self.entry.section =3D doc_inline_oneline.group(1) + self.entry.contents =3D doc_inline_oneline.group(2) + + if self.entry.contents !=3D "": + self.entry.contents +=3D "\n" + self.dump_section(start_new=3DFalse) + + elif doc_inline_start.search(line): + self.state =3D self.STATE_INLINE + self.inline_doc_state =3D self.STATE_INLINE_NAME + + elif self.entry.decl_type =3D=3D 'function': + self.process_proto_function(ln, line) + + else: + self.process_proto_type(ln, line) + + def process_docblock(self, ln, line): + """STATE_DOCBLOCK: within a DOC: block.""" + + if doc_end.search(line): + self.dump_section() + self.output_declaration("doc", None, + sectionlist=3Dself.entry.sectionlist, + sections=3Dself.entry.sections, = module=3Dself.config.modulename) + self.reset_state(ln) + + elif doc_content.search(line): + self.entry.contents +=3D doc_content.group(1) + "\n" + + def run(self): + """ + Open and process each line of a C source file. + he parsing is controlled via a state machine, and the line is pass= ed + to a different process function depending on the state. The process + function may update the state as needed. + """ + + cont =3D False + prev =3D "" + prev_ln =3D None + + try: + with open(self.fname, "r", encoding=3D"utf8", + errors=3D"backslashreplace") as fp: + for ln, line in enumerate(fp): + + line =3D line.expandtabs().strip("\n") + + # Group continuation lines on prototypes + if self.state =3D=3D self.STATE_PROTO: + if line.endswith("\\"): + prev +=3D line.removesuffix("\\") + cont =3D True + + if not prev_ln: + prev_ln =3D ln + + continue + + if cont: + ln =3D prev_ln + line =3D prev + line + prev =3D "" + cont =3D False + prev_ln =3D None + + self.config.log.debug("%d %s%s: %s", + ln, self.st_name[self.state], + self.st_inline_name[self.inline_= doc_state], + line) + + # TODO: not all states allow EXPORT_SYMBOL*, so this + # can be optimized later on to speedup parsing + self.process_export(self.config.function_table, line) + + # Hand this line to the appropriate state handler + if self.state =3D=3D self.STATE_NORMAL: + self.process_normal(ln, line) + elif self.state =3D=3D self.STATE_NAME: + self.process_name(ln, line) + elif self.state in [self.STATE_BODY, self.STATE_BODY_M= AYBE, + self.STATE_BODY_WITH_BLANK_LINE]: + self.process_body(ln, line) + elif self.state =3D=3D self.STATE_INLINE: # scanning = for inline parameters + self.process_inline(ln, line) + elif self.state =3D=3D self.STATE_PROTO: + self.process_proto(ln, line) + elif self.state =3D=3D self.STATE_DOCBLOCK: + self.process_docblock(ln, line) + except OSError: + self.config.log.error(f"Error: Cannot open file {self.fname}") + self.config.errors +=3D 1 + + +class GlobSourceFiles: + """ + Parse C source code file names and directories via an Interactor. + + """ + + def __init__(self, srctree=3DNone, valid_extensions=3DNone): + """ + Initialize valid extensions with a tuple. + + If not defined, assume default C extensions (.c and .h) + + It would be possible to use python's glob function, but it is + very slow, and it is not interactive. So, it would wait to read all + directories before actually do something. + + So, let's use our own implementation. + """ + + if not valid_extensions: + self.extensions =3D (".c", ".h") + else: + self.extensions =3D valid_extensions + + self.srctree =3D srctree + + def _parse_dir(self, dirname): + """Internal function to parse files recursively""" + + with os.scandir(dirname) as obj: + for entry in obj: + name =3D os.path.join(dirname, entry.name) + + if entry.is_dir(): + yield from self._parse_dir(name) + + if not entry.is_file(): + continue + + basename =3D os.path.basename(name) + + if not basename.endswith(self.extensions): + continue + + yield name + + def parse_files(self, file_list, file_not_found_cb): + for fname in file_list: + if self.srctree: + f =3D os.path.join(self.srctree, fname) + else: + f =3D fname + + if os.path.isdir(f): + yield from self._parse_dir(f) + elif os.path.isfile(f): + yield f + elif file_not_found_cb: + file_not_found_cb(fname) + + +class KernelFiles(): + + def parse_file(self, fname): + + doc =3D KernelDoc(self.config, fname) + doc.run() + + return doc + + def process_export_file(self, fname): + try: + with open(fname, "r", encoding=3D"utf8", + errors=3D"backslashreplace") as fp: + for line in fp: + KernelDoc.process_export(self.config.function_table, l= ine) + + except IOError: + print(f"Error: Cannot open fname {fname}", fname=3Dsys.stderr) + self.config.errors +=3D 1 + + def file_not_found_cb(self, fname): + self.config.log.error("Cannot find file %s", fname) + self.config.errors +=3D 1 + + def __init__(self, files=3DNone, verbose=3DFalse, out_style=3DNone, + werror=3DFalse, wreturn=3DFalse, wshort_desc=3DFalse, + wcontents_before_sections=3DFalse, + logger=3DNone, modulename=3DNone, export_file=3DNone): + """Initialize startup variables and parse all files""" + + + if not verbose: + verbose =3D bool(os.environ.get("KBUILD_VERBOSE", 0)) + + if not modulename: + modulename =3D "Kernel API" + + dt =3D datetime.now() + if os.environ.get("KBUILD_BUILD_TIMESTAMP", None): + # use UTC TZ + to_zone =3D tz.gettz('UTC') + dt =3D dt.astimezone(to_zone) + + if not werror: + kcflags =3D os.environ.get("KCFLAGS", None) + if kcflags: + match =3D re.search(r"(\s|^)-Werror(\s|$)/", kcflags) + if match: + werror =3D True + + # reading this variable is for backwards compat just in case + # someone was calling it with the variable from outside the + # kernel's build system + kdoc_werror =3D os.environ.get("KDOC_WERROR", None) + if kdoc_werror: + werror =3D kdoc_werror + + # Set global config data used on all files + self.config =3D argparse.Namespace + + self.config.verbose =3D verbose + self.config.werror =3D werror + self.config.wreturn =3D wreturn + self.config.wshort_desc =3D wshort_desc + self.config.wcontents_before_sections =3D wcontents_before_sections + self.config.modulename =3D modulename + + self.config.function_table =3D set() + self.config.source_map =3D {} + + if not logger: + self.config.log =3D logging.getLogger("kernel-doc") + else: + self.config.log =3D logger + + self.config.kernel_version =3D os.environ.get("KERNELVERSION", + "unknown kernel versio= n'") + self.config.src_tree =3D os.environ.get("SRCTREE", None) + + self.out_style =3D out_style + self.export_file =3D export_file + + # Initialize internal variables + + self.config.errors =3D 0 + self.results =3D [] + + self.file_list =3D files + self.files =3D set() + + def parse(self): + """ + Parse all files + """ + + glob =3D GlobSourceFiles(srctree=3Dself.config.src_tree) + + # Let's use a set here to avoid duplicating files + + for fname in glob.parse_files(self.file_list, self.file_not_found_= cb): + if fname in self.files: + continue + + self.files.add(fname) + + res =3D self.parse_file(fname) + self.results.append((res.fname, res.entries)) + + if not self.files: + sys.exit(1) + + # If a list of export files was provided, parse EXPORT_SYMBOL* + # from the ones not already parsed + + if self.export_file: + files =3D self.files + + glob =3D GlobSourceFiles(srctree=3Dself.config.src_tree) + + for fname in glob.parse_files(self.export_file, + self.file_not_found_cb): + if fname not in files: + files.add(fname) + + self.process_export_file(fname) + + def out_msg(self, fname, name, arg): + # TODO: filter out unwanted parts + + return self.out_style.msg(fname, name, arg) + + def msg(self, enable_lineno=3DFalse, export=3DFalse, internal=3DFalse, + symbol=3DNone, nosymbol=3DNone): + + function_table =3D self.config.function_table + + if symbol: + for s in symbol: + function_table.add(s) + + # Output none mode: only warnings will be shown + if not self.out_style: + return + + self.out_style.set_config(self.config) + + self.out_style.set_filter(export, internal, symbol, nosymbol, + function_table, enable_lineno) + + for fname, arg_tuple in self.results: + for name, arg in arg_tuple: + if self.out_msg(fname, name, arg): + ln =3D arg.get("ln", 0) + dtype =3D arg.get('type', "") + + self.config.log.warning("%s:%d Can't handle %s", + fname, ln, dtype) + + +class OutputFormat: + # output mode. + OUTPUT_ALL =3D 0 # output all symbols and doc sections + OUTPUT_INCLUDE =3D 1 # output only specified symbols + OUTPUT_EXPORTED =3D 2 # output exported symbols + OUTPUT_INTERNAL =3D 3 # output non-exported symbols + + # Virtual member to be overriden at the inherited classes + highlights =3D [] + + def __init__(self): + """Declare internal vars and set mode to OUTPUT_ALL""" + + self.out_mode =3D self.OUTPUT_ALL + self.enable_lineno =3D None + self.nosymbol =3D {} + self.symbol =3D None + self.function_table =3D set() + self.config =3D None + + def set_config(self, config): + self.config =3D config + + def set_filter(self, export, internal, symbol, nosymbol, function_tabl= e, + enable_lineno): + """ + Initialize filter variables according with the requested mode. + + Only one choice is valid between export, internal and symbol. + + The nosymbol filter can be used on all modes. + """ + + self.enable_lineno =3D enable_lineno + + if symbol: + self.out_mode =3D self.OUTPUT_INCLUDE + function_table =3D symbol + elif export: + self.out_mode =3D self.OUTPUT_EXPORTED + elif internal: + self.out_mode =3D self.OUTPUT_INTERNAL + else: + self.out_mode =3D self.OUTPUT_ALL + + if nosymbol: + self.nosymbol =3D set(nosymbol) + + if function_table: + self.function_table =3D function_table + + def highlight_block(self, block): + """ + Apply the RST highlights to a sub-block of text. + """ + + for r, sub in self.highlights: + block =3D r.sub(sub, block) + + return block + + def check_doc(self, name): + """Check if DOC should be output""" + + if self.out_mode =3D=3D self.OUTPUT_ALL: + return True + + if self.out_mode =3D=3D self.OUTPUT_INCLUDE: + if name in self.nosymbol: + return False + + if name in self.function_table: + return True + + return False + + def check_declaration(self, dtype, name): + if name in self.nosymbol: + return False + + if self.out_mode =3D=3D self.OUTPUT_ALL: + return True + + if self.out_mode in [ self.OUTPUT_INCLUDE, self.OUTPUT_EXPORTED ]: + if name in self.function_table: + return True + + if self.out_mode =3D=3D self.OUTPUT_INTERNAL: + if dtype !=3D "function": + return True + + if name not in self.function_table: + return True + + return False + + def check_function(self, fname, name, args): + return True + + def check_enum(self, fname, name, args): + return True + + def check_typedef(self, fname, name, args): + return True + + def msg(self, fname, name, args): + + dtype =3D args.get('type', "") + + if dtype =3D=3D "doc": + self.out_doc(fname, name, args) + return False + + if not self.check_declaration(dtype, name): + return False + + if dtype =3D=3D "function": + self.out_function(fname, name, args) + return False + + if dtype =3D=3D "enum": + self.out_enum(fname, name, args) + return False + + if dtype =3D=3D "typedef": + self.out_typedef(fname, name, args) + return False + + if dtype in ["struct", "union"]: + self.out_struct(fname, name, args) + return False + + # Warn if some type requires an output logic + self.config.log.warning("doesn't now how to output '%s' block", + dtype) + + return True + + # Virtual methods to be overridden by inherited classes + def out_doc(self, fname, name, args): + pass + + def out_function(self, fname, name, args): + pass + + def out_enum(self, fname, name, args): + pass + + def out_typedef(self, fname, name, args): + pass + + def out_struct(self, fname, name, args): + pass + + +class RestFormat(OutputFormat): + # """Consts and functions used by ReST output""" + + highlights =3D [ + (type_constant, r"``\1``"), + (type_constant2, r"``\1``"), + + # Note: need to escape () to avoid func matching later + (type_member_func, r":c:type:`\1\2\3\\(\\) <\1>`"), + (type_member, r":c:type:`\1\2\3 <\1>`"), + (type_fp_param, r"**\1\\(\\)**"), + (type_fp_param2, r"**\1\\(\\)**"), + (type_func, r"\1()"), + (type_enum, r":c:type:`\1 <\2>`"), + (type_struct, r":c:type:`\1 <\2>`"), + (type_typedef, r":c:type:`\1 <\2>`"), + (type_union, r":c:type:`\1 <\2>`"), + + # in rst this can refer to any type + (type_fallback, r":c:type:`\1`"), + (type_param_ref, r"**\1\2**") + ] + blankline =3D "\n" + + sphinx_literal =3D Re(r'^[^.].*::$', cache=3DFalse) + sphinx_cblock =3D Re(r'^\.\.\ +code-block::', cache=3DFalse) + + def __init__(self): + """ + Creates class variables. + + Not really mandatory, but it is a good coding style and makes + pylint happy. + """ + + super().__init__() + self.lineprefix =3D "" + + def print_lineno (self, ln): + """Outputs a line number""" + + if self.enable_lineno and ln: + print(f".. LINENO {ln}") + + def output_highlight(self, args): + input_text =3D args + output =3D "" + in_literal =3D False + litprefix =3D "" + block =3D "" + + for line in input_text.strip("\n").split("\n"): + + # If we're in a literal block, see if we should drop out of it. + # Otherwise, pass the line straight through unmunged. + if in_literal: + if line.strip(): # If the line is not blank + # If this is the first non-blank line in a literal blo= ck, + # figure out the proper indent. + if not litprefix: + r =3D Re(r'^(\s*)') + if r.match(line): + litprefix =3D '^' + r.group(1) + else: + litprefix =3D "" + + output +=3D line + "\n" + elif not Re(litprefix).match(line): + in_literal =3D False + else: + output +=3D line + "\n" + else: + output +=3D line + "\n" + + # Not in a literal block (or just dropped out) + if not in_literal: + block +=3D line + "\n" + if self.sphinx_literal.match(line) or self.sphinx_cblock.m= atch(line): + in_literal =3D True + litprefix =3D "" + output +=3D self.highlight_block(block) + block =3D "" + + # Handle any remaining block + if block: + output +=3D self.highlight_block(block) + + # Print the output with the line prefix + for line in output.strip("\n").split("\n"): + print(self.lineprefix + line) + + def out_section(self, args, out_reference=3DFalse): + """ + Outputs a block section. + + This could use some work; it's used to output the DOC: sections, a= nd + starts by putting out the name of the doc section itself, but that + tends to duplicate a header already in the template file. + """ + + sectionlist =3D args.get('sectionlist', []) + sections =3D args.get('sections', {}) + section_start_lines =3D args.get('section_start_lines', {}) + + for section in sectionlist: + # Skip sections that are in the nosymbol_table + if section in self.nosymbol: + continue + + if not self.out_mode =3D=3D self.OUTPUT_INCLUDE: + if out_reference: + print(f".. _{section}:\n") + + if not self.symbol: + print(f'{self.lineprefix}**{section}**\n') + + self.print_lineno(section_start_lines.get(section, 0)) + self.output_highlight(sections[section]) + print() + print() + + def out_doc(self, fname, name, args): + if not self.check_doc(name): + return + + self.out_section(args, out_reference=3DTrue) + + def out_function(self, fname, name, args): + + oldprefix =3D self.lineprefix + signature =3D "" + + func_macro =3D args.get('func_macro', False) + if func_macro: + signature =3D args['function'] + else: + if args.get('functiontype'): + signature =3D args['functiontype'] + " " + signature +=3D args['function'] + " (" + + parameterlist =3D args.get('parameterlist', []) + parameterdescs =3D args.get('parameterdescs', {}) + parameterdesc_start_lines =3D args.get('parameterdesc_start_lines'= , {}) + + ln =3D args.get('ln', 0) + + count =3D 0 + for parameter in parameterlist: + if count !=3D 0: + signature +=3D ", " + count +=3D 1 + dtype =3D args['parametertypes'].get(parameter, "") + + if function_pointer.search(dtype): + signature +=3D function_pointer.group(1) + parameter + fun= ction_pointer.group(3) + else: + signature +=3D dtype + + if not func_macro: + signature +=3D ")" + + if args.get('typedef') or not args.get('functiontype'): + print(f".. c:macro:: {args['function']}\n") + + if args.get('typedef'): + self.print_lineno(ln) + print(" **Typedef**: ", end=3D"") + self.lineprefix =3D "" + self.output_highlight(args.get('purpose', "")) + print("\n\n**Syntax**\n") + print(f" ``{signature}``\n") + else: + print(f"``{signature}``\n") + else: + print(f".. c:function:: {signature}\n") + + if not args.get('typedef'): + self.print_lineno(ln) + self.lineprefix =3D " " + self.output_highlight(args.get('purpose', "")) + print() + + # Put descriptive text into a container (HTML
) to help set + # function prototypes apart + self.lineprefix =3D " " + + if parameterlist: + print(".. container:: kernelindent\n") + print(f"{self.lineprefix}**Parameters**\n") + + for parameter in parameterlist: + parameter_name =3D Re(r'\[.*').sub('', parameter) + dtype =3D args['parametertypes'].get(parameter, "") + + if dtype: + print(f"{self.lineprefix}``{dtype}``") + else: + print(f"{self.lineprefix}``{parameter}``") + + self.print_lineno(parameterdesc_start_lines.get(parameter_name= , 0)) + + self.lineprefix =3D " " + if parameter_name in parameterdescs and \ + parameterdescs[parameter_name] !=3D KernelDoc.undescribed: + + self.output_highlight(parameterdescs[parameter_name]) + print() + else: + print(f"{self.lineprefix}*undescribed*\n") + self.lineprefix =3D " " + + self.out_section(args) + self.lineprefix =3D oldprefix + + def out_enum(self, fname, name, args): + + oldprefix =3D self.lineprefix + name =3D args.get('enum', '') + parameterlist =3D args.get('parameterlist', []) + parameterdescs =3D args.get('parameterdescs', {}) + ln =3D args.get('ln', 0) + + print(f"\n\n.. c:enum:: {name}\n") + + self.print_lineno(ln) + self.lineprefix =3D " " + self.output_highlight(args.get('purpose', '')) + print() + + print(".. container:: kernelindent\n") + outer =3D self.lineprefix + " " + self.lineprefix =3D outer + " " + print(f"{outer}**Constants**\n") + + for parameter in parameterlist: + print(f"{outer}``{parameter}``") + + if parameterdescs.get(parameter, '') !=3D KernelDoc.undescribe= d: + self.output_highlight(parameterdescs[parameter]) + else: + print(f"{self.lineprefix}*undescribed*\n") + print() + + self.lineprefix =3D oldprefix + self.out_section(args) + + def out_typedef(self, fname, name, args): + + oldprefix =3D self.lineprefix + name =3D args.get('typedef', '') + ln =3D args.get('ln', 0) + + print(f"\n\n.. c:type:: {name}\n") + + self.print_lineno(ln) + self.lineprefix =3D " " + + self.output_highlight(args.get('purpose', '')) + + print() + + self.lineprefix =3D oldprefix + self.out_section(args) + + def out_struct(self, fname, name, args): + + name =3D args.get('struct', "") + purpose =3D args.get('purpose', "") + declaration =3D args.get('definition', "") + dtype =3D args.get('type', "struct") + ln =3D args.get('ln', 0) + + parameterlist =3D args.get('parameterlist', []) + parameterdescs =3D args.get('parameterdescs', {}) + parameterdesc_start_lines =3D args.get('parameterdesc_start_lines'= , {}) + + print(f"\n\n.. c:{dtype}:: {name}\n") + + self.print_lineno(ln) + + oldprefix =3D self.lineprefix + self.lineprefix +=3D " " + + self.output_highlight(purpose) + print() + + print(".. container:: kernelindent\n") + print(f"{self.lineprefix}**Definition**::\n") + + self.lineprefix =3D self.lineprefix + " " + + declaration =3D declaration.replace("\t", self.lineprefix) + + print(f"{self.lineprefix}{dtype} {name}" + ' {') + print(f"{declaration}{self.lineprefix}" + "};\n") + + self.lineprefix =3D " " + print(f"{self.lineprefix}**Members**\n") + for parameter in parameterlist: + if not parameter or parameter.startswith("#"): + continue + + parameter_name =3D parameter.split("[", maxsplit=3D1)[0] + + if parameterdescs.get(parameter_name) =3D=3D KernelDoc.undescr= ibed: + continue + + self.print_lineno(parameterdesc_start_lines.get(parameter_name= , 0)) + + print(f"{self.lineprefix}``{parameter}``") + + self.lineprefix =3D " " + self.output_highlight(parameterdescs[parameter_name]) + self.lineprefix =3D " " + + print() + + print() + + self.lineprefix =3D oldprefix + self.out_section(args) + + +class ManFormat(OutputFormat): + """Consts and functions used by man pages output""" + + highlights =3D ( + (type_constant, r"\1"), + (type_constant2, r"\1"), + (type_func, r"\\fB\1\\fP"), + (type_enum, r"\\fI\1\\fP"), + (type_struct, r"\\fI\1\\fP"), + (type_typedef, r"\\fI\1\\fP"), + (type_union, r"\\fI\1\\fP"), + (type_param, r"\\fI\1\\fP"), + (type_param_ref, r"\\fI\1\2\\fP"), + (type_member, r"\\fI\1\2\3\\fP"), + (type_fallback, r"\\fI\1\\fP") + ) + blankline =3D "" + + def __init__(self): + """ + Creates class variables. + + Not really mandatory, but it is a good coding style and makes + pylint happy. + """ + + super().__init__() + + dt =3D datetime.now() + if os.environ.get("KBUILD_BUILD_TIMESTAMP", None): + # use UTC TZ + to_zone =3D tz.gettz('UTC') + dt =3D dt.astimezone(to_zone) + + self.man_date =3D dt.strftime("%B %Y") + + def output_highlight(self, block): + + contents =3D self.highlight_block(block) + + if isinstance(contents, list): + contents =3D "\n".join(contents) + + for line in contents.strip("\n").split("\n"): + line =3D Re(r"^\s*").sub("", line) + + if line and line[0] =3D=3D ".": + print("\\&" + line) + else: + print(line) + + def out_doc(self, fname, name, args): + module =3D args.get('module') + sectionlist =3D args.get('sectionlist', []) + sections =3D args.get('sections', {}) + + print(f'.TH "{module}" 9 "{module}" "{self.man_date}" "API Manual"= LINUX') + + for section in sectionlist: + print(f'.SH "{section}"') + self.output_highlight(sections.get(section)) + + def out_function(self, fname, name, args): + """output function in man""" + + parameterlist =3D args.get('parameterlist', []) + parameterdescs =3D args.get('parameterdescs', {}) + sectionlist =3D args.get('sectionlist', []) + sections =3D args.get('sections', {}) + + print(f'.TH "{args['function']}" 9 "{args['function']}" "{self.man= _date}" "Kernel Hacker\'s Manual" LINUX') + + print(".SH NAME") + print(f"{args['function']} \\- {args['purpose']}") + + print(".SH SYNOPSIS") + if args.get('functiontype', ''): + print(f'.B "{args['functiontype']}" {args['function']}') + else: + print(f'.B "{args['function']}') + + count =3D 0 + parenth =3D "(" + post =3D "," + + for parameter in parameterlist: + if count =3D=3D len(parameterlist) - 1: + post =3D ");" + + dtype =3D args['parametertypes'].get(parameter, "") + if function_pointer.match(dtype): + # Pointer-to-function + print(f'".BI "{parenth}{function_pointer.group(1)}" " ") (= {function_pointer.group(2)}){post}"') + else: + dtype =3D Re(r'([^\*])$').sub(r'\1 ', dtype) + + print(f'.BI "{parenth}{dtype}" "{post}"') + count +=3D 1 + parenth =3D "" + + if parameterlist: + print(".SH ARGUMENTS") + + for parameter in parameterlist: + parameter_name =3D re.sub(r'\[.*', '', parameter) + + print(f'.IP "{parameter}" 12') + self.output_highlight(parameterdescs.get(parameter_name, "")) + + for section in sectionlist: + print(f'.SH "{section.upper()}"') + self.output_highlight(sections[section]) + + def out_enum(self, fname, name, args): + + name =3D args.get('enum', '') + parameterlist =3D args.get('parameterlist', []) + sectionlist =3D args.get('sectionlist', []) + sections =3D args.get('sections', {}) + + print(f'.TH "{args['module']}" 9 "enum {args['enum']}" "{self.man_= date}" "API Manual" LINUX') + + print(".SH NAME") + print(f"enum {args['enum']} \\- {args['purpose']}") + + print(".SH SYNOPSIS") + print(f"enum {args['enum']}" + " {") + + count =3D 0 + for parameter in parameterlist: + print(f'.br\n.BI " {parameter}"') + if count =3D=3D len(parameterlist) - 1: + print("\n};") + else: + print(", \n.br") + + count +=3D 1 + + print(".SH Constants") + + for parameter in parameterlist: + parameter_name =3D Re(r'\[.*').sub('', parameter) + print(f'.IP "{parameter}" 12') + self.output_highlight(args['parameterdescs'].get(parameter_nam= e, "")) + + for section in sectionlist: + print(f'.SH "{section}"') + self.output_highlight(sections[section]) + + def out_typedef(self, fname, name, args): + module =3D args.get('module') + typedef =3D args.get('typedef') + purpose =3D args.get('purpose') + sectionlist =3D args.get('sectionlist', []) + sections =3D args.get('sections', {}) + + print(f'.TH "{module}" 9 "{typedef}" "{self.man_date}" "API Manual= " LINUX') + + print(".SH NAME") + print(f"typedef {typedef} \\- {purpose}") + + for section in sectionlist: + print(f'.SH "{section}"') + self.output_highlight(sections.get(section)) + + def out_struct(self, fname, name, args): + module =3D args.get('module') + struct_type =3D args.get('type') + struct_name =3D args.get('struct') + purpose =3D args.get('purpose') + definition =3D args.get('definition') + sectionlist =3D args.get('sectionlist', []) + parameterlist =3D args.get('parameterlist', []) + sections =3D args.get('sections', {}) + parameterdescs =3D args.get('parameterdescs', {}) + + print(f'.TH "{module}" 9 "{struct_type} {struct_name}" "{self.man_= date}" "API Manual" LINUX') + + print(".SH NAME") + print(f"{struct_type} {struct_name} \\- {purpose}") + + # Replace tabs with two spaces and handle newlines + declaration =3D definition.replace("\t", " ") + declaration =3D Re(r"\n").sub('"\n.br\n.BI "', declaration) + + print(".SH SYNOPSIS") + print(f"{struct_type} {struct_name} " + "{" +"\n.br") + print(f'.BI "{declaration}\n' + "};\n.br\n") + + print(".SH Members") + for parameter in parameterlist: + if parameter.startswith("#"): + continue + + parameter_name =3D re.sub(r"\[.*", "", parameter) + + if parameterdescs.get(parameter_name) =3D=3D KernelDoc.undescr= ibed: + continue + + print(f'.IP "{parameter}" 12') + self.output_highlight(parameterdescs.get(parameter_name)) + + for section in sectionlist: + print(f'.SH "{section}"') + self.output_highlight(sections.get(section)) + + +# Command line interface + + +DESC =3D """ +Read C language source or header FILEs, extract embedded documentation com= ments, +and print formatted documentation to standard output. + +The documentation comments are identified by the "/**" opening comment mar= k. + +See Documentation/doc-guide/kernel-doc.rst for the documentation comment s= yntax. +""" + +EXPORT_FILE_DESC =3D """ +Specify an additional FILE in which to look for EXPORT_SYMBOL information. + +May be used multiple times. +""" + +EXPORT_DESC =3D """ +Only output documentation for the symbols that have been +exported using EXPORT_SYMBOL() and related macros in any input +FILE or -export-file FILE. +""" + +INTERNAL_DESC =3D """ +Only output documentation for the symbols that have NOT been +exported using EXPORT_SYMBOL() and related macros in any input +FILE or -export-file FILE. +""" + +FUNCTION_DESC =3D """ +Only output documentation for the given function or DOC: section +title. All other functions and DOC: sections are ignored. + +May be used multiple times. +""" + +NOSYMBOL_DESC =3D """ +Exclude the specified symbol from the output documentation. + +May be used multiple times. +""" + +FILES_DESC =3D """ +Header and C source files to be parsed. +""" + +WARN_CONTENTS_BEFORE_SECTIONS_DESC =3D """ +Warns if there are contents before sections (deprecated). + +This option is kept just for backward-compatibility, but it does nothing, +neither here nor at the original Perl script. +""" + + +def main(): + """Main program""" + + parser =3D argparse.ArgumentParser(formatter_class=3Dargparse.RawTextH= elpFormatter, + description=3DDESC) + + # Normal arguments + + parser.add_argument("-v", "-verbose", "--verbose", action=3D"store_tru= e", + help=3D"Verbose output, more warnings and other in= formation.") + + parser.add_argument("-d", "-debug", "--debug", action=3D"store_true", + help=3D"Enable debug messages") + + parser.add_argument("-M", "-modulename", "--modulename", + help=3D"Allow setting a module name at the output.= ") + + parser.add_argument("-l", "-enable-lineno", "--enable_lineno", + action=3D"store_true", + help=3D"Enable line number output (only in ReST mo= de)") + + # Arguments to control the warning behavior + + parser.add_argument("-Wreturn", "--wreturn", action=3D"store_true", + help=3D"Warns about the lack of a return markup on= functions.") + + parser.add_argument("-Wshort-desc", "-Wshort-description", "--wshort-d= esc", + action=3D"store_true", + help=3D"Warns if initial short description is miss= ing") + + parser.add_argument("-Wcontents-before-sections", + "--wcontents-before-sections", action=3D"store_tru= e", + help=3DWARN_CONTENTS_BEFORE_SECTIONS_DESC) + + parser.add_argument("-Wall", "--wall", action=3D"store_true", + help=3D"Enable all types of warnings") + + parser.add_argument("-Werror", "--werror", action=3D"store_true", + help=3D"Treat warnings as errors.") + + parser.add_argument("-export-file", "--export-file", action=3D'append', + help=3DEXPORT_FILE_DESC) + + # Output format mutually-exclusive group + + out_group =3D parser.add_argument_group("Output format selection (mutu= ally exclusive)") + + out_fmt =3D out_group.add_mutually_exclusive_group() + + out_fmt.add_argument("-m", "-man", "--man", action=3D"store_true", + help=3D"Output troff manual page format.") + out_fmt.add_argument("-r", "-rst", "--rst", action=3D"store_true", + help=3D"Output reStructuredText format (default).= ") + out_fmt.add_argument("-N", "-none", "--none", action=3D"store_true", + help=3D"Do not output documentation, only warning= s.") + + # Output selection mutually-exclusive group + + sel_group =3D parser.add_argument_group("Output selection (mutually ex= clusive)") + sel_mut =3D sel_group.add_mutually_exclusive_group() + + sel_mut.add_argument("-e", "-export", "--export", action=3D'store_true= ', + help=3DEXPORT_DESC) + + sel_mut.add_argument("-i", "-internal", "--internal", action=3D'store_= true', + help=3DINTERNAL_DESC) + + sel_mut.add_argument("-s", "-function", "--symbol", action=3D'append', + help=3DFUNCTION_DESC) + + # This one is valid for all 3 types of filter + parser.add_argument("-n", "-nosymbol", "--nosymbol", action=3D'append', + help=3DNOSYMBOL_DESC) + + parser.add_argument("files", metavar=3D"FILE", + nargs=3D"+", help=3DFILES_DESC) + + args =3D parser.parse_args() + + if args.wall: + args.wreturn =3D True + args.wshort_desc =3D True + args.wcontents_before_sections =3D True + + if not args.debug: + level =3D logging.INFO + else: + level =3D logging.DEBUG + + if args.man: + out_style =3D ManFormat() + elif args.none: + out_style =3D None + else: + out_style =3D RestFormat() + + logging.basicConfig(level=3Dlevel, format=3D"%(levelname)s: %(message)= s") + + kfiles =3D KernelFiles(files=3Dargs.files, verbose=3Dargs.verbose, + out_style=3Dout_style, werror=3Dargs.werror, + wreturn=3Dargs.wreturn, wshort_desc=3Dargs.wshort= _desc, + wcontents_before_sections=3Dargs.wcontents_before= _sections, + modulename=3Dargs.modulename, + export_file=3Dargs.export_file) + + kfiles.parse() + + kfiles.msg(enable_lineno=3Dargs.enable_lineno, export=3Dargs.export, + internal=3Dargs.internal, symbol=3Dargs.symbol, + nosymbol=3Dargs.nosymbol) + + +# Call main method +if __name__ =3D=3D "__main__": + main() --=20 2.48.1 From nobody Sat Feb 7 14:01:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6490F1D47A2; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; cv=none; b=nacHvk1YJGbFtiR42hKxAxbA61m2QPaHMs6CqEMfhZCeMjjc9RT5YpEfUOEd020+2dRJl2XF7v1R35UwhNhYj0ZC8DoMHa55u65CYMwCpB0TXDPwbuwhqj3WD5Qlk6YHx2OVlGDFltwlEkS+YCv2fi5xOjykoTP1D7Kj4b7ohEI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; c=relaxed/simple; bh=lpGcciKaov3Y13LlsgFK4uZcLIWPLc8PIxEf/5bFm3o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=g59qH3jLSGKqJJvU7KJfQkvukIihlaQtTVHiw3FlG0POhMnGvnSZPQ9xDtIuIKhMWVw6j5xcZ+L0rK/WAV0DnSzJiK+XAHL35I0LHPZ8kG8BOl9NOZz3MdW8e/0+r3dAm1z4dybChoeg5TOLOtsFxK9es+0FWToe7TrT0f4x14Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=OLuEhCTU; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="OLuEhCTU" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2B566C4AF09; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739953972; bh=lpGcciKaov3Y13LlsgFK4uZcLIWPLc8PIxEf/5bFm3o=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OLuEhCTUybix+m2Y9LwAyWRufBCiM+8Y1cLOCgDRRY8P6AWTCxJG3LSZVjVscaP/u w9krR0tO/34P332mfixPuCHGzzzeSTnzpb3eUt5FmOrocIuWxr6aV4EoPe3jxBKqxR PKk2/lJpUjFhRf1Y8Qrwzl7wtBiZ1fETzj3g0f/HqZ4+0JGs+PKzjXrhk/7FI68nOB /wm/1votpgBXitPuT6s5yqK4/XK0FKSFjVtODTnFoPwfEKtoh9hYHvGrRfqKXgce5A HwNElk2v/o0FMvmUxSoi9+tfMc8BqmgI8BefFEXnoWz1Lap+/L7jADSj5N3zTOT1yb gMmDlvaJnJCBQ== Received: from mchehab by mail.kernel.org with local (Exim 4.98) (envelope-from ) id 1tkfVi-0000000Gv4v-0oY1; Wed, 19 Feb 2025 09:32:50 +0100 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , "Mauro Carvalho Chehab" , linux-kernel@vger.kernel.org Subject: [PATCH 10/27] scripts/kernel-doc.py: output warnings the same way as kerneldoc Date: Wed, 19 Feb 2025 09:32:26 +0100 Message-ID: <5152600de87339a70576f952cba67a84131003c0.1739952783.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" Add a formatter to logging to produce outputs in a similar way to kernel-doc. This should help making it more compatible with existing scripts. Signed-off-by: Mauro Carvalho Chehab --- scripts/kernel-doc.py | 20 ++++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-) diff --git a/scripts/kernel-doc.py b/scripts/kernel-doc.py index 5cf5ed63f215..8bc0470d3720 100755 --- a/scripts/kernel-doc.py +++ b/scripts/kernel-doc.py @@ -2640,6 +2640,11 @@ neither here nor at the original Perl script. """ =20 =20 +class MsgFormatter(logging.Formatter): + def format(self, record): + record.levelname =3D record.levelname.capitalize() + return logging.Formatter.format(self, record) + def main(): """Main program""" =20 @@ -2724,10 +2729,19 @@ def main(): args.wshort_desc =3D True args.wcontents_before_sections =3D True =20 + logger =3D logging.getLogger() + if not args.debug: - level =3D logging.INFO + logger.setLevel(logging.INFO) else: - level =3D logging.DEBUG + logger.setLevel(logging.DEBUG) + + formatter =3D MsgFormatter('%(levelname)s: %(message)s') + + handler =3D logging.StreamHandler() + handler.setFormatter(formatter) + + logger.addHandler(handler) =20 if args.man: out_style =3D ManFormat() @@ -2736,8 +2750,6 @@ def main(): else: out_style =3D RestFormat() =20 - logging.basicConfig(level=3Dlevel, format=3D"%(levelname)s: %(message)= s") - kfiles =3D KernelFiles(files=3Dargs.files, verbose=3Dargs.verbose, out_style=3Dout_style, werror=3Dargs.werror, wreturn=3Dargs.wreturn, wshort_desc=3Dargs.wshort= _desc, --=20 2.48.1 From nobody Sat Feb 7 14:01:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 832921D6DB1; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; cv=none; b=DlfPWahJZLtgapmj4NTWkSKS1y5djnfrVcehUYDhqIsFanqQo2s8sI+stwXYuIH2cEMTsJyx11jdIWWFhQFVt9jTcT4Al26a/xMwQaYkMNH5+6cJewpEL7hVcwSNCSIA7scujU8TuTt8g9VNkpPopHMlWgYJatBX1tSpf0emsos= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; c=relaxed/simple; bh=g3yjKUYPJng0/MFnsVcxoDKnYWevbS21H1bMLGz3lcY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=jrepDbew+aI4sL8WRc8YJ6vTWOymavLI397x38AeB/biG93VWLJa3mzttl76q7IorIDUHa/CQBvgSSJAOOd+j99ELJ5fZBITdhknfb2gl3NYNZmAT7RHEl7NNC9zwWgywgAF7wSnD2Wx1tgj9qG24uWRz0TG2zWzoWqNR8+3FT0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=pocYnLOh; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="pocYnLOh" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 351A9C4CEFC; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739953972; bh=g3yjKUYPJng0/MFnsVcxoDKnYWevbS21H1bMLGz3lcY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pocYnLOhrSNnAbRY//C4qPP93xioq+v0SfxjfF/Ra+E+vYPx6tReflGH46n1XO14Z qtPFMfhiA9fdtfwZ99X0nlaCPM11queFP5k67f2SeGRA5QDU+WD7fKH3FlGQwYLXJf PbNyTeJZzzBTPN8KocKUh1M2sdIzN0cbRBUD7179O5neDdxew26ysYSkVhYb1ngz97 x8fCqdNDi3bmavaXoTO2viLGAVo1E2fghEGPrKWZ9a5vMwLIh3gMEnEYxmz0QEbmt8 KTr1ScQdthVgSJf26FiCUDmYwQNwH2CsKXf7VPlFSacHijCHCa9i2vYkeTMViKM7KC u41zIorP7f9+g== Received: from mchehab by mail.kernel.org with local (Exim 4.98) (envelope-from ) id 1tkfVi-0000000Gv4z-0vWf; Wed, 19 Feb 2025 09:32:50 +0100 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , "Mauro Carvalho Chehab" , linux-kernel@vger.kernel.org Subject: [PATCH 11/27] scripts/kernel-doc.py: better handle empty sections Date: Wed, 19 Feb 2025 09:32:27 +0100 Message-ID: <2de0e2ced74c8d57810b83c5efc9e8e1555f0a63.1739952783.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" While doing the conversion, we opted to skip empty sections (description, return), but this makes harder to see the differences between kernel-doc (Perl) and kernel-doc.py. Also, the logic doesn't always work properly. So, change the way this is done by adding an extra step to remove such sections, doing it only for Return and Description. Signed-off-by: Mauro Carvalho Chehab --- scripts/kernel-doc.py | 31 ++++++++++++++++++++++++++++--- 1 file changed, 28 insertions(+), 3 deletions(-) diff --git a/scripts/kernel-doc.py b/scripts/kernel-doc.py index 8bc0470d3720..886256e87692 100755 --- a/scripts/kernel-doc.py +++ b/scripts/kernel-doc.py @@ -242,6 +242,19 @@ class KernelDoc: name =3D self.entry.section contents =3D self.entry.contents =20 + # TODO: we can prevent dumping empty sections here with: + # + # if self.entry.contents.strip("\n"): + # if start_new: + # self.entry.section =3D self.section_default + # self.entry.contents =3D "" + # + # return + # + # But, as we want to be producing the same output of the + # venerable kernel-doc Perl tool, let's just output everything, + # at least for now + if type_param.match(name): name =3D type_param.group(1) =20 @@ -298,6 +311,19 @@ class KernelDoc: =20 args["type"] =3D dtype =20 + # TODO: use colletions.OrderedDict + + sections =3D args.get('sections', {}) + sectionlist =3D args.get('sectionlist', []) + + # Drop empty sections + # TODO: improve it to emit warnings + for section in [ "Description", "Return" ]: + if section in sectionlist: + if not sections[section].rstrip(): + del sections[section] + sectionlist.remove(section) + self.entries.append((name, args)) =20 self.config.log.debug("Output: %s:%s =3D %s", dtype, name, pformat= (args)) @@ -401,7 +427,7 @@ class KernelDoc: # to ignore "[blah" in a parameter string. =20 self.entry.parameterlist.append(param) - org_arg =3D Re(r'\s\s+').sub(' ', org_arg, count=3D1) + org_arg =3D Re(r'\s\s+').sub(' ', org_arg) self.entry.parametertypes[param] =3D org_arg =20 def save_struct_actual(self, actual): @@ -1309,8 +1335,7 @@ class KernelDoc: return =20 if doc_end.search(line): - if self.entry.contents.strip("\n"): - self.dump_section() + self.dump_section() =20 # Look for doc_com + + doc_end: r =3D Re(r'\s*\*\s*[a-zA-Z_0-9:\.]+\*/') --=20 2.48.1 From nobody Sat Feb 7 14:01:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C80F21DDC3F; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; cv=none; b=aXa92H2ov+oXZwtjomSa+364EH0HiMcq8slIyNA019PQVx/lx/T7xDu0bBoYsVF56q6XAVwcxZmdmgseEyIkFeZpSOMZYj1Qjv4gRVggsPyGSugeFZn8yj1cwDB211nelK4JenNk2R8tXN3TAhDUuhglAvxgA0rtf4SQBVccOuc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; c=relaxed/simple; bh=TgQCSO8WmP+NxroK7/npEWybYRuZ/nrFGN7hk4txen8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=iw5HSCpA/UtwJYCzJs010ydWQrAZxVSIIwh/c6N0Ti5Rh2hAvtx2w5EoWo97PpkhgO4qWujN4SHkTxLRclQpFzbFlN1tcjTXLW/FriQHqv+SrCc77/Yj5awkdRBRwf6cLkW9Kei/0/TqulD0uISR8jsuBcHcUXS220lKaWm0jg4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=SQY+mera; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="SQY+mera" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2880FC4CEFB; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739953972; bh=TgQCSO8WmP+NxroK7/npEWybYRuZ/nrFGN7hk4txen8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=SQY+meraLhiiIS43VnHcrXj5TpZN50Nt1FYvoiQoOUT3AdjFQ9aPpwAJiIqTZfaTC ZftGPjS3yXV0yCE3CBInID66pr2Lg3iLeJKPwy6Es3FNsNXmqoHozST9HJxFFaRXut K83aoBHAeVI196KKkOBd27p6eoACY+KKFx0l1qPz1xWL7IO7ZcYgfNxd1EXersQPpQ 6sZUc5TdHsJYmm3V0wjhtykaSG/j/jVWaeVikWld3H/vDBSSbbP6can+ivOrA9L84E QPMwHYP2GBsW2BLg28Lyjsf/e0i3iKzDLYYXgep9RLQeNjtrL18NuWpQQGUVcxK0ug I/eyHOT3XmtZA== Received: from mchehab by mail.kernel.org with local (Exim 4.98) (envelope-from ) id 1tkfVi-0000000Gv53-12So; Wed, 19 Feb 2025 09:32:50 +0100 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , "Mauro Carvalho Chehab" , linux-kernel@vger.kernel.org Subject: [PATCH 12/27] scripts/kernel-doc.py: properly handle struct_group macros Date: Wed, 19 Feb 2025 09:32:28 +0100 Message-ID: X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" Handing nested parenthesis with regular expressions is not an easy task. It is even harder with Python's re module, as it has a limited subset of regular expressions, missing more advanced features. We might use instead Python regex module, but still the regular expressions are very hard to understand. So, instead, add a logic to properly match delimiters. Signed-off-by: Mauro Carvalho Chehab --- scripts/kernel-doc.py | 220 ++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 213 insertions(+), 7 deletions(-) diff --git a/scripts/kernel-doc.py b/scripts/kernel-doc.py index 886256e87692..3decc94edb1e 100755 --- a/scripts/kernel-doc.py +++ b/scripts/kernel-doc.py @@ -92,6 +92,172 @@ class Re: def group(self, num): return self.last_match.group(num) =20 +class NestedMatch: + """ + Finding nested delimiters is hard with regular expressions. It is + even harder on Python with its normal re module, as there are several + advanced regular expressions that are missing. + + This is the case of this pattern: + + '\\bSTRUCT_GROUP(\\(((?:(?>[^)(]+)|(?1))*)\\))[^;]*;' + + which is used to properly match open/close parenthesis of the + string search STRUCT_GROUP(), + + Add a class that counts pairs of delimiters, using it to match and + replace nested expressions. + + The original approach was suggested by: + https://stackoverflow.com/questions/5454322/python-how-to-match-ne= sted-parentheses-with-regex + + Although I re-implemented it to make it more generic and match 3 types + of delimiters. The logic checks if delimiters are paired. If not, it + will ignore the search string. + """ + + # TODO: + # Right now, regular expressions to match it are defined only up to + # the start delimiter, e.g.: + # + # \bSTRUCT_GROUP\( + # + # is similar to: STRUCT_GROUP\((.*)\) + # except that the content inside the match group is delimiter's aligne= d. + # + # The content inside parenthesis are converted into a single replace + # group (e.g. r`\1'). + # + # It would be nice to change such definition to support multiple + # match groups, allowing a regex equivalent to. + # + # FOO\((.*), (.*), (.*)\) + # + # it is probably easier to define it not as a regular expression, but + # with some lexical definition like: + # + # FOO(arg1, arg2, arg3) + + + DELIMITER_PAIRS =3D { + '{': '}', + '(': ')', + '[': ']', + } + + RE_DELIM =3D re.compile(r'[\{\}\[\]\(\)]') + + def _search(self, regex, line): + """ + Finds paired blocks for a regex that ends with a delimiter. + + The suggestion of using finditer to match pairs came from: + https://stackoverflow.com/questions/5454322/python-how-to-match-ne= sted-parentheses-with-regex + but I ended using a different implementation to align all three ty= pes + of delimiters and seek for an initial regular expression. + + The algorithm seeks for open/close paired delimiters and place them + into a stack, yielding a start/stop position of each match when t= he + stack is zeroed. + + The algorithm shoud work fine for properly paired lines, but will + silently ignore end delimiters that preceeds an start delimiter. + This should be OK for kernel-doc parser, as unaligned delimiters + would cause compilation errors. So, we don't need to rise exceptio= ns + to cover such issues. + """ + + stack =3D [] + + for match_re in regex.finditer(line): + start =3D match_re.start() + offset =3D match_re.end() + + d =3D line[offset -1] + if d not in self.DELIMITER_PAIRS: + continue + + end =3D self.DELIMITER_PAIRS[d] + stack.append(end) + + for match in self.RE_DELIM.finditer(line[offset:]): + pos =3D match.start() + offset + + d =3D line[pos] + + if d in self.DELIMITER_PAIRS: + end =3D self.DELIMITER_PAIRS[d] + + stack.append(end) + continue + + # Does the end delimiter match what it is expected? + if stack and d =3D=3D stack[-1]: + stack.pop() + + if not stack: + yield start, offset, pos + 1 + break + + def search(self, regex, line): + """ + This is similar to re.search: + + It matches a regex that it is followed by a delimiter, + returning occurrences only if all delimiters are paired. + """ + + for t in self._search(regex, line): + + yield line[t[0]:t[2]] + + def sub(self, regex, sub, line, count=3D0): + """ + This is similar to re.sub: + + It matches a regex that it is followed by a delimiter, + replacing occurrences only if all delimiters are paired. + + if r'\1' is used, it works just like re: it places there the + matched paired data with the delimiter stripped. + + If count is different than zero, it will replace at most count + items. + """ + out =3D "" + + cur_pos =3D 0 + n =3D 0 + + found =3D False + for start, end, pos in self._search(regex, line): + out +=3D line[cur_pos:start] + + # Value, ignoring start/end delimiters + value =3D line[end:pos - 1] + + # replaces \1 at the sub string, if \1 is used there + new_sub =3D sub + new_sub =3D new_sub.replace(r'\1', value) + + out +=3D new_sub + + # Drop end ';' if any + if line[pos] =3D=3D ';': + pos +=3D 1 + + cur_pos =3D pos + n +=3D 1 + + if count and count >=3D n: + break + + # Append the remaining string + l =3D len(line) + out +=3D line[cur_pos:l] + + return out + # # Regular expressions used to parse kernel-doc markups at KernelDoc class. # @@ -663,22 +829,49 @@ class KernelDoc: (Re(r'\s*____cacheline_aligned_in_smp', re.S), ' '), (Re(r'\s*____cacheline_aligned', re.S), ' '), =20 - # Unwrap struct_group() based on this definition: + # Unwrap struct_group macros based on this definition: # __struct_group(TAG, NAME, ATTRS, MEMBERS...) # which has variants like: struct_group(NAME, MEMBERS...) + # Only MEMBERS arguments require documentation. + # + # Parsing them happens on two steps: + # + # 1. drop struct group arguments that aren't at MEMBERS, + # storing them as STRUCT_GROUP(MEMBERS) + # + # 2. remove STRUCT_GROUP() ancillary macro. + # + # The original logic used to remove STRUCT_GROUP() using an + # advanced regex: + # + # \bSTRUCT_GROUP(\(((?:(?>[^)(]+)|(?1))*)\))[^;]*; + # + # with two patterns that are incompatible with + # Python re module, as it has: + # + # - a recursive pattern: (?1) + # - an atomic grouping: (?>...) + # + # I tried a simpler version: but it didn't work either: + # \bSTRUCT_GROUP\(([^\)]+)\)[^;]*; + # + # As it doesn't properly match the end parenthesis on some cas= es. + # + # So, a better solution was crafted: there's now a NestedMatch + # class that ensures that delimiters after a search are proper= ly + # matched. So, the implementation to drop STRUCT_GROUP() will = be + # handled in separate. =20 (Re(r'\bstruct_group\s*\(([^,]*,)', re.S), r'STRUCT_GROUP('), (Re(r'\bstruct_group_attr\s*\(([^,]*,){2}', re.S), r'STRUCT_G= ROUP('), (Re(r'\bstruct_group_tagged\s*\(([^,]*),([^,]*),', re.S), r's= truct \1 \2; STRUCT_GROUP('), (Re(r'\b__struct_group\s*\(([^,]*,){3}', re.S), r'STRUCT_GROU= P('), =20 - # This is incompatible with Python re, as it uses: - # recursive patterns ((?1)) and atomic grouping ((?>...)): - # '\bSTRUCT_GROUP(\(((?:(?>[^)(]+)|(?1))*)\))[^;]*;' - # Let's see if this works instead: - (Re(r'\bSTRUCT_GROUP\(([^\)]+)\)[^;]*;', re.S), r'\1'), - # Replace macros + # + # TODO: it is better to also move those to the NestedMatch log= ic, + # to ensure that parenthesis will be properly matched. + (Re(r'__ETHTOOL_DECLARE_LINK_MODE_MASK\s*\(([^\)]+)\)', re.S),= r'DECLARE_BITMAP(\1, __ETHTOOL_LINK_MODE_MASK_NBITS)'), (Re(r'DECLARE_PHY_INTERFACE_MASK\s*\(([^\)]+)\)', re.S), r'DE= CLARE_BITMAP(\1, PHY_INTERFACE_MODE_MAX)'), (Re(r'DECLARE_BITMAP\s*\(' + args_pattern + r',\s*' + args_pat= tern + r'\)', re.S), r'unsigned long \1[BITS_TO_LONGS(\2)]'), @@ -690,9 +883,22 @@ class KernelDoc: (Re(r'DEFINE_DMA_UNMAP_LEN\s*\(' + args_pattern + r'\)', re.S)= , r'__u32 \1'), ] =20 + # Regexes here are guaranteed to have the end limiter matching + # the start delimiter. Yet, right now, only one replace group + # is allowed. + + sub_nested_prefixes =3D [ + (re.compile(r'\bSTRUCT_GROUP\('), r'\1'), + ] + for search, sub in sub_prefixes: members =3D search.sub(sub, members) =20 + nested =3D NestedMatch() + + for search, sub in sub_nested_prefixes: + members =3D nested.sub(search, sub, members) + # Keeps the original declaration as-is declaration =3D members =20 --=20 2.48.1 From nobody Sat Feb 7 14:01:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D438A1DE2BE; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; cv=none; b=G57din5didBDB+eall6tg8qCvAAKxSB6sGReDvsVmaNnJ/prtwUEFnVTLuoUcvEuhdGQPkX7YCHfJtxHb4ZC8Exh9Xolj0QLNj4mt+n+xerv8kQVp4dNdA93wmHrPCiwPkhe/ZXbsdiun0Pw776fMZMQ3J5g4M7+ZZDV/ReDC1M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; c=relaxed/simple; bh=2to40pLeg8q0+VRpVnj4qXIaza6aM03uTe1Bu/KQtIo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=LdIjkK3sA31gshmf8kmU/IOsunKES0Rzrlenf6RoxqOHqWrh7woRhEkggIQRRpKH5AcidYtoFXL4uUhgaQL5s7NXkdSnTcvruNYs/fSz/VnQzGYB/JgbAWjSwVg6lq0xnR0AwqsLwV7zh2iFTbdQQWlibLm9ruCd3fPzUkHNiYA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=CHM6CJLy; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="CHM6CJLy" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2D06DC4CEFA; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739953972; bh=2to40pLeg8q0+VRpVnj4qXIaza6aM03uTe1Bu/KQtIo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CHM6CJLyZmC3+lgPADbu6GYg8HsLe7dTLfULnlNN5LWrsQUHv9zamtDEElrGCA3GL DJeVTYupRdvs6WBn/x9cMgOiz0sjE4qC1Cip/tnpjYELs86GMP3j1vcAC7sXJ4F/b0 b939th7cnQzMU8SObA/4WTOSOKCrNlY+qfR+v389KULStqlJJDe3n5/8frbEy3r2vc 2b9yeottJFaJMmdwLApAqh52mhj6TFA3R7Vow0v6nYvLsepB4yf95rUKY8WZ2L631U Tdu+Sb3/rLNGIxfZw/aDkEgTUABBaWACT66f0ACUZLu6ffZ+3YvpFastegpMk5kiI6 ZBq2cEygKyZDg== Received: from mchehab by mail.kernel.org with local (Exim 4.98) (envelope-from ) id 1tkfVi-0000000Gv57-19rA; Wed, 19 Feb 2025 09:32:50 +0100 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , "Mauro Carvalho Chehab" , linux-kernel@vger.kernel.org Subject: [PATCH 13/27] scripts/kernel-doc.py: move regex methods to a separate file Date: Wed, 19 Feb 2025 09:32:29 +0100 Message-ID: <518d077882e30f07923f8d746ba5fa0e4429c361.1739952783.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" In preparation for letting kerneldoc Sphinx extension to import Python libraries, move regex ancillary classes to a separate file. Signed-off-by: Mauro Carvalho Chehab --- scripts/kernel-doc.py | 223 +---------------------------- scripts/lib/kdoc/kdoc_re.py | 272 ++++++++++++++++++++++++++++++++++++ 2 files changed, 277 insertions(+), 218 deletions(-) create mode 100755 scripts/lib/kdoc/kdoc_re.py diff --git a/scripts/kernel-doc.py b/scripts/kernel-doc.py index 3decc94edb1e..faae66aa6ead 100755 --- a/scripts/kernel-doc.py +++ b/scripts/kernel-doc.py @@ -35,228 +35,15 @@ from pprint import pformat =20 from dateutil import tz =20 -# Local cache for regular expressions -re_cache =3D {} +# Import Python modules =20 +LIB_DIR =3D "lib/kdoc" +SRC_DIR =3D os.path.dirname(os.path.realpath(__file__)) =20 -class Re: - """ - Helper class to simplify regex declaration and usage, +sys.path.insert(0, os.path.join(SRC_DIR, LIB_DIR)) =20 - It calls re.compile for a given pattern. It also allows adding - regular expressions and define sub at class init time. +from kdoc_re import Re, NestedMatch =20 - Regular expressions can be cached via an argument, helping to speedup - searches. - """ - - def _add_regex(self, string, flags): - if string in re_cache: - self.regex =3D re_cache[string] - else: - self.regex =3D re.compile(string, flags=3Dflags) - - if self.cache: - re_cache[string] =3D self.regex - - def __init__(self, string, cache=3DTrue, flags=3D0): - self.cache =3D cache - self.last_match =3D None - - self._add_regex(string, flags) - - def __str__(self): - return self.regex.pattern - - def __add__(self, other): - return Re(str(self) + str(other), cache=3Dself.cache or other.cach= e, - flags=3Dself.regex.flags | other.regex.flags) - - def match(self, string): - self.last_match =3D self.regex.match(string) - return self.last_match - - def search(self, string): - self.last_match =3D self.regex.search(string) - return self.last_match - - def findall(self, string): - return self.regex.findall(string) - - def split(self, string): - return self.regex.split(string) - - def sub(self, sub, string, count=3D0): - return self.regex.sub(sub, string, count=3Dcount) - - def group(self, num): - return self.last_match.group(num) - -class NestedMatch: - """ - Finding nested delimiters is hard with regular expressions. It is - even harder on Python with its normal re module, as there are several - advanced regular expressions that are missing. - - This is the case of this pattern: - - '\\bSTRUCT_GROUP(\\(((?:(?>[^)(]+)|(?1))*)\\))[^;]*;' - - which is used to properly match open/close parenthesis of the - string search STRUCT_GROUP(), - - Add a class that counts pairs of delimiters, using it to match and - replace nested expressions. - - The original approach was suggested by: - https://stackoverflow.com/questions/5454322/python-how-to-match-ne= sted-parentheses-with-regex - - Although I re-implemented it to make it more generic and match 3 types - of delimiters. The logic checks if delimiters are paired. If not, it - will ignore the search string. - """ - - # TODO: - # Right now, regular expressions to match it are defined only up to - # the start delimiter, e.g.: - # - # \bSTRUCT_GROUP\( - # - # is similar to: STRUCT_GROUP\((.*)\) - # except that the content inside the match group is delimiter's aligne= d. - # - # The content inside parenthesis are converted into a single replace - # group (e.g. r`\1'). - # - # It would be nice to change such definition to support multiple - # match groups, allowing a regex equivalent to. - # - # FOO\((.*), (.*), (.*)\) - # - # it is probably easier to define it not as a regular expression, but - # with some lexical definition like: - # - # FOO(arg1, arg2, arg3) - - - DELIMITER_PAIRS =3D { - '{': '}', - '(': ')', - '[': ']', - } - - RE_DELIM =3D re.compile(r'[\{\}\[\]\(\)]') - - def _search(self, regex, line): - """ - Finds paired blocks for a regex that ends with a delimiter. - - The suggestion of using finditer to match pairs came from: - https://stackoverflow.com/questions/5454322/python-how-to-match-ne= sted-parentheses-with-regex - but I ended using a different implementation to align all three ty= pes - of delimiters and seek for an initial regular expression. - - The algorithm seeks for open/close paired delimiters and place them - into a stack, yielding a start/stop position of each match when t= he - stack is zeroed. - - The algorithm shoud work fine for properly paired lines, but will - silently ignore end delimiters that preceeds an start delimiter. - This should be OK for kernel-doc parser, as unaligned delimiters - would cause compilation errors. So, we don't need to rise exceptio= ns - to cover such issues. - """ - - stack =3D [] - - for match_re in regex.finditer(line): - start =3D match_re.start() - offset =3D match_re.end() - - d =3D line[offset -1] - if d not in self.DELIMITER_PAIRS: - continue - - end =3D self.DELIMITER_PAIRS[d] - stack.append(end) - - for match in self.RE_DELIM.finditer(line[offset:]): - pos =3D match.start() + offset - - d =3D line[pos] - - if d in self.DELIMITER_PAIRS: - end =3D self.DELIMITER_PAIRS[d] - - stack.append(end) - continue - - # Does the end delimiter match what it is expected? - if stack and d =3D=3D stack[-1]: - stack.pop() - - if not stack: - yield start, offset, pos + 1 - break - - def search(self, regex, line): - """ - This is similar to re.search: - - It matches a regex that it is followed by a delimiter, - returning occurrences only if all delimiters are paired. - """ - - for t in self._search(regex, line): - - yield line[t[0]:t[2]] - - def sub(self, regex, sub, line, count=3D0): - """ - This is similar to re.sub: - - It matches a regex that it is followed by a delimiter, - replacing occurrences only if all delimiters are paired. - - if r'\1' is used, it works just like re: it places there the - matched paired data with the delimiter stripped. - - If count is different than zero, it will replace at most count - items. - """ - out =3D "" - - cur_pos =3D 0 - n =3D 0 - - found =3D False - for start, end, pos in self._search(regex, line): - out +=3D line[cur_pos:start] - - # Value, ignoring start/end delimiters - value =3D line[end:pos - 1] - - # replaces \1 at the sub string, if \1 is used there - new_sub =3D sub - new_sub =3D new_sub.replace(r'\1', value) - - out +=3D new_sub - - # Drop end ';' if any - if line[pos] =3D=3D ';': - pos +=3D 1 - - cur_pos =3D pos - n +=3D 1 - - if count and count >=3D n: - break - - # Append the remaining string - l =3D len(line) - out +=3D line[cur_pos:l] - - return out =20 # # Regular expressions used to parse kernel-doc markups at KernelDoc class. diff --git a/scripts/lib/kdoc/kdoc_re.py b/scripts/lib/kdoc/kdoc_re.py new file mode 100755 index 000000000000..45ddba8090e5 --- /dev/null +++ b/scripts/lib/kdoc/kdoc_re.py @@ -0,0 +1,272 @@ +#!/usr/bin/env python3 +# Copyright(c) 2025: Mauro Carvalho Chehab . +# SPDX-License-Identifier: GPL-2.0 + +""" +Regular expression ancillary classes. + +Those help caching regular expressions and do matching for kernel-doc. +""" + +import re + +# Local cache for regular expressions +re_cache =3D {} + + +class Re: + """ + Helper class to simplify regex declaration and usage, + + It calls re.compile for a given pattern. It also allows adding + regular expressions and define sub at class init time. + + Regular expressions can be cached via an argument, helping to speedup + searches. + """ + + def _add_regex(self, string, flags): + """ + Adds a new regex or re-use it from the cache. + """ + + if string in re_cache: + self.regex =3D re_cache[string] + else: + self.regex =3D re.compile(string, flags=3Dflags) + + if self.cache: + re_cache[string] =3D self.regex + + def __init__(self, string, cache=3DTrue, flags=3D0): + """ + Compile a regular expression and initialize internal vars. + """ + + self.cache =3D cache + self.last_match =3D None + + self._add_regex(string, flags) + + def __str__(self): + """ + Return the regular expression pattern. + """ + return self.regex.pattern + + def __add__(self, other): + """ + Allows adding two regular expressions into one. + """ + + return Re(str(self) + str(other), cache=3Dself.cache or other.cach= e, + flags=3Dself.regex.flags | other.regex.flags) + + def match(self, string): + """ + Handles a re.match storing its results + """ + + self.last_match =3D self.regex.match(string) + return self.last_match + + def search(self, string): + """ + Handles a re.search storing its results + """ + + self.last_match =3D self.regex.search(string) + return self.last_match + + def findall(self, string): + """ + Alias to re.findall + """ + + return self.regex.findall(string) + + def split(self, string): + """ + Alias to re.split + """ + + return self.regex.split(string) + + def sub(self, sub, string, count=3D0): + """ + Alias to re.sub + """ + + return self.regex.sub(sub, string, count=3Dcount) + + def group(self, num): + """ + Returns the group results of the last match + """ + + return self.last_match.group(num) + + +class NestedMatch: + """ + Finding nested delimiters is hard with regular expressions. It is + even harder on Python with its normal re module, as there are several + advanced regular expressions that are missing. + + This is the case of this pattern: + + '\\bSTRUCT_GROUP(\\(((?:(?>[^)(]+)|(?1))*)\\))[^;]*;' + + which is used to properly match open/close parenthesis of the + string search STRUCT_GROUP(), + + Add a class that counts pairs of delimiters, using it to match and + replace nested expressions. + + The original approach was suggested by: + https://stackoverflow.com/questions/5454322/python-how-to-match-ne= sted-parentheses-with-regex + + Although I re-implemented it to make it more generic and match 3 types + of delimiters. The logic checks if delimiters are paired. If not, it + will ignore the search string. + """ + + # TODO: + # Right now, regular expressions to match it are defined only up to + # the start delimiter, e.g.: + # + # \bSTRUCT_GROUP\( + # + # is similar to: STRUCT_GROUP\((.*)\) + # except that the content inside the match group is delimiter's aligne= d. + # + # The content inside parenthesis are converted into a single replace + # group (e.g. r`\1'). + # + # It would be nice to change such definition to support multiple + # match groups, allowing a regex equivalent to. + # + # FOO\((.*), (.*), (.*)\) + # + # it is probably easier to define it not as a regular expression, but + # with some lexical definition like: + # + # FOO(arg1, arg2, arg3) + + DELIMITER_PAIRS =3D { + '{': '}', + '(': ')', + '[': ']', + } + + RE_DELIM =3D re.compile(r'[\{\}\[\]\(\)]') + + def _search(self, regex, line): + """ + Finds paired blocks for a regex that ends with a delimiter. + + The suggestion of using finditer to match pairs came from: + https://stackoverflow.com/questions/5454322/python-how-to-match-ne= sted-parentheses-with-regex + but I ended using a different implementation to align all three ty= pes + of delimiters and seek for an initial regular expression. + + The algorithm seeks for open/close paired delimiters and place them + into a stack, yielding a start/stop position of each match when t= he + stack is zeroed. + + The algorithm shoud work fine for properly paired lines, but will + silently ignore end delimiters that preceeds an start delimiter. + This should be OK for kernel-doc parser, as unaligned delimiters + would cause compilation errors. So, we don't need to rise exceptio= ns + to cover such issues. + """ + + stack =3D [] + + for match_re in regex.finditer(line): + start =3D match_re.start() + offset =3D match_re.end() + + d =3D line[offset - 1] + if d not in self.DELIMITER_PAIRS: + continue + + end =3D self.DELIMITER_PAIRS[d] + stack.append(end) + + for match in self.RE_DELIM.finditer(line[offset:]): + pos =3D match.start() + offset + + d =3D line[pos] + + if d in self.DELIMITER_PAIRS: + end =3D self.DELIMITER_PAIRS[d] + + stack.append(end) + continue + + # Does the end delimiter match what it is expected? + if stack and d =3D=3D stack[-1]: + stack.pop() + + if not stack: + yield start, offset, pos + 1 + break + + def search(self, regex, line): + """ + This is similar to re.search: + + It matches a regex that it is followed by a delimiter, + returning occurrences only if all delimiters are paired. + """ + + for t in self._search(regex, line): + + yield line[t[0]:t[2]] + + def sub(self, regex, sub, line, count=3D0): + """ + This is similar to re.sub: + + It matches a regex that it is followed by a delimiter, + replacing occurrences only if all delimiters are paired. + + if r'\1' is used, it works just like re: it places there the + matched paired data with the delimiter stripped. + + If count is different than zero, it will replace at most count + items. + """ + out =3D "" + + cur_pos =3D 0 + n =3D 0 + + for start, end, pos in self._search(regex, line): + out +=3D line[cur_pos:start] + + # Value, ignoring start/end delimiters + value =3D line[end:pos - 1] + + # replaces \1 at the sub string, if \1 is used there + new_sub =3D sub + new_sub =3D new_sub.replace(r'\1', value) + + out +=3D new_sub + + # Drop end ';' if any + if line[pos] =3D=3D ';': + pos +=3D 1 + + cur_pos =3D pos + n +=3D 1 + + if count and count >=3D n: + break + + # Append the remaining string + l =3D len(line) + out +=3D line[cur_pos:l] + + return out --=20 2.48.1 From nobody Sat Feb 7 14:01:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 103201DE3CE; Wed, 19 Feb 2025 08:32:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953973; cv=none; b=goJXZaysC+Sv0OJtUZDhbJphvV9ccWYmwq2FWBFVWRXAt7V+A8erDaBemxD0H4RzSHobm+bOW8MFKpLgSYEEG1A0AtOarTatM9RwZQhCHRLsPjLQcZNizHxvP46F7Zwkd8doPXDkOe17ieWqzTLGWoYDtdMM9RjnuJkD71HeQTY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953973; c=relaxed/simple; bh=oGmptKFwK8sfoy5nmo7fNyWjrlXUopST/S8FlqSqVmQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=LnYnhpRiEcirhAC7K3i4DMztZ9gyDyLO9JWgEo2sMf6vhKo55+8VDAPI0t8ZmuPxl5SqC10IzcYi/1JC0BcfJAji/dWf0Gwjhckit86SdkBQtz1Q4qoDkzD8YohwRDzZhNUy6qgxbStv846uv2QqT95vYQ5DqD8WQ6+MVu7+/sc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=O/a7HQGy; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="O/a7HQGy" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2A994C4CEF8; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739953972; bh=oGmptKFwK8sfoy5nmo7fNyWjrlXUopST/S8FlqSqVmQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=O/a7HQGye5VdIZ0IULRKlpIMEJqj1f1fgRjRnXRPyiTgwB1QUdU2BwxDyeboWc9Dy uyJ7/EQODBZRCXZL6jFL/IfThYmp0z0YXKANPdgcteINr0BK9ZTSTkDJpCtL4qvAQc ZgjjbsT0fd0PYqORvQleRJMj25E3JAqEYf1ILrrIB+RstW1iXNUHF6JF1N5OzKsVTT APnH9+lbT55caXQHKgQBX1OYZrnDMbg9IJVDFgVq1faIhMrFRGwFSDD3w2aryY1E/V +U/YSyXUgj3bv9GiM/icrOczWJ15bin5IHu8dpgGMd2PTqrH5D6dZJXfxcxobiI5cb 5CEnTiTZmWpTw== Received: from mchehab by mail.kernel.org with local (Exim 4.98) (envelope-from ) id 1tkfVi-0000000Gv5C-1IEj; Wed, 19 Feb 2025 09:32:50 +0100 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , "Gustavo A. R. Silva" , "Mauro Carvalho Chehab" , Kees Cook , linux-hardening@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 14/27] scripts/kernel-doc.py: move KernelDoc class to a separate file Date: Wed, 19 Feb 2025 09:32:30 +0100 Message-ID: X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" In preparation for letting kerneldoc Sphinx extension to import Python libraries, move regex ancillary classes to a separate file. Signed-off-by: Mauro Carvalho Chehab --- scripts/kernel-doc.py | 1634 +----------------------------- scripts/lib/kdoc/kdoc_parser.py | 1689 +++++++++++++++++++++++++++++++ 2 files changed, 1691 insertions(+), 1632 deletions(-) create mode 100755 scripts/lib/kdoc/kdoc_parser.py diff --git a/scripts/kernel-doc.py b/scripts/kernel-doc.py index faae66aa6ead..193a30fcfb7c 100755 --- a/scripts/kernel-doc.py +++ b/scripts/kernel-doc.py @@ -42,53 +42,15 @@ SRC_DIR =3D os.path.dirname(os.path.realpath(__file__)) =20 sys.path.insert(0, os.path.join(SRC_DIR, LIB_DIR)) =20 -from kdoc_re import Re, NestedMatch +from kdoc_parser import KernelDoc, type_param +from kdoc_re import Re =20 - -# -# Regular expressions used to parse kernel-doc markups at KernelDoc class. -# -# Let's declare them in lowercase outside any class to make easier to -# convert from the python script. -# -# As those are evaluated at the beginning, no need to cache them -# - - -# Allow whitespace at end of comment start. -doc_start =3D Re(r'^/\*\*\s*$', cache=3DFalse) - -doc_end =3D Re(r'\*/', cache=3DFalse) -doc_com =3D Re(r'\s*\*\s*', cache=3DFalse) -doc_com_body =3D Re(r'\s*\* ?', cache=3DFalse) -doc_decl =3D doc_com + Re(r'(\w+)', cache=3DFalse) - -# @params and a strictly limited set of supported section names -# Specifically: -# Match @word: -# @...: -# @{section-name}: -# while trying to not match literal block starts like "example::" -# -doc_sect =3D doc_com + \ - Re(r'\s*(\@[.\w]+|\@\.\.\.|description|context|returns?|notes?= |examples?)\s*:([^:].*)?$', - flags=3Dre.I, cache=3DFalse) - -doc_content =3D doc_com_body + Re(r'(.*)', cache=3DFalse) -doc_block =3D doc_com + Re(r'DOC:\s*(.*)?', cache=3DFalse) -doc_inline_start =3D Re(r'^\s*/\*\*\s*$', cache=3DFalse) -doc_inline_sect =3D Re(r'\s*\*\s*(@\s*[\w][\w\.]*\s*):(.*)', cache=3DFalse) -doc_inline_end =3D Re(r'^\s*\*/\s*$', cache=3DFalse) -doc_inline_oneline =3D Re(r'^\s*/\*\*\s*(@[\w\s]+):\s*(.*)\s*\*/\s*$', cac= he=3DFalse) function_pointer =3D Re(r"([^\(]*\(\*)\s*\)\s*\(([^\)]*)\)", cache=3DFalse) -attribute =3D Re(r"__attribute__\s*\(\([a-z0-9,_\*\s\(\)]*\)\)", - flags=3Dre.I | re.S, cache=3DFalse) =20 # match expressions used to find embedded type information type_constant =3D Re(r"\b``([^\`]+)``\b", cache=3DFalse) type_constant2 =3D Re(r"\%([-_*\w]+)", cache=3DFalse) type_func =3D Re(r"(\w+)\(\)", cache=3DFalse) -type_param =3D Re(r"\@(\w*((\.\w+)|(->\w+))*(\.\.\.)?)", cache=3DFalse) type_param_ref =3D Re(r"([\!~\*]?)\@(\w*((\.\w+)|(->\w+))*(\.\.\.)?)", cac= he=3DFalse) =20 # Special RST handling for func ptr params @@ -106,1598 +68,6 @@ type_member =3D Re(r"\&([_\w]+)(\.|->)([_\w]+)", cach= e=3DFalse) type_fallback =3D Re(r"\&([_\w]+)", cache=3DFalse) type_member_func =3D type_member + Re(r"\(\)", cache=3DFalse) =20 -export_symbol =3D Re(r'^\s*EXPORT_SYMBOL(_GPL)?\s*\(\s*(\w+)\s*\)\s*', cac= he=3DFalse) -export_symbol_ns =3D Re(r'^\s*EXPORT_SYMBOL_NS(_GPL)?\s*\(\s*(\w+)\s*,\s*"= \S+"\)\s*', cache=3DFalse) - -class KernelDoc: - # Parser states - STATE_NORMAL =3D 0 # normal code - STATE_NAME =3D 1 # looking for function name - STATE_BODY_MAYBE =3D 2 # body - or maybe more description - STATE_BODY =3D 3 # the body of the comment - STATE_BODY_WITH_BLANK_LINE =3D 4 # the body which has a blank line - STATE_PROTO =3D 5 # scanning prototype - STATE_DOCBLOCK =3D 6 # documentation block - STATE_INLINE =3D 7 # gathering doc outside main block - - st_name =3D [ - "NORMAL", - "NAME", - "BODY_MAYBE", - "BODY", - "BODY_WITH_BLANK_LINE", - "PROTO", - "DOCBLOCK", - "INLINE", - ] - - # Inline documentation state - STATE_INLINE_NA =3D 0 # not applicable ($state !=3D STATE_INLINE) - STATE_INLINE_NAME =3D 1 # looking for member name (@foo:) - STATE_INLINE_TEXT =3D 2 # looking for member documentation - STATE_INLINE_END =3D 3 # done - STATE_INLINE_ERROR =3D 4 # error - Comment without header was found. - # Spit a warning as it's not - # proper kernel-doc and ignore the rest. - - st_inline_name =3D [ - "", - "_NAME", - "_TEXT", - "_END", - "_ERROR", - ] - - # Section names - - section_default =3D "Description" # default section - section_intro =3D "Introduction" - section_context =3D "Context" - section_return =3D "Return" - - undescribed =3D "-- undescribed --" - - def __init__(self, config, fname): - """Initialize internal variables""" - - self.fname =3D fname - self.config =3D config - - # Initial state for the state machines - self.state =3D self.STATE_NORMAL - self.inline_doc_state =3D self.STATE_INLINE_NA - - # Store entry currently being processed - self.entry =3D None - - # Place all potential outputs into an array - self.entries =3D [] - - def show_warnings(self, dtype, declaration_name): - # TODO: implement it - - return True - - # TODO: rename to emit_message - def emit_warning(self, ln, msg, warning=3DTrue): - """Emit a message""" - - if warning: - self.config.log.warning("%s:%d %s", self.fname, ln, msg) - else: - self.config.log.info("%s:%d %s", self.fname, ln, msg) - - def dump_section(self, start_new=3DTrue): - """ - Dumps section contents to arrays/hashes intended for that purpose. - """ - - name =3D self.entry.section - contents =3D self.entry.contents - - # TODO: we can prevent dumping empty sections here with: - # - # if self.entry.contents.strip("\n"): - # if start_new: - # self.entry.section =3D self.section_default - # self.entry.contents =3D "" - # - # return - # - # But, as we want to be producing the same output of the - # venerable kernel-doc Perl tool, let's just output everything, - # at least for now - - if type_param.match(name): - name =3D type_param.group(1) - - self.entry.parameterdescs[name] =3D contents - self.entry.parameterdesc_start_lines[name] =3D self.entry.new_= start_line - - self.entry.sectcheck +=3D name + " " - self.entry.new_start_line =3D 0 - - elif name =3D=3D "@...": - name =3D "..." - self.entry.parameterdescs[name] =3D contents - self.entry.sectcheck +=3D name + " " - self.entry.parameterdesc_start_lines[name] =3D self.entry.new_= start_line - self.entry.new_start_line =3D 0 - - else: - if name in self.entry.sections and self.entry.sections[name] != =3D "": - # Only warn on user-specified duplicate section names - if name !=3D self.section_default: - self.emit_warning(self.entry.new_start_line, - f"duplicate section name '{name}'\n") - self.entry.sections[name] +=3D contents - else: - self.entry.sections[name] =3D contents - self.entry.sectionlist.append(name) - self.entry.section_start_lines[name] =3D self.entry.new_st= art_line - self.entry.new_start_line =3D 0 - -# self.config.log.debug("Section: %s : %s", name, pformat(vars(self= .entry))) - - if start_new: - self.entry.section =3D self.section_default - self.entry.contents =3D "" - - # TODO: rename it to store_declaration - def output_declaration(self, dtype, name, **args): - """ - Stores the entry into an entry array. - - The actual output and output filters will be handled elsewhere - """ - - # The implementation here is different than the original kernel-do= c: - # instead of checking for output filters or actually output anythi= ng, - # it just stores the declaration content at self.entries, as the - # output will happen on a separate class. - # - # For now, we're keeping the same name of the function just to make - # easier to compare the source code of both scripts - - if "declaration_start_line" not in args: - args["declaration_start_line"] =3D self.entry.declaration_star= t_line - - args["type"] =3D dtype - - # TODO: use colletions.OrderedDict - - sections =3D args.get('sections', {}) - sectionlist =3D args.get('sectionlist', []) - - # Drop empty sections - # TODO: improve it to emit warnings - for section in [ "Description", "Return" ]: - if section in sectionlist: - if not sections[section].rstrip(): - del sections[section] - sectionlist.remove(section) - - self.entries.append((name, args)) - - self.config.log.debug("Output: %s:%s =3D %s", dtype, name, pformat= (args)) - - def reset_state(self, ln): - """ - Ancillary routine to create a new entry. It initializes all - variables used by the state machine. - """ - - self.entry =3D argparse.Namespace - - self.entry.contents =3D "" - self.entry.function =3D "" - self.entry.sectcheck =3D "" - self.entry.struct_actual =3D "" - self.entry.prototype =3D "" - - self.entry.parameterlist =3D [] - self.entry.parameterdescs =3D {} - self.entry.parametertypes =3D {} - self.entry.parameterdesc_start_lines =3D {} - - self.entry.section_start_lines =3D {} - self.entry.sectionlist =3D [] - self.entry.sections =3D {} - - self.entry.anon_struct_union =3D False - - self.entry.leading_space =3D None - - # State flags - self.state =3D self.STATE_NORMAL - self.inline_doc_state =3D self.STATE_INLINE_NA - self.entry.brcount =3D 0 - - self.entry.in_doc_sect =3D False - self.entry.declaration_start_line =3D ln - - def push_parameter(self, ln, decl_type, param, dtype, - org_arg, declaration_name): - if self.entry.anon_struct_union and dtype =3D=3D "" and param =3D= =3D "}": - return # Ignore the ending }; from anonymous struct/union - - self.entry.anon_struct_union =3D False - - param =3D Re(r'[\[\)].*').sub('', param, count=3D1) - - if dtype =3D=3D "" and param.endswith("..."): - if Re(r'\w\.\.\.$').search(param): - # For named variable parameters of the form `x...`, - # remove the dots - param =3D param[:-3] - else: - # Handles unnamed variable parameters - param =3D "..." - - if param not in self.entry.parameterdescs or \ - not self.entry.parameterdescs[param]: - - self.entry.parameterdescs[param] =3D "variable arguments" - - elif dtype =3D=3D "" and (not param or param =3D=3D "void"): - param =3D "void" - self.entry.parameterdescs[param] =3D "no arguments" - - elif dtype =3D=3D "" and param in ["struct", "union"]: - # Handle unnamed (anonymous) union or struct - dtype =3D param - param =3D "{unnamed_" + param + "}" - self.entry.parameterdescs[param] =3D "anonymous\n" - self.entry.anon_struct_union =3D True - - # Handle cache group enforcing variables: they do not need - # to be described in header files - elif "__cacheline_group" in param: - # Ignore __cacheline_group_begin and __cacheline_group_end - return - - # Warn if parameter has no description - # (but ignore ones starting with # as these are not parameters - # but inline preprocessor statements) - if param not in self.entry.parameterdescs and not param.startswith= ("#"): - self.entry.parameterdescs[param] =3D self.undescribed - - if self.show_warnings(dtype, declaration_name) and "." not in = param: - if decl_type =3D=3D 'function': - dname =3D f"{decl_type} parameter" - else: - dname =3D f"{decl_type} member" - - self.emit_warning(ln, - f"{dname} '{param}' not described in '{d= eclaration_name}'") - - # Strip spaces from param so that it is one continuous string on - # parameterlist. This fixes a problem where check_sections() - # cannot find a parameter like "addr[6 + 2]" because it actually - # appears as "addr[6", "+", "2]" on the parameter list. - # However, it's better to maintain the param string unchanged for - # output, so just weaken the string compare in check_sections() - # to ignore "[blah" in a parameter string. - - self.entry.parameterlist.append(param) - org_arg =3D Re(r'\s\s+').sub(' ', org_arg) - self.entry.parametertypes[param] =3D org_arg - - def save_struct_actual(self, actual): - """ - Strip all spaces from the actual param so that it looks like - one string item. - """ - - actual =3D Re(r'\s*').sub("", actual, count=3D1) - - self.entry.struct_actual +=3D actual + " " - - def create_parameter_list(self, ln, decl_type, args, splitter, declara= tion_name): - - # temporarily replace all commas inside function pointer definition - arg_expr =3D Re(r'(\([^\),]+),') - while arg_expr.search(args): - args =3D arg_expr.sub(r"\1#", args) - - for arg in args.split(splitter): - # Strip comments - arg =3D Re(r'\/\*.*\*\/').sub('', arg) - - # Ignore argument attributes - arg =3D Re(r'\sPOS0?\s').sub(' ', arg) - - # Strip leading/trailing spaces - arg =3D arg.strip() - arg =3D Re(r'\s+').sub(' ', arg, count=3D1) - - if arg.startswith('#'): - # Treat preprocessor directive as a typeless variable just= to fill - # corresponding data structures "correctly". Catch it late= r in - # output_* subs. - - # Treat preprocessor directive as a typeless variable - self.push_parameter(ln, decl_type, arg, "", - "", declaration_name) - - elif Re(r'\(.+\)\s*\(').search(arg): - # Pointer-to-function - - arg =3D arg.replace('#', ',') - - r =3D Re(r'[^\(]+\(\*?\s*([\w\[\]\.]*)\s*\)') - if r.match(arg): - param =3D r.group(1) - else: - self.emit_warning(ln, f"Invalid param: {arg}") - param =3D arg - - dtype =3D Re(r'([^\(]+\(\*?)\s*' + re.escape(param)).sub(r= '\1', arg) - self.save_struct_actual(param) - self.push_parameter(ln, decl_type, param, dtype, - arg, declaration_name) - - elif Re(r'\(.+\)\s*\[').search(arg): - # Array-of-pointers - - arg =3D arg.replace('#', ',') - r =3D Re(r'[^\(]+\(\s*\*\s*([\w\[\]\.]*?)\s*(\s*\[\s*[\w]+= \s*\]\s*)*\)') - if r.match(arg): - param =3D r.group(1) - else: - self.emit_warning(ln, f"Invalid param: {arg}") - param =3D arg - - dtype =3D Re(r'([^\(]+\(\*?)\s*' + re.escape(param)).sub(r= '\1', arg) - - self.save_struct_actual(param) - self.push_parameter(ln, decl_type, param, dtype, - arg, declaration_name) - - elif arg: - arg =3D Re(r'\s*:\s*').sub(":", arg) - arg =3D Re(r'\s*\[').sub('[', arg) - - args =3D Re(r'\s*,\s*').split(arg) - if args[0] and '*' in args[0]: - args[0] =3D re.sub(r'(\*+)\s*', r' \1', args[0]) - - first_arg =3D [] - r =3D Re(r'^(.*\s+)(.*?\[.*\].*)$') - if args[0] and r.match(args[0]): - args.pop(0) - first_arg.extend(r.group(1)) - first_arg.append(r.group(2)) - else: - first_arg =3D Re(r'\s+').split(args.pop(0)) - - args.insert(0, first_arg.pop()) - dtype =3D ' '.join(first_arg) - - for param in args: - if Re(r'^(\*+)\s*(.*)').match(param): - r =3D Re(r'^(\*+)\s*(.*)') - if not r.match(param): - self.emit_warning(ln, f"Invalid param: {param}= ") - continue - - param =3D r.group(1) - - self.save_struct_actual(r.group(2)) - self.push_parameter(ln, decl_type, r.group(2), - f"{dtype} {r.group(1)}", - arg, declaration_name) - - elif Re(r'(.*?):(\w+)').search(param): - r =3D Re(r'(.*?):(\w+)') - if not r.match(param): - self.emit_warning(ln, f"Invalid param: {param}= ") - continue - - if dtype !=3D "": # Skip unnamed bit-fields - self.save_struct_actual(r.group(1)) - self.push_parameter(ln, decl_type, r.group(1), - f"{dtype}:{r.group(2)}", - arg, declaration_name) - else: - self.save_struct_actual(param) - self.push_parameter(ln, decl_type, param, dtype, - arg, declaration_name) - - def check_sections(self, ln, decl_name, decl_type, sectcheck, prmschec= k): - sects =3D sectcheck.split() - prms =3D prmscheck.split() - err =3D False - - for sx in range(len(sects)): # pylint: disable=3D= C0200 - err =3D True - for px in range(len(prms)): # pylint: disable=3D= C0200 - prm_clean =3D prms[px] - prm_clean =3D Re(r'\[.*\]').sub('', prm_clean) - prm_clean =3D attribute.sub('', prm_clean) - - # ignore array size in a parameter string; - # however, the original param string may contain - # spaces, e.g.: addr[6 + 2] - # and this appears in @prms as "addr[6" since the - # parameter list is split at spaces; - # hence just ignore "[..." for the sections check; - prm_clean =3D Re(r'\[.*').sub('', prm_clean) - - if prm_clean =3D=3D sects[sx]: - err =3D False - break - - if err: - if decl_type =3D=3D 'function': - dname =3D f"{decl_type} parameter" - else: - dname =3D f"{decl_type} member" - - self.emit_warning(ln, - f"Excess {dname} '{sects[sx]}' descripti= on in '{decl_name}'") - - def check_return_section(self, ln, declaration_name, return_type): - - if not self.config.wreturn: - return - - # Ignore an empty return type (It's a macro) - # Ignore functions with a "void" return type (but not "void *") - if not return_type or Re(r'void\s*\w*\s*$').search(return_type): - return - - if not self.entry.sections.get("Return", None): - self.emit_warning(ln, - f"No description found for return value of '= {declaration_name}'") - - def dump_struct(self, ln, proto): - """ - Store an entry for an struct or union - """ - - type_pattern =3D r'(struct|union)' - - qualifiers =3D [ - "__attribute__", - "__packed", - "__aligned", - "____cacheline_aligned_in_smp", - "____cacheline_aligned", - ] - - definition_body =3D r'\{(.*)\}\s*' + "(?:" + '|'.join(qualifiers) = + ")?" - struct_members =3D Re(type_pattern + r'([^\{\};]+)(\{)([^\{\}]*)(\= })([^\{\}\;]*)(\;)') - - # Extract struct/union definition - members =3D None - declaration_name =3D None - decl_type =3D None - - r =3D Re(type_pattern + r'\s+(\w+)\s*' + definition_body) - if r.search(proto): - decl_type =3D r.group(1) - declaration_name =3D r.group(2) - members =3D r.group(3) - else: - r =3D Re(r'typedef\s+' + type_pattern + r'\s*' + definition_bo= dy + r'\s*(\w+)\s*;') - - if r.search(proto): - decl_type =3D r.group(1) - declaration_name =3D r.group(3) - members =3D r.group(2) - - if not members: - self.emit_warning(ln, f"{proto} error: Cannot parse struct or = union!") - self.config.errors +=3D 1 - return - - if self.entry.identifier !=3D declaration_name: - self.emit_warning(ln, - f"expecting prototype for {decl_type} {self.= entry.identifier}. Prototype was for {decl_type} {declaration_name} instead= \n") - return - - args_pattern =3Dr'([^,)]+)' - - sub_prefixes =3D [ - (Re(r'\/\*\s*private:.*?\/\*\s*public:.*?\*\/', re.S | re.I), = ''), - (Re(r'\/\*\s*private:.*', re.S| re.I), ''), - - # Strip comments - (Re(r'\/\*.*?\*\/', re.S), ''), - - # Strip attributes - (attribute, ' '), - (Re(r'\s*__aligned\s*\([^;]*\)', re.S), ' '), - (Re(r'\s*__counted_by\s*\([^;]*\)', re.S), ' '), - (Re(r'\s*__counted_by_(le|be)\s*\([^;]*\)', re.S), ' '), - (Re(r'\s*__packed\s*', re.S), ' '), - (Re(r'\s*CRYPTO_MINALIGN_ATTR', re.S), ' '), - (Re(r'\s*____cacheline_aligned_in_smp', re.S), ' '), - (Re(r'\s*____cacheline_aligned', re.S), ' '), - - # Unwrap struct_group macros based on this definition: - # __struct_group(TAG, NAME, ATTRS, MEMBERS...) - # which has variants like: struct_group(NAME, MEMBERS...) - # Only MEMBERS arguments require documentation. - # - # Parsing them happens on two steps: - # - # 1. drop struct group arguments that aren't at MEMBERS, - # storing them as STRUCT_GROUP(MEMBERS) - # - # 2. remove STRUCT_GROUP() ancillary macro. - # - # The original logic used to remove STRUCT_GROUP() using an - # advanced regex: - # - # \bSTRUCT_GROUP(\(((?:(?>[^)(]+)|(?1))*)\))[^;]*; - # - # with two patterns that are incompatible with - # Python re module, as it has: - # - # - a recursive pattern: (?1) - # - an atomic grouping: (?>...) - # - # I tried a simpler version: but it didn't work either: - # \bSTRUCT_GROUP\(([^\)]+)\)[^;]*; - # - # As it doesn't properly match the end parenthesis on some cas= es. - # - # So, a better solution was crafted: there's now a NestedMatch - # class that ensures that delimiters after a search are proper= ly - # matched. So, the implementation to drop STRUCT_GROUP() will = be - # handled in separate. - - (Re(r'\bstruct_group\s*\(([^,]*,)', re.S), r'STRUCT_GROUP('), - (Re(r'\bstruct_group_attr\s*\(([^,]*,){2}', re.S), r'STRUCT_G= ROUP('), - (Re(r'\bstruct_group_tagged\s*\(([^,]*),([^,]*),', re.S), r's= truct \1 \2; STRUCT_GROUP('), - (Re(r'\b__struct_group\s*\(([^,]*,){3}', re.S), r'STRUCT_GROU= P('), - - # Replace macros - # - # TODO: it is better to also move those to the NestedMatch log= ic, - # to ensure that parenthesis will be properly matched. - - (Re(r'__ETHTOOL_DECLARE_LINK_MODE_MASK\s*\(([^\)]+)\)', re.S),= r'DECLARE_BITMAP(\1, __ETHTOOL_LINK_MODE_MASK_NBITS)'), - (Re(r'DECLARE_PHY_INTERFACE_MASK\s*\(([^\)]+)\)', re.S), r'DE= CLARE_BITMAP(\1, PHY_INTERFACE_MODE_MAX)'), - (Re(r'DECLARE_BITMAP\s*\(' + args_pattern + r',\s*' + args_pat= tern + r'\)', re.S), r'unsigned long \1[BITS_TO_LONGS(\2)]'), - (Re(r'DECLARE_HASHTABLE\s*\(' + args_pattern + r',\s*' + args_= pattern + r'\)', re.S), r'unsigned long \1[1 << ((\2) - 1)]'), - (Re(r'DECLARE_KFIFO\s*\(' + args_pattern + r',\s*' + args_patt= ern + r',\s*' + args_pattern + r'\)', re.S), r'\2 *\1'), - (Re(r'DECLARE_KFIFO_PTR\s*\(' + args_pattern + r',\s*' + args_= pattern + r'\)', re.S), r'\2 *\1'), - (Re(r'(?:__)?DECLARE_FLEX_ARRAY\s*\(' + args_pattern + r',\s*'= + args_pattern + r'\)', re.S), r'\1 \2[]'), - (Re(r'DEFINE_DMA_UNMAP_ADDR\s*\(' + args_pattern + r'\)', re.S= ), r'dma_addr_t \1'), - (Re(r'DEFINE_DMA_UNMAP_LEN\s*\(' + args_pattern + r'\)', re.S)= , r'__u32 \1'), - ] - - # Regexes here are guaranteed to have the end limiter matching - # the start delimiter. Yet, right now, only one replace group - # is allowed. - - sub_nested_prefixes =3D [ - (re.compile(r'\bSTRUCT_GROUP\('), r'\1'), - ] - - for search, sub in sub_prefixes: - members =3D search.sub(sub, members) - - nested =3D NestedMatch() - - for search, sub in sub_nested_prefixes: - members =3D nested.sub(search, sub, members) - - # Keeps the original declaration as-is - declaration =3D members - - # Split nested struct/union elements - # - # This loop was simpler at the original kernel-doc perl version, as - # while ($members =3D~ m/$struct_members/) { ... } - # reads 'members' string on each interaction. - # - # Python behavior is different: it parses 'members' only once, - # creating a list of tuples from the first interaction. - # - # On other words, this won't get nested structs. - # - # So, we need to have an extra loop on Python to override such - # re limitation. - - while True: - tuples =3D struct_members.findall(members) - if not tuples: - break - - for t in tuples: - newmember =3D "" - maintype =3D t[0] - s_ids =3D t[5] - content =3D t[3] - - oldmember =3D "".join(t) - - for s_id in s_ids.split(','): - s_id =3D s_id.strip() - - newmember +=3D f"{maintype} {s_id}; " - s_id =3D Re(r'[:\[].*').sub('', s_id) - s_id =3D Re(r'^\s*\**(\S+)\s*').sub(r'\1', s_id) - - for arg in content.split(';'): - arg =3D arg.strip() - - if not arg: - continue - - r =3D Re(r'^([^\(]+\(\*?\s*)([\w\.]*)(\s*\).*)') - if r.match(arg): - # Pointer-to-function - dtype =3D r.group(1) - name =3D r.group(2) - extra =3D r.group(3) - - if not name: - continue - - if not s_id: - # Anonymous struct/union - newmember +=3D f"{dtype}{name}{extra}; " - else: - newmember +=3D f"{dtype}{s_id}.{name}{extr= a}; " - - else: - arg =3D arg.strip() - # Handle bitmaps - arg =3D Re(r':\s*\d+\s*').sub('', arg) - - # Handle arrays - arg =3D Re(r'\[.*\]').sub('', arg) - - # Handle multiple IDs - arg =3D Re(r'\s*,\s*').sub(',', arg) - - - r =3D Re(r'(.*)\s+([\S+,]+)') - - if r.search(arg): - dtype =3D r.group(1) - names =3D r.group(2) - else: - newmember +=3D f"{arg}; " - continue - - for name in names.split(','): - name =3D Re(r'^\s*\**(\S+)\s*').sub(r'\1',= name).strip() - - if not name: - continue - - if not s_id: - # Anonymous struct/union - newmember +=3D f"{dtype} {name}; " - else: - newmember +=3D f"{dtype} {s_id}.{name}= ; " - - members =3D members.replace(oldmember, newmember) - - # Ignore other nested elements, like enums - members =3D re.sub(r'(\{[^\{\}]*\})', '', members) - - self.create_parameter_list(ln, decl_type, members, ';', - declaration_name) - self.check_sections(ln, declaration_name, decl_type, - self.entry.sectcheck, self.entry.struct_actual) - - # Adjust declaration for better display - declaration =3D Re(r'([\{;])').sub(r'\1\n', declaration) - declaration =3D Re(r'\}\s+;').sub('};', declaration) - - # Better handle inlined enums - while True: - r =3D Re(r'(enum\s+\{[^\}]+),([^\n])') - if not r.search(declaration): - break - - declaration =3D r.sub(r'\1,\n\2', declaration) - - def_args =3D declaration.split('\n') - level =3D 1 - declaration =3D "" - for clause in def_args: - - clause =3D clause.strip() - clause =3D Re(r'\s+').sub(' ', clause, count=3D1) - - if not clause: - continue - - if '}' in clause and level > 1: - level -=3D 1 - - if not Re(r'^\s*#').match(clause): - declaration +=3D "\t" * level - - declaration +=3D "\t" + clause + "\n" - if "{" in clause and "}" not in clause: - level +=3D 1 - - self.output_declaration(decl_type, declaration_name, - struct=3Ddeclaration_name, - module=3Dself.entry.modulename, - definition=3Ddeclaration, - parameterlist=3Dself.entry.parameterlist, - parameterdescs=3Dself.entry.parameterdescs, - parametertypes=3Dself.entry.parametertypes, - sectionlist=3Dself.entry.sectionlist, - sections=3Dself.entry.sections, - purpose=3Dself.entry.declaration_purpose) - - def dump_enum(self, ln, proto): - - # Ignore members marked private - proto =3D Re(r'\/\*\s*private:.*?\/\*\s*public:.*?\*\/', flags=3Dr= e.S).sub('', proto) - proto =3D Re(r'\/\*\s*private:.*}', flags=3Dre.S).sub('}', proto) - - # Strip comments - proto =3D Re(r'\/\*.*?\*\/', flags=3Dre.S).sub('', proto) - - # Strip #define macros inside enums - proto =3D Re(r'#\s*((define|ifdef|if)\s+|endif)[^;]*;', flags=3Dre= .S).sub('', proto) - - members =3D None - declaration_name =3D None - - r =3D Re(r'typedef\s+enum\s*\{(.*)\}\s*(\w*)\s*;') - if r.search(proto): - declaration_name =3D r.group(2) - members =3D r.group(1).rstrip() - else: - r =3D Re(r'enum\s+(\w*)\s*\{(.*)\}') - if r.match(proto): - declaration_name =3D r.group(1) - members =3D r.group(2).rstrip() - - if not members: - self.emit_warning(ln, f"{proto}: error: Cannot parse enum!") - self.config.errors +=3D 1 - return - - if self.entry.identifier !=3D declaration_name: - if self.entry.identifier =3D=3D "": - self.emit_warning(ln, - f"{proto}: wrong kernel-doc identifier o= n prototype") - else: - self.emit_warning(ln, - f"expecting prototype for enum {self.ent= ry.identifier}. Prototype was for enum {declaration_name} instead") - return - - if not declaration_name: - declaration_name =3D "(anonymous)" - - member_set =3D set() - - members =3D Re(r'\([^;]*?[\)]').sub('', members) - - for arg in members.split(','): - if not arg: - continue - arg =3D Re(r'^\s*(\w+).*').sub(r'\1', arg) - self.entry.parameterlist.append(arg) - if arg not in self.entry.parameterdescs: - self.entry.parameterdescs[arg] =3D self.undescribed - if self.show_warnings("enum", declaration_name): - self.emit_warning(ln, - f"Enum value '{arg}' not described i= n enum '{declaration_name}'") - member_set.add(arg) - - for k in self.entry.parameterdescs: - if k not in member_set: - if self.show_warnings("enum", declaration_name): - self.emit_warning(ln, - f"Excess enum value '%{k}' descripti= on in '{declaration_name}'") - - self.output_declaration('enum', declaration_name, - enum=3Ddeclaration_name, - module=3Dself.config.modulename, - parameterlist=3Dself.entry.parameterlist, - parameterdescs=3Dself.entry.parameterdescs, - sectionlist=3Dself.entry.sectionlist, - sections=3Dself.entry.sections, - purpose=3Dself.entry.declaration_purpose) - - def dump_declaration(self, ln, prototype): - if self.entry.decl_type =3D=3D "enum": - self.dump_enum(ln, prototype) - return - - if self.entry.decl_type =3D=3D "typedef": - self.dump_typedef(ln, prototype) - return - - if self.entry.decl_type in ["union", "struct"]: - self.dump_struct(ln, prototype) - return - - # TODO: handle other types - self.output_declaration(self.entry.decl_type, prototype, - entry=3Dself.entry) - - def dump_function(self, ln, prototype): - - func_macro =3D False - return_type =3D '' - decl_type =3D 'function' - - # Prefixes that would be removed - sub_prefixes =3D [ - (r"^static +", "", 0), - (r"^extern +", "", 0), - (r"^asmlinkage +", "", 0), - (r"^inline +", "", 0), - (r"^__inline__ +", "", 0), - (r"^__inline +", "", 0), - (r"^__always_inline +", "", 0), - (r"^noinline +", "", 0), - (r"^__FORTIFY_INLINE +", "", 0), - (r"__init +", "", 0), - (r"__init_or_module +", "", 0), - (r"__deprecated +", "", 0), - (r"__flatten +", "", 0), - (r"__meminit +", "", 0), - (r"__must_check +", "", 0), - (r"__weak +", "", 0), - (r"__sched +", "", 0), - (r"_noprof", "", 0), - (r"__printf\s*\(\s*\d*\s*,\s*\d*\s*\) +", "", 0), - (r"__(?:re)?alloc_size\s*\(\s*\d+\s*(?:,\s*\d+\s*)?\) +", "", = 0), - (r"__diagnose_as\s*\(\s*\S+\s*(?:,\s*\d+\s*)*\) +", "", 0), - (r"DECL_BUCKET_PARAMS\s*\(\s*(\S+)\s*,\s*(\S+)\s*\)", r"\1, \2= ", 0), - (r"__attribute_const__ +", "", 0), - - # It seems that Python support for re.X is broken: - # At least for me (Python 3.13), this didn't work -# (r""" -# __attribute__\s*\(\( -# (?: -# [\w\s]+ # attribute name -# (?:\([^)]*\))? # attribute arguments -# \s*,? # optional comma at the end -# )+ -# \)\)\s+ -# """, "", re.X), - - # So, remove whitespaces and comments from it - (r"__attribute__\s*\(\((?:[\w\s]+(?:\([^)]*\))?\s*,?)+\)\)\s+"= , "", 0), - ] - - for search, sub, flags in sub_prefixes: - prototype =3D Re(search, flags).sub(sub, prototype) - - # Macros are a special case, as they change the prototype format - new_proto =3D Re(r"^#\s*define\s+").sub("", prototype) - if new_proto !=3D prototype: - is_define_proto =3D True - prototype =3D new_proto - else: - is_define_proto =3D False - - # Yes, this truly is vile. We are looking for: - # 1. Return type (may be nothing if we're looking at a macro) - # 2. Function name - # 3. Function parameters. - # - # All the while we have to watch out for function pointer paramete= rs - # (which IIRC is what the two sections are for), C types (these - # regexps don't even start to express all the possibilities), and - # so on. - # - # If you mess with these regexps, it's a good idea to check that - # the following functions' documentation still comes out right: - # - parport_register_device (function pointer parameters) - # - atomic_set (macro) - # - pci_match_device, __copy_to_user (long return type) - - name =3D r'[a-zA-Z0-9_~:]+' - prototype_end1 =3D r'[^\(]*' - prototype_end2 =3D r'[^\{]*' - prototype_end =3D fr'\(({prototype_end1}|{prototype_end2})\)' - - # Besides compiling, Perl qr{[\w\s]+} works as a non-capturing gro= up. - # So, this needs to be mapped in Python with (?:...)? or (?:...)+ - - type1 =3D r'(?:[\w\s]+)?' - type2 =3D r'(?:[\w\s]+\*+)+' - - found =3D False - - if is_define_proto: - r =3D Re(r'^()(' + name + r')\s+') - - if r.search(prototype): - return_type =3D '' - declaration_name =3D r.group(2) - func_macro =3D True - - found =3D True - - if not found: - patterns =3D [ - rf'^()({name})\s*{prototype_end}', - rf'^({type1})\s+({name})\s*{prototype_end}', - rf'^({type2})\s*({name})\s*{prototype_end}', - ] - - for p in patterns: - r =3D Re(p) - - if r.match(prototype): - - return_type =3D r.group(1) - declaration_name =3D r.group(2) - args =3D r.group(3) - - self.create_parameter_list(ln, decl_type, args, ',', - declaration_name) - - found =3D True - break - if not found: - self.emit_warning(ln, - f"cannot understand function prototype: '{pr= ototype}'") - return - - if self.entry.identifier !=3D declaration_name: - self.emit_warning(ln, - f"expecting prototype for {self.entry.identi= fier}(). Prototype was for {declaration_name}() instead") - return - - prms =3D " ".join(self.entry.parameterlist) - self.check_sections(ln, declaration_name, "function", - self.entry.sectcheck, prms) - - self.check_return_section(ln, declaration_name, return_type) - - if 'typedef' in return_type: - self.output_declaration(decl_type, declaration_name, - function=3Ddeclaration_name, - typedef=3DTrue, - module=3Dself.config.modulename, - functiontype=3Dreturn_type, - parameterlist=3Dself.entry.parameterlist, - parameterdescs=3Dself.entry.parameterdescs, - parametertypes=3Dself.entry.parametertypes, - sectionlist=3Dself.entry.sectionlist, - sections=3Dself.entry.sections, - purpose=3Dself.entry.declaration_purpose, - func_macro=3Dfunc_macro) - else: - self.output_declaration(decl_type, declaration_name, - function=3Ddeclaration_name, - typedef=3DFalse, - module=3Dself.config.modulename, - functiontype=3Dreturn_type, - parameterlist=3Dself.entry.parameterlist, - parameterdescs=3Dself.entry.parameterdescs, - parametertypes=3Dself.entry.parametertypes, - sectionlist=3Dself.entry.sectionlist, - sections=3Dself.entry.sections, - purpose=3Dself.entry.declaration_purpose, - func_macro=3Dfunc_macro) - - def dump_typedef(self, ln, proto): - typedef_type =3D r'((?:\s+[\w\*]+\b){1,8})\s*' - typedef_ident =3D r'\*?\s*(\w\S+)\s*' - typedef_args =3D r'\s*\((.*)\);' - - typedef1 =3D Re(r'typedef' + typedef_type + r'\(' + typedef_ident = + r'\)' + typedef_args) - typedef2 =3D Re(r'typedef' + typedef_type + typedef_ident + typede= f_args) - - # Strip comments - proto =3D Re(r'/\*.*?\*/', flags=3Dre.S).sub('', proto) - - # Parse function typedef prototypes - for r in [typedef1, typedef2]: - if not r.match(proto): - continue - - return_type =3D r.group(1).strip() - declaration_name =3D r.group(2) - args =3D r.group(3) - - if self.entry.identifier !=3D declaration_name: - self.emit_warning(ln, - f"expecting prototype for typedef {self.= entry.identifier}. Prototype was for typedef {declaration_name} instead\n") - return - - decl_type =3D 'function' - self.create_parameter_list(ln, decl_type, args, ',', declarati= on_name) - - self.output_declaration(decl_type, declaration_name, - function=3Ddeclaration_name, - typedef=3DTrue, - module=3Dself.entry.modulename, - functiontype=3Dreturn_type, - parameterlist=3Dself.entry.parameterlist, - parameterdescs=3Dself.entry.parameterdescs, - parametertypes=3Dself.entry.parametertypes, - sectionlist=3Dself.entry.sectionlist, - sections=3Dself.entry.sections, - purpose=3Dself.entry.declaration_purpose) - return - - # Handle nested parentheses or brackets - r =3D Re(r'(\(*.\)\s*|\[*.\]\s*);$') - while r.search(proto): - proto =3D r.sub('', proto) - - # Parse simple typedefs - r =3D Re(r'typedef.*\s+(\w+)\s*;') - if r.match(proto): - declaration_name =3D r.group(1) - - if self.entry.identifier !=3D declaration_name: - self.emit_warning(ln, f"expecting prototype for typedef {s= elf.entry.identifier}. Prototype was for typedef {declaration_name} instead= \n") - return - - self.output_declaration('typedef', declaration_name, - typedef=3Ddeclaration_name, - module=3Dself.entry.modulename, - sectionlist=3Dself.entry.sectionlist, - sections=3Dself.entry.sections, - purpose=3Dself.entry.declaration_purpose) - return - - self.emit_warning(ln, "error: Cannot parse typedef!") - self.config.errors +=3D 1 - - @staticmethod - def process_export(function_table, line): - """ - process EXPORT_SYMBOL* tags - - This method is called both internally and externally, so, it - doesn't use self. - """ - - if export_symbol.search(line): - symbol =3D export_symbol.group(2) - function_table.add(symbol) - - if export_symbol_ns.search(line): - symbol =3D export_symbol_ns.group(2) - function_table.add(symbol) - - def process_normal(self, ln, line): - """ - STATE_NORMAL: looking for the /** to begin everything. - """ - - if not doc_start.match(line): - return - - # start a new entry - self.reset_state(ln + 1) - self.entry.in_doc_sect =3D False - - # next line is always the function name - self.state =3D self.STATE_NAME - - def process_name(self, ln, line): - """ - STATE_NAME: Looking for the "name - description" line - """ - - if doc_block.search(line): - self.entry.new_start_line =3D ln - - if not doc_block.group(1): - self.entry.section =3D self.section_intro - else: - self.entry.section =3D doc_block.group(1) - - self.state =3D self.STATE_DOCBLOCK - return - - if doc_decl.search(line): - self.entry.identifier =3D doc_decl.group(1) - self.entry.is_kernel_comment =3D False - - decl_start =3D str(doc_com) # comment block asterisk - fn_type =3D r"(?:\w+\s*\*\s*)?" # type (for non-functions) - parenthesis =3D r"(?:\(\w*\))?" # optional parenthesis on fu= nction - decl_end =3D r"(?:[-:].*)" # end of the name part - - # test for pointer declaration type, foo * bar() - desc - r =3D Re(fr"^{decl_start}([\w\s]+?){parenthesis}?\s*{decl_end}= ?$") - if r.search(line): - self.entry.identifier =3D r.group(1) - - # Test for data declaration - r =3D Re(r"^\s*\*?\s*(struct|union|enum|typedef)\b\s*(\w*)") - if r.search(line): - self.entry.decl_type =3D r.group(1) - self.entry.identifier =3D r.group(2) - self.entry.is_kernel_comment =3D True - else: - # Look for foo() or static void foo() - description; - # or misspelt identifier - - r1 =3D Re(fr"^{decl_start}{fn_type}(\w+)\s*{parenthesis}\s= *{decl_end}?$") - r2 =3D Re(fr"^{decl_start}{fn_type}(\w+[^-:]*){parenthesis= }\s*{decl_end}$") - - for r in [r1, r2]: - if r.search(line): - self.entry.identifier =3D r.group(1) - self.entry.decl_type =3D "function" - - r =3D Re(r"define\s+") - self.entry.identifier =3D r.sub("", self.entry.ide= ntifier) - self.entry.is_kernel_comment =3D True - break - - self.entry.identifier =3D self.entry.identifier.strip(" ") - - self.state =3D self.STATE_BODY - - # if there's no @param blocks need to set up default section h= ere - self.entry.section =3D self.section_default - self.entry.new_start_line =3D ln + 1 - - r =3D Re("[-:](.*)") - if r.search(line): - # strip leading/trailing/multiple spaces - self.entry.descr =3D r.group(1).strip(" ") - - r =3D Re(r"\s+") - self.entry.descr =3D r.sub(" ", self.entry.descr) - self.entry.declaration_purpose =3D self.entry.descr - self.state =3D self.STATE_BODY_MAYBE - else: - self.entry.declaration_purpose =3D "" - - if not self.entry.is_kernel_comment: - self.emit_warning(ln, - f"This comment starts with '/**', but is= n't a kernel-doc comment. Refer Documentation/doc-guide/kernel-doc.rst\n{li= ne}") - self.state =3D self.STATE_NORMAL - - if not self.entry.declaration_purpose and self.config.wshort_d= esc: - self.emit_warning(ln, - f"missing initial short description on l= ine:\n{line}") - - if not self.entry.identifier and self.entry.decl_type !=3D "en= um": - self.emit_warning(ln, - f"wrong kernel-doc identifier on line:\n= {line}") - self.state =3D self.STATE_NORMAL - - if self.config.verbose: - self.emit_warning(ln, - f"Scanning doc for {self.entry.decl_type= } {self.entry.identifier}", - warning=3DFalse) - - return - - # Failed to find an identifier. Emit a warning - self.emit_warning(ln, f"Cannot find identifier on line:\n{line}") - - def process_body(self, ln, line): - """ - STATE_BODY and STATE_BODY_MAYBE: the bulk of a kerneldoc comment. - """ - - if self.state =3D=3D self.STATE_BODY_WITH_BLANK_LINE: - r =3D Re(r"\s*\*\s?\S") - if r.match(line): - self.dump_section() - self.entry.section =3D self.section_default - self.entry.new_start_line =3D line - self.entry.contents =3D "" - - if doc_sect.search(line): - self.entry.in_doc_sect =3D True - newsection =3D doc_sect.group(1) - - if newsection.lower() in ["description", "context"]: - newsection =3D newsection.title() - - # Special case: @return is a section, not a param description - if newsection.lower() in ["@return", "@returns", - "return", "returns"]: - newsection =3D "Return" - - # Perl kernel-doc has a check here for contents before section= s. - # the logic there is always false, as in_doc_sect variable is - # always true. So, just don't implement Wcontents_before_secti= ons - - # .title() - newcontents =3D doc_sect.group(2) - if not newcontents: - newcontents =3D "" - - if self.entry.contents.strip("\n"): - self.dump_section() - - self.entry.new_start_line =3D ln - self.entry.section =3D newsection - self.entry.leading_space =3D None - - self.entry.contents =3D newcontents.lstrip() - if self.entry.contents: - self.entry.contents +=3D "\n" - - self.state =3D self.STATE_BODY - return - - if doc_end.search(line): - self.dump_section() - - # Look for doc_com + + doc_end: - r =3D Re(r'\s*\*\s*[a-zA-Z_0-9:\.]+\*/') - if r.match(line): - self.emit_warning(ln, f"suspicious ending line: {line}") - - self.entry.prototype =3D "" - self.entry.new_start_line =3D ln + 1 - - self.state =3D self.STATE_PROTO - return - - if doc_content.search(line): - cont =3D doc_content.group(1) - - if cont =3D=3D "": - if self.entry.section =3D=3D self.section_context: - self.dump_section() - - self.entry.new_start_line =3D ln - self.state =3D self.STATE_BODY - else: - if self.entry.section !=3D self.section_default: - self.state =3D self.STATE_BODY_WITH_BLANK_LINE - else: - self.state =3D self.STATE_BODY - - self.entry.contents +=3D "\n" - - elif self.state =3D=3D self.STATE_BODY_MAYBE: - - # Continued declaration purpose - self.entry.declaration_purpose =3D self.entry.declaration_= purpose.rstrip() - self.entry.declaration_purpose +=3D " " + cont - - r =3D Re(r"\s+") - self.entry.declaration_purpose =3D r.sub(' ', - self.entry.declarat= ion_purpose) - - else: - if self.entry.section.startswith('@') or \ - self.entry.section =3D=3D self.section_context: - if self.entry.leading_space is None: - r =3D Re(r'^(\s+)') - if r.match(cont): - self.entry.leading_space =3D len(r.group(1)) - else: - self.entry.leading_space =3D 0 - - # Double-check if leading space are realy spaces - pos =3D 0 - for i in range(0, self.entry.leading_space): - if cont[i] !=3D " ": - break - pos +=3D 1 - - cont =3D cont[pos:] - - # NEW LOGIC: - # In case it is different, update it - if self.entry.leading_space !=3D pos: - self.entry.leading_space =3D pos - - self.entry.contents +=3D cont + "\n" - return - - # Unknown line, ignore - self.emit_warning(ln, f"bad line: {line}") - - def process_inline(self, ln, line): - """STATE_INLINE: docbook comments within a prototype.""" - - if self.inline_doc_state =3D=3D self.STATE_INLINE_NAME and \ - doc_inline_sect.search(line): - self.entry.section =3D doc_inline_sect.group(1) - self.entry.new_start_line =3D ln - - self.entry.contents =3D doc_inline_sect.group(2).lstrip() - if self.entry.contents !=3D "": - self.entry.contents +=3D "\n" - - self.inline_doc_state =3D self.STATE_INLINE_TEXT - # Documentation block end */ - return - - if doc_inline_end.search(line): - if self.entry.contents not in ["", "\n"]: - self.dump_section() - - self.state =3D self.STATE_PROTO - self.inline_doc_state =3D self.STATE_INLINE_NA - return - - if doc_content.search(line): - if self.inline_doc_state =3D=3D self.STATE_INLINE_TEXT: - self.entry.contents +=3D doc_content.group(1) + "\n" - if not self.entry.contents.strip(" ").rstrip("\n"): - self.entry.contents =3D "" - - elif self.inline_doc_state =3D=3D self.STATE_INLINE_NAME: - self.emit_warning(ln, - f"Incorrect use of kernel-doc format: {l= ine}") - - self.inline_doc_state =3D self.STATE_INLINE_ERROR - - def syscall_munge(self, ln, proto): - """ - Handle syscall definitions - """ - - is_void =3D False - - # Strip newlines/CR's - proto =3D re.sub(r'[\r\n]+', ' ', proto) - - # Check if it's a SYSCALL_DEFINE0 - if 'SYSCALL_DEFINE0' in proto: - is_void =3D True - - # Replace SYSCALL_DEFINE with correct return type & function name - proto =3D Re(r'SYSCALL_DEFINE.*\(').sub('long sys_', proto) - - r =3D Re(r'long\s+(sys_.*?),') - if r.search(proto): - proto =3D proto.replace(',', '(', count=3D1) - elif is_void: - proto =3D proto.replace(')', '(void)', count=3D1) - - # Now delete all of the odd-numbered commas in the proto - # so that argument types & names don't have a comma between them - count =3D 0 - length =3D len(proto) - - if is_void: - length =3D 0 # skip the loop if is_void - - for ix in range(length): - if proto[ix] =3D=3D ',': - count +=3D 1 - if count % 2 =3D=3D 1: - proto =3D proto[:ix] + ' ' + proto[ix+1:] - - return proto - - def tracepoint_munge(self, ln, proto): - """ - Handle tracepoint definitions - """ - - tracepointname =3D None - tracepointargs =3D None - - # Match tracepoint name based on different patterns - r =3D Re(r'TRACE_EVENT\((.*?),') - if r.search(proto): - tracepointname =3D r.group(1) - - r =3D Re(r'DEFINE_SINGLE_EVENT\((.*?),') - if r.search(proto): - tracepointname =3D r.group(1) - - r =3D Re(r'DEFINE_EVENT\((.*?),(.*?),') - if r.search(proto): - tracepointname =3D r.group(2) - - if tracepointname: - tracepointname =3D tracepointname.lstrip() - - r =3D Re(r'TP_PROTO\((.*?)\)') - if r.search(proto): - tracepointargs =3D r.group(1) - - if not tracepointname or not tracepointargs: - self.emit_warning(ln, - f"Unrecognized tracepoint format:\n{proto}\n= ") - else: - proto =3D f"static inline void trace_{tracepointname}({tracepo= intargs})" - self.entry.identifier =3D f"trace_{self.entry.identifier}" - - return proto - - def process_proto_function(self, ln, line): - """Ancillary routine to process a function prototype""" - - # strip C99-style comments to end of line - r =3D Re(r"\/\/.*$", re.S) - line =3D r.sub('', line) - - if Re(r'\s*#\s*define').match(line): - self.entry.prototype =3D line - elif line.startswith('#'): - # Strip other macros like #ifdef/#ifndef/#endif/... - pass - else: - r =3D Re(r'([^\{]*)') - if r.match(line): - self.entry.prototype +=3D r.group(1) + " " - - if '{' in line or ';' in line or Re(r'\s*#\s*define').match(line): - # strip comments - r =3D Re(r'/\*.*?\*/') - self.entry.prototype =3D r.sub('', self.entry.prototype) - - # strip newlines/cr's - r =3D Re(r'[\r\n]+') - self.entry.prototype =3D r.sub(' ', self.entry.prototype) - - # strip leading spaces - r =3D Re(r'^\s+') - self.entry.prototype =3D r.sub('', self.entry.prototype) - - # Handle self.entry.prototypes for function pointers like: - # int (*pcs_config)(struct foo) - - r =3D Re(r'^(\S+\s+)\(\s*\*(\S+)\)') - self.entry.prototype =3D r.sub(r'\1\2', self.entry.prototype) - - if 'SYSCALL_DEFINE' in self.entry.prototype: - self.entry.prototype =3D self.syscall_munge(ln, - self.entry.proto= type) - - r =3D Re(r'TRACE_EVENT|DEFINE_EVENT|DEFINE_SINGLE_EVENT') - if r.search(self.entry.prototype): - self.entry.prototype =3D self.tracepoint_munge(ln, - self.entry.pr= ototype) - - self.dump_function(ln, self.entry.prototype) - self.reset_state(ln) - - def process_proto_type(self, ln, line): - """Ancillary routine to process a type""" - - # Strip newlines/cr's. - line =3D Re(r'[\r\n]+', re.S).sub(' ', line) - - # Strip leading spaces - line =3D Re(r'^\s+', re.S).sub('', line) - - # Strip trailing spaces - line =3D Re(r'\s+$', re.S).sub('', line) - - # Strip C99-style comments to the end of the line - line =3D Re(r"\/\/.*$", re.S).sub('', line) - - # To distinguish preprocessor directive from regular declaration l= ater. - if line.startswith('#'): - line +=3D ";" - - r =3D Re(r'([^\{\};]*)([\{\};])(.*)') - while True: - if r.search(line): - if self.entry.prototype: - self.entry.prototype +=3D " " - self.entry.prototype +=3D r.group(1) + r.group(2) - - self.entry.brcount +=3D r.group(2).count('{') - self.entry.brcount -=3D r.group(2).count('}') - - self.entry.brcount =3D max(self.entry.brcount, 0) - - if r.group(2) =3D=3D ';' and self.entry.brcount =3D=3D 0: - self.dump_declaration(ln, self.entry.prototype) - self.reset_state(ln) - break - - line =3D r.group(3) - else: - self.entry.prototype +=3D line - break - - def process_proto(self, ln, line): - """STATE_PROTO: reading a function/whatever prototype.""" - - if doc_inline_oneline.search(line): - self.entry.section =3D doc_inline_oneline.group(1) - self.entry.contents =3D doc_inline_oneline.group(2) - - if self.entry.contents !=3D "": - self.entry.contents +=3D "\n" - self.dump_section(start_new=3DFalse) - - elif doc_inline_start.search(line): - self.state =3D self.STATE_INLINE - self.inline_doc_state =3D self.STATE_INLINE_NAME - - elif self.entry.decl_type =3D=3D 'function': - self.process_proto_function(ln, line) - - else: - self.process_proto_type(ln, line) - - def process_docblock(self, ln, line): - """STATE_DOCBLOCK: within a DOC: block.""" - - if doc_end.search(line): - self.dump_section() - self.output_declaration("doc", None, - sectionlist=3Dself.entry.sectionlist, - sections=3Dself.entry.sections, = module=3Dself.config.modulename) - self.reset_state(ln) - - elif doc_content.search(line): - self.entry.contents +=3D doc_content.group(1) + "\n" - - def run(self): - """ - Open and process each line of a C source file. - he parsing is controlled via a state machine, and the line is pass= ed - to a different process function depending on the state. The process - function may update the state as needed. - """ - - cont =3D False - prev =3D "" - prev_ln =3D None - - try: - with open(self.fname, "r", encoding=3D"utf8", - errors=3D"backslashreplace") as fp: - for ln, line in enumerate(fp): - - line =3D line.expandtabs().strip("\n") - - # Group continuation lines on prototypes - if self.state =3D=3D self.STATE_PROTO: - if line.endswith("\\"): - prev +=3D line.removesuffix("\\") - cont =3D True - - if not prev_ln: - prev_ln =3D ln - - continue - - if cont: - ln =3D prev_ln - line =3D prev + line - prev =3D "" - cont =3D False - prev_ln =3D None - - self.config.log.debug("%d %s%s: %s", - ln, self.st_name[self.state], - self.st_inline_name[self.inline_= doc_state], - line) - - # TODO: not all states allow EXPORT_SYMBOL*, so this - # can be optimized later on to speedup parsing - self.process_export(self.config.function_table, line) - - # Hand this line to the appropriate state handler - if self.state =3D=3D self.STATE_NORMAL: - self.process_normal(ln, line) - elif self.state =3D=3D self.STATE_NAME: - self.process_name(ln, line) - elif self.state in [self.STATE_BODY, self.STATE_BODY_M= AYBE, - self.STATE_BODY_WITH_BLANK_LINE]: - self.process_body(ln, line) - elif self.state =3D=3D self.STATE_INLINE: # scanning = for inline parameters - self.process_inline(ln, line) - elif self.state =3D=3D self.STATE_PROTO: - self.process_proto(ln, line) - elif self.state =3D=3D self.STATE_DOCBLOCK: - self.process_docblock(ln, line) - except OSError: - self.config.log.error(f"Error: Cannot open file {self.fname}") - self.config.errors +=3D 1 - - class GlobSourceFiles: """ Parse C source code file names and directories via an Interactor. diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser= .py new file mode 100755 index 000000000000..6d6395e32093 --- /dev/null +++ b/scripts/lib/kdoc/kdoc_parser.py @@ -0,0 +1,1689 @@ +#!/usr/bin/env python3 +# pylint: disable=3DC0301,C0302,R0904,R0912,R0913,R0914,R0915,R0917,R1702 +# Copyright(c) 2025: Mauro Carvalho Chehab . +# SPDX-License-Identifier: GPL-2.0 + +""" +kdoc_parser +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Read a C language source or header FILE and extract embedded +documentation comments +""" + +import argparse +import re +from pprint import pformat + +from kdoc_re import NestedMatch, Re + + +# +# Regular expressions used to parse kernel-doc markups at KernelDoc class. +# +# Let's declare them in lowercase outside any class to make easier to +# convert from the python script. +# +# As those are evaluated at the beginning, no need to cache them +# + +# Allow whitespace at end of comment start. +doc_start =3D Re(r'^/\*\*\s*$', cache=3DFalse) + +doc_end =3D Re(r'\*/', cache=3DFalse) +doc_com =3D Re(r'\s*\*\s*', cache=3DFalse) +doc_com_body =3D Re(r'\s*\* ?', cache=3DFalse) +doc_decl =3D doc_com + Re(r'(\w+)', cache=3DFalse) + +# @params and a strictly limited set of supported section names +# Specifically: +# Match @word: +# @...: +# @{section-name}: +# while trying to not match literal block starts like "example::" +# +doc_sect =3D doc_com + \ + Re(r'\s*(\@[.\w]+|\@\.\.\.|description|context|returns?|notes?= |examples?)\s*:([^:].*)?$', + flags=3Dre.I, cache=3DFalse) + +doc_content =3D doc_com_body + Re(r'(.*)', cache=3DFalse) +doc_block =3D doc_com + Re(r'DOC:\s*(.*)?', cache=3DFalse) +doc_inline_start =3D Re(r'^\s*/\*\*\s*$', cache=3DFalse) +doc_inline_sect =3D Re(r'\s*\*\s*(@\s*[\w][\w\.]*\s*):(.*)', cache=3DFalse) +doc_inline_end =3D Re(r'^\s*\*/\s*$', cache=3DFalse) +doc_inline_oneline =3D Re(r'^\s*/\*\*\s*(@[\w\s]+):\s*(.*)\s*\*/\s*$', cac= he=3DFalse) +attribute =3D Re(r"__attribute__\s*\(\([a-z0-9,_\*\s\(\)]*\)\)", + flags=3Dre.I | re.S, cache=3DFalse) + +export_symbol =3D Re(r'^\s*EXPORT_SYMBOL(_GPL)?\s*\(\s*(\w+)\s*\)\s*', cac= he=3DFalse) +export_symbol_ns =3D Re(r'^\s*EXPORT_SYMBOL_NS(_GPL)?\s*\(\s*(\w+)\s*,\s*"= \S+"\)\s*', cache=3DFalse) + +type_param =3D Re(r"\@(\w*((\.\w+)|(->\w+))*(\.\.\.)?)", cache=3DFalse) + + +class KernelDoc: + """ + Read a C language source or header FILE and extract embedded + documentation comments. + """ + + # Parser states + STATE_NORMAL =3D 0 # normal code + STATE_NAME =3D 1 # looking for function name + STATE_BODY_MAYBE =3D 2 # body - or maybe more description + STATE_BODY =3D 3 # the body of the comment + STATE_BODY_WITH_BLANK_LINE =3D 4 # the body which has a blank line + STATE_PROTO =3D 5 # scanning prototype + STATE_DOCBLOCK =3D 6 # documentation block + STATE_INLINE =3D 7 # gathering doc outside main block + + st_name =3D [ + "NORMAL", + "NAME", + "BODY_MAYBE", + "BODY", + "BODY_WITH_BLANK_LINE", + "PROTO", + "DOCBLOCK", + "INLINE", + ] + + # Inline documentation state + STATE_INLINE_NA =3D 0 # not applicable ($state !=3D STATE_INLINE) + STATE_INLINE_NAME =3D 1 # looking for member name (@foo:) + STATE_INLINE_TEXT =3D 2 # looking for member documentation + STATE_INLINE_END =3D 3 # done + STATE_INLINE_ERROR =3D 4 # error - Comment without header was found. + # Spit a warning as it's not + # proper kernel-doc and ignore the rest. + + st_inline_name =3D [ + "", + "_NAME", + "_TEXT", + "_END", + "_ERROR", + ] + + # Section names + + section_default =3D "Description" # default section + section_intro =3D "Introduction" + section_context =3D "Context" + section_return =3D "Return" + + undescribed =3D "-- undescribed --" + + def __init__(self, config, fname): + """Initialize internal variables""" + + self.fname =3D fname + self.config =3D config + + # Initial state for the state machines + self.state =3D self.STATE_NORMAL + self.inline_doc_state =3D self.STATE_INLINE_NA + + # Store entry currently being processed + self.entry =3D None + + # Place all potential outputs into an array + self.entries =3D [] + + def show_warnings(self, dtype, declaration_name): # pylint: disable= =3DW0613 + """ + Allow filtering out warnings + """ + + # TODO: implement it + + return True + + # TODO: rename to emit_message + def emit_warning(self, ln, msg, warning=3DTrue): + """Emit a message""" + + if warning: + self.config.log.warning("%s:%d %s", self.fname, ln, msg) + else: + self.config.log.info("%s:%d %s", self.fname, ln, msg) + + def dump_section(self, start_new=3DTrue): + """ + Dumps section contents to arrays/hashes intended for that purpose. + """ + + name =3D self.entry.section + contents =3D self.entry.contents + + # TODO: we can prevent dumping empty sections here with: + # + # if self.entry.contents.strip("\n"): + # if start_new: + # self.entry.section =3D self.section_default + # self.entry.contents =3D "" + # + # return + # + # But, as we want to be producing the same output of the + # venerable kernel-doc Perl tool, let's just output everything, + # at least for now + + if type_param.match(name): + name =3D type_param.group(1) + + self.entry.parameterdescs[name] =3D contents + self.entry.parameterdesc_start_lines[name] =3D self.entry.new_= start_line + + self.entry.sectcheck +=3D name + " " + self.entry.new_start_line =3D 0 + + elif name =3D=3D "@...": + name =3D "..." + self.entry.parameterdescs[name] =3D contents + self.entry.sectcheck +=3D name + " " + self.entry.parameterdesc_start_lines[name] =3D self.entry.new_= start_line + self.entry.new_start_line =3D 0 + + else: + if name in self.entry.sections and self.entry.sections[name] != =3D "": + # Only warn on user-specified duplicate section names + if name !=3D self.section_default: + self.emit_warning(self.entry.new_start_line, + f"duplicate section name '{name}'\n") + self.entry.sections[name] +=3D contents + else: + self.entry.sections[name] =3D contents + self.entry.sectionlist.append(name) + self.entry.section_start_lines[name] =3D self.entry.new_st= art_line + self.entry.new_start_line =3D 0 + +# self.config.log.debug("Section: %s : %s", name, pformat(vars(self= .entry))) + + if start_new: + self.entry.section =3D self.section_default + self.entry.contents =3D "" + + # TODO: rename it to store_declaration + def output_declaration(self, dtype, name, **args): + """ + Stores the entry into an entry array. + + The actual output and output filters will be handled elsewhere + """ + + # The implementation here is different than the original kernel-do= c: + # instead of checking for output filters or actually output anythi= ng, + # it just stores the declaration content at self.entries, as the + # output will happen on a separate class. + # + # For now, we're keeping the same name of the function just to make + # easier to compare the source code of both scripts + + if "declaration_start_line" not in args: + args["declaration_start_line"] =3D self.entry.declaration_star= t_line + + args["type"] =3D dtype + + # TODO: use colletions.OrderedDict + + sections =3D args.get('sections', {}) + sectionlist =3D args.get('sectionlist', []) + + # Drop empty sections + # TODO: improve it to emit warnings + for section in ["Description", "Return"]: + if section in sectionlist: + if not sections[section].rstrip(): + del sections[section] + sectionlist.remove(section) + + self.entries.append((name, args)) + + self.config.log.debug("Output: %s:%s =3D %s", dtype, name, pformat= (args)) + + def reset_state(self, ln): + """ + Ancillary routine to create a new entry. It initializes all + variables used by the state machine. + """ + + self.entry =3D argparse.Namespace + + self.entry.contents =3D "" + self.entry.function =3D "" + self.entry.sectcheck =3D "" + self.entry.struct_actual =3D "" + self.entry.prototype =3D "" + + self.entry.parameterlist =3D [] + self.entry.parameterdescs =3D {} + self.entry.parametertypes =3D {} + self.entry.parameterdesc_start_lines =3D {} + + self.entry.section_start_lines =3D {} + self.entry.sectionlist =3D [] + self.entry.sections =3D {} + + self.entry.anon_struct_union =3D False + + self.entry.leading_space =3D None + + # State flags + self.state =3D self.STATE_NORMAL + self.inline_doc_state =3D self.STATE_INLINE_NA + self.entry.brcount =3D 0 + + self.entry.in_doc_sect =3D False + self.entry.declaration_start_line =3D ln + + def push_parameter(self, ln, decl_type, param, dtype, + org_arg, declaration_name): + """ + Store parameters and their descriptions at self.entry. + """ + + if self.entry.anon_struct_union and dtype =3D=3D "" and param =3D= =3D "}": + return # Ignore the ending }; from anonymous struct/union + + self.entry.anon_struct_union =3D False + + param =3D Re(r'[\[\)].*').sub('', param, count=3D1) + + if dtype =3D=3D "" and param.endswith("..."): + if Re(r'\w\.\.\.$').search(param): + # For named variable parameters of the form `x...`, + # remove the dots + param =3D param[:-3] + else: + # Handles unnamed variable parameters + param =3D "..." + + if param not in self.entry.parameterdescs or \ + not self.entry.parameterdescs[param]: + + self.entry.parameterdescs[param] =3D "variable arguments" + + elif dtype =3D=3D "" and (not param or param =3D=3D "void"): + param =3D "void" + self.entry.parameterdescs[param] =3D "no arguments" + + elif dtype =3D=3D "" and param in ["struct", "union"]: + # Handle unnamed (anonymous) union or struct + dtype =3D param + param =3D "{unnamed_" + param + "}" + self.entry.parameterdescs[param] =3D "anonymous\n" + self.entry.anon_struct_union =3D True + + # Handle cache group enforcing variables: they do not need + # to be described in header files + elif "__cacheline_group" in param: + # Ignore __cacheline_group_begin and __cacheline_group_end + return + + # Warn if parameter has no description + # (but ignore ones starting with # as these are not parameters + # but inline preprocessor statements) + if param not in self.entry.parameterdescs and not param.startswith= ("#"): + self.entry.parameterdescs[param] =3D self.undescribed + + if self.show_warnings(dtype, declaration_name) and "." not in = param: + if decl_type =3D=3D 'function': + dname =3D f"{decl_type} parameter" + else: + dname =3D f"{decl_type} member" + + self.emit_warning(ln, + f"{dname} '{param}' not described in '{d= eclaration_name}'") + + # Strip spaces from param so that it is one continuous string on + # parameterlist. This fixes a problem where check_sections() + # cannot find a parameter like "addr[6 + 2]" because it actually + # appears as "addr[6", "+", "2]" on the parameter list. + # However, it's better to maintain the param string unchanged for + # output, so just weaken the string compare in check_sections() + # to ignore "[blah" in a parameter string. + + self.entry.parameterlist.append(param) + org_arg =3D Re(r'\s\s+').sub(' ', org_arg) + self.entry.parametertypes[param] =3D org_arg + + def save_struct_actual(self, actual): + """ + Strip all spaces from the actual param so that it looks like + one string item. + """ + + actual =3D Re(r'\s*').sub("", actual, count=3D1) + + self.entry.struct_actual +=3D actual + " " + + def create_parameter_list(self, ln, decl_type, args, + splitter, declaration_name): + """ + Creates a list of parameters, storing them at self.entry. + """ + + # temporarily replace all commas inside function pointer definition + arg_expr =3D Re(r'(\([^\),]+),') + while arg_expr.search(args): + args =3D arg_expr.sub(r"\1#", args) + + for arg in args.split(splitter): + # Strip comments + arg =3D Re(r'\/\*.*\*\/').sub('', arg) + + # Ignore argument attributes + arg =3D Re(r'\sPOS0?\s').sub(' ', arg) + + # Strip leading/trailing spaces + arg =3D arg.strip() + arg =3D Re(r'\s+').sub(' ', arg, count=3D1) + + if arg.startswith('#'): + # Treat preprocessor directive as a typeless variable just= to fill + # corresponding data structures "correctly". Catch it late= r in + # output_* subs. + + # Treat preprocessor directive as a typeless variable + self.push_parameter(ln, decl_type, arg, "", + "", declaration_name) + + elif Re(r'\(.+\)\s*\(').search(arg): + # Pointer-to-function + + arg =3D arg.replace('#', ',') + + r =3D Re(r'[^\(]+\(\*?\s*([\w\[\]\.]*)\s*\)') + if r.match(arg): + param =3D r.group(1) + else: + self.emit_warning(ln, f"Invalid param: {arg}") + param =3D arg + + dtype =3D Re(r'([^\(]+\(\*?)\s*' + re.escape(param)).sub(r= '\1', arg) + self.save_struct_actual(param) + self.push_parameter(ln, decl_type, param, dtype, + arg, declaration_name) + + elif Re(r'\(.+\)\s*\[').search(arg): + # Array-of-pointers + + arg =3D arg.replace('#', ',') + r =3D Re(r'[^\(]+\(\s*\*\s*([\w\[\]\.]*?)\s*(\s*\[\s*[\w]+= \s*\]\s*)*\)') + if r.match(arg): + param =3D r.group(1) + else: + self.emit_warning(ln, f"Invalid param: {arg}") + param =3D arg + + dtype =3D Re(r'([^\(]+\(\*?)\s*' + re.escape(param)).sub(r= '\1', arg) + + self.save_struct_actual(param) + self.push_parameter(ln, decl_type, param, dtype, + arg, declaration_name) + + elif arg: + arg =3D Re(r'\s*:\s*').sub(":", arg) + arg =3D Re(r'\s*\[').sub('[', arg) + + args =3D Re(r'\s*,\s*').split(arg) + if args[0] and '*' in args[0]: + args[0] =3D re.sub(r'(\*+)\s*', r' \1', args[0]) + + first_arg =3D [] + r =3D Re(r'^(.*\s+)(.*?\[.*\].*)$') + if args[0] and r.match(args[0]): + args.pop(0) + first_arg.extend(r.group(1)) + first_arg.append(r.group(2)) + else: + first_arg =3D Re(r'\s+').split(args.pop(0)) + + args.insert(0, first_arg.pop()) + dtype =3D ' '.join(first_arg) + + for param in args: + if Re(r'^(\*+)\s*(.*)').match(param): + r =3D Re(r'^(\*+)\s*(.*)') + if not r.match(param): + self.emit_warning(ln, f"Invalid param: {param}= ") + continue + + param =3D r.group(1) + + self.save_struct_actual(r.group(2)) + self.push_parameter(ln, decl_type, r.group(2), + f"{dtype} {r.group(1)}", + arg, declaration_name) + + elif Re(r'(.*?):(\w+)').search(param): + r =3D Re(r'(.*?):(\w+)') + if not r.match(param): + self.emit_warning(ln, f"Invalid param: {param}= ") + continue + + if dtype !=3D "": # Skip unnamed bit-fields + self.save_struct_actual(r.group(1)) + self.push_parameter(ln, decl_type, r.group(1), + f"{dtype}:{r.group(2)}", + arg, declaration_name) + else: + self.save_struct_actual(param) + self.push_parameter(ln, decl_type, param, dtype, + arg, declaration_name) + + def check_sections(self, ln, decl_name, decl_type, sectcheck, prmschec= k): + """ + Check for errors inside sections, emitting warnings if not found + parameters are described. + """ + + sects =3D sectcheck.split() + prms =3D prmscheck.split() + err =3D False + + for sx in range(len(sects)): # pylint: disable=3D= C0200 + err =3D True + for px in range(len(prms)): # pylint: disable=3D= C0200 + prm_clean =3D prms[px] + prm_clean =3D Re(r'\[.*\]').sub('', prm_clean) + prm_clean =3D attribute.sub('', prm_clean) + + # ignore array size in a parameter string; + # however, the original param string may contain + # spaces, e.g.: addr[6 + 2] + # and this appears in @prms as "addr[6" since the + # parameter list is split at spaces; + # hence just ignore "[..." for the sections check; + prm_clean =3D Re(r'\[.*').sub('', prm_clean) + + if prm_clean =3D=3D sects[sx]: + err =3D False + break + + if err: + if decl_type =3D=3D 'function': + dname =3D f"{decl_type} parameter" + else: + dname =3D f"{decl_type} member" + + self.emit_warning(ln, + f"Excess {dname} '{sects[sx]}' descripti= on in '{decl_name}'") + + def check_return_section(self, ln, declaration_name, return_type): + """ + If the function doesn't return void, warns about the lack of a + return description. + """ + + if not self.config.wreturn: + return + + # Ignore an empty return type (It's a macro) + # Ignore functions with a "void" return type (but not "void *") + if not return_type or Re(r'void\s*\w*\s*$').search(return_type): + return + + if not self.entry.sections.get("Return", None): + self.emit_warning(ln, + f"No description found for return value of '= {declaration_name}'") + + def dump_struct(self, ln, proto): + """ + Store an entry for an struct or union + """ + + type_pattern =3D r'(struct|union)' + + qualifiers =3D [ + "__attribute__", + "__packed", + "__aligned", + "____cacheline_aligned_in_smp", + "____cacheline_aligned", + ] + + definition_body =3D r'\{(.*)\}\s*' + "(?:" + '|'.join(qualifiers) = + ")?" + struct_members =3D Re(type_pattern + r'([^\{\};]+)(\{)([^\{\}]*)(\= })([^\{\}\;]*)(\;)') + + # Extract struct/union definition + members =3D None + declaration_name =3D None + decl_type =3D None + + r =3D Re(type_pattern + r'\s+(\w+)\s*' + definition_body) + if r.search(proto): + decl_type =3D r.group(1) + declaration_name =3D r.group(2) + members =3D r.group(3) + else: + r =3D Re(r'typedef\s+' + type_pattern + r'\s*' + definition_bo= dy + r'\s*(\w+)\s*;') + + if r.search(proto): + decl_type =3D r.group(1) + declaration_name =3D r.group(3) + members =3D r.group(2) + + if not members: + self.emit_warning(ln, f"{proto} error: Cannot parse struct or = union!") + self.config.errors +=3D 1 + return + + if self.entry.identifier !=3D declaration_name: + self.emit_warning(ln, + f"expecting prototype for {decl_type} {self.= entry.identifier}. Prototype was for {decl_type} {declaration_name} instead= \n") + return + + args_pattern =3D r'([^,)]+)' + + sub_prefixes =3D [ + (Re(r'\/\*\s*private:.*?\/\*\s*public:.*?\*\/', re.S | re.I), = ''), + (Re(r'\/\*\s*private:.*', re.S | re.I), ''), + + # Strip comments + (Re(r'\/\*.*?\*\/', re.S), ''), + + # Strip attributes + (attribute, ' '), + (Re(r'\s*__aligned\s*\([^;]*\)', re.S), ' '), + (Re(r'\s*__counted_by\s*\([^;]*\)', re.S), ' '), + (Re(r'\s*__counted_by_(le|be)\s*\([^;]*\)', re.S), ' '), + (Re(r'\s*__packed\s*', re.S), ' '), + (Re(r'\s*CRYPTO_MINALIGN_ATTR', re.S), ' '), + (Re(r'\s*____cacheline_aligned_in_smp', re.S), ' '), + (Re(r'\s*____cacheline_aligned', re.S), ' '), + + # Unwrap struct_group macros based on this definition: + # __struct_group(TAG, NAME, ATTRS, MEMBERS...) + # which has variants like: struct_group(NAME, MEMBERS...) + # Only MEMBERS arguments require documentation. + # + # Parsing them happens on two steps: + # + # 1. drop struct group arguments that aren't at MEMBERS, + # storing them as STRUCT_GROUP(MEMBERS) + # + # 2. remove STRUCT_GROUP() ancillary macro. + # + # The original logic used to remove STRUCT_GROUP() using an + # advanced regex: + # + # \bSTRUCT_GROUP(\(((?:(?>[^)(]+)|(?1))*)\))[^;]*; + # + # with two patterns that are incompatible with + # Python re module, as it has: + # + # - a recursive pattern: (?1) + # - an atomic grouping: (?>...) + # + # I tried a simpler version: but it didn't work either: + # \bSTRUCT_GROUP\(([^\)]+)\)[^;]*; + # + # As it doesn't properly match the end parenthesis on some cas= es. + # + # So, a better solution was crafted: there's now a NestedMatch + # class that ensures that delimiters after a search are proper= ly + # matched. So, the implementation to drop STRUCT_GROUP() will = be + # handled in separate. + + (Re(r'\bstruct_group\s*\(([^,]*,)', re.S), r'STRUCT_GROUP('), + (Re(r'\bstruct_group_attr\s*\(([^,]*,){2}', re.S), r'STRUCT_GR= OUP('), + (Re(r'\bstruct_group_tagged\s*\(([^,]*),([^,]*),', re.S), r'st= ruct \1 \2; STRUCT_GROUP('), + (Re(r'\b__struct_group\s*\(([^,]*,){3}', re.S), r'STRUCT_GROUP= ('), + + # Replace macros + # + # TODO: it is better to also move those to the NestedMatch log= ic, + # to ensure that parenthesis will be properly matched. + + (Re(r'__ETHTOOL_DECLARE_LINK_MODE_MASK\s*\(([^\)]+)\)', re.S),= r'DECLARE_BITMAP(\1, __ETHTOOL_LINK_MODE_MASK_NBITS)'), + (Re(r'DECLARE_PHY_INTERFACE_MASK\s*\(([^\)]+)\)', re.S), r'DEC= LARE_BITMAP(\1, PHY_INTERFACE_MODE_MAX)'), + (Re(r'DECLARE_BITMAP\s*\(' + args_pattern + r',\s*' + args_pat= tern + r'\)', re.S), r'unsigned long \1[BITS_TO_LONGS(\2)]'), + (Re(r'DECLARE_HASHTABLE\s*\(' + args_pattern + r',\s*' + args_= pattern + r'\)', re.S), r'unsigned long \1[1 << ((\2) - 1)]'), + (Re(r'DECLARE_KFIFO\s*\(' + args_pattern + r',\s*' + args_patt= ern + r',\s*' + args_pattern + r'\)', re.S), r'\2 *\1'), + (Re(r'DECLARE_KFIFO_PTR\s*\(' + args_pattern + r',\s*' + args_= pattern + r'\)', re.S), r'\2 *\1'), + (Re(r'(?:__)?DECLARE_FLEX_ARRAY\s*\(' + args_pattern + r',\s*'= + args_pattern + r'\)', re.S), r'\1 \2[]'), + (Re(r'DEFINE_DMA_UNMAP_ADDR\s*\(' + args_pattern + r'\)', re.S= ), r'dma_addr_t \1'), + (Re(r'DEFINE_DMA_UNMAP_LEN\s*\(' + args_pattern + r'\)', re.S)= , r'__u32 \1'), + ] + + # Regexes here are guaranteed to have the end limiter matching + # the start delimiter. Yet, right now, only one replace group + # is allowed. + + sub_nested_prefixes =3D [ + (re.compile(r'\bSTRUCT_GROUP\('), r'\1'), + ] + + for search, sub in sub_prefixes: + members =3D search.sub(sub, members) + + nested =3D NestedMatch() + + for search, sub in sub_nested_prefixes: + members =3D nested.sub(search, sub, members) + + # Keeps the original declaration as-is + declaration =3D members + + # Split nested struct/union elements + # + # This loop was simpler at the original kernel-doc perl version, as + # while ($members =3D~ m/$struct_members/) { ... } + # reads 'members' string on each interaction. + # + # Python behavior is different: it parses 'members' only once, + # creating a list of tuples from the first interaction. + # + # On other words, this won't get nested structs. + # + # So, we need to have an extra loop on Python to override such + # re limitation. + + while True: + tuples =3D struct_members.findall(members) + if not tuples: + break + + for t in tuples: + newmember =3D "" + maintype =3D t[0] + s_ids =3D t[5] + content =3D t[3] + + oldmember =3D "".join(t) + + for s_id in s_ids.split(','): + s_id =3D s_id.strip() + + newmember +=3D f"{maintype} {s_id}; " + s_id =3D Re(r'[:\[].*').sub('', s_id) + s_id =3D Re(r'^\s*\**(\S+)\s*').sub(r'\1', s_id) + + for arg in content.split(';'): + arg =3D arg.strip() + + if not arg: + continue + + r =3D Re(r'^([^\(]+\(\*?\s*)([\w\.]*)(\s*\).*)') + if r.match(arg): + # Pointer-to-function + dtype =3D r.group(1) + name =3D r.group(2) + extra =3D r.group(3) + + if not name: + continue + + if not s_id: + # Anonymous struct/union + newmember +=3D f"{dtype}{name}{extra}; " + else: + newmember +=3D f"{dtype}{s_id}.{name}{extr= a}; " + + else: + arg =3D arg.strip() + # Handle bitmaps + arg =3D Re(r':\s*\d+\s*').sub('', arg) + + # Handle arrays + arg =3D Re(r'\[.*\]').sub('', arg) + + # Handle multiple IDs + arg =3D Re(r'\s*,\s*').sub(',', arg) + + r =3D Re(r'(.*)\s+([\S+,]+)') + + if r.search(arg): + dtype =3D r.group(1) + names =3D r.group(2) + else: + newmember +=3D f"{arg}; " + continue + + for name in names.split(','): + name =3D Re(r'^\s*\**(\S+)\s*').sub(r'\1',= name).strip() + + if not name: + continue + + if not s_id: + # Anonymous struct/union + newmember +=3D f"{dtype} {name}; " + else: + newmember +=3D f"{dtype} {s_id}.{name}= ; " + + members =3D members.replace(oldmember, newmember) + + # Ignore other nested elements, like enums + members =3D re.sub(r'(\{[^\{\}]*\})', '', members) + + self.create_parameter_list(ln, decl_type, members, ';', + declaration_name) + self.check_sections(ln, declaration_name, decl_type, + self.entry.sectcheck, self.entry.struct_actual) + + # Adjust declaration for better display + declaration =3D Re(r'([\{;])').sub(r'\1\n', declaration) + declaration =3D Re(r'\}\s+;').sub('};', declaration) + + # Better handle inlined enums + while True: + r =3D Re(r'(enum\s+\{[^\}]+),([^\n])') + if not r.search(declaration): + break + + declaration =3D r.sub(r'\1,\n\2', declaration) + + def_args =3D declaration.split('\n') + level =3D 1 + declaration =3D "" + for clause in def_args: + + clause =3D clause.strip() + clause =3D Re(r'\s+').sub(' ', clause, count=3D1) + + if not clause: + continue + + if '}' in clause and level > 1: + level -=3D 1 + + if not Re(r'^\s*#').match(clause): + declaration +=3D "\t" * level + + declaration +=3D "\t" + clause + "\n" + if "{" in clause and "}" not in clause: + level +=3D 1 + + self.output_declaration(decl_type, declaration_name, + struct=3Ddeclaration_name, + module=3Dself.entry.modulename, + definition=3Ddeclaration, + parameterlist=3Dself.entry.parameterlist, + parameterdescs=3Dself.entry.parameterdescs, + parametertypes=3Dself.entry.parametertypes, + sectionlist=3Dself.entry.sectionlist, + sections=3Dself.entry.sections, + purpose=3Dself.entry.declaration_purpose) + + def dump_enum(self, ln, proto): + """ + Stores an enum inside self.entries array. + """ + + # Ignore members marked private + proto =3D Re(r'\/\*\s*private:.*?\/\*\s*public:.*?\*\/', flags=3Dr= e.S).sub('', proto) + proto =3D Re(r'\/\*\s*private:.*}', flags=3Dre.S).sub('}', proto) + + # Strip comments + proto =3D Re(r'\/\*.*?\*\/', flags=3Dre.S).sub('', proto) + + # Strip #define macros inside enums + proto =3D Re(r'#\s*((define|ifdef|if)\s+|endif)[^;]*;', flags=3Dre= .S).sub('', proto) + + members =3D None + declaration_name =3D None + + r =3D Re(r'typedef\s+enum\s*\{(.*)\}\s*(\w*)\s*;') + if r.search(proto): + declaration_name =3D r.group(2) + members =3D r.group(1).rstrip() + else: + r =3D Re(r'enum\s+(\w*)\s*\{(.*)\}') + if r.match(proto): + declaration_name =3D r.group(1) + members =3D r.group(2).rstrip() + + if not members: + self.emit_warning(ln, f"{proto}: error: Cannot parse enum!") + self.config.errors +=3D 1 + return + + if self.entry.identifier !=3D declaration_name: + if self.entry.identifier =3D=3D "": + self.emit_warning(ln, + f"{proto}: wrong kernel-doc identifier o= n prototype") + else: + self.emit_warning(ln, + f"expecting prototype for enum {self.ent= ry.identifier}. Prototype was for enum {declaration_name} instead") + return + + if not declaration_name: + declaration_name =3D "(anonymous)" + + member_set =3D set() + + members =3D Re(r'\([^;]*?[\)]').sub('', members) + + for arg in members.split(','): + if not arg: + continue + arg =3D Re(r'^\s*(\w+).*').sub(r'\1', arg) + self.entry.parameterlist.append(arg) + if arg not in self.entry.parameterdescs: + self.entry.parameterdescs[arg] =3D self.undescribed + if self.show_warnings("enum", declaration_name): + self.emit_warning(ln, + f"Enum value '{arg}' not described i= n enum '{declaration_name}'") + member_set.add(arg) + + for k in self.entry.parameterdescs: + if k not in member_set: + if self.show_warnings("enum", declaration_name): + self.emit_warning(ln, + f"Excess enum value '%{k}' descripti= on in '{declaration_name}'") + + self.output_declaration('enum', declaration_name, + enum=3Ddeclaration_name, + module=3Dself.config.modulename, + parameterlist=3Dself.entry.parameterlist, + parameterdescs=3Dself.entry.parameterdescs, + sectionlist=3Dself.entry.sectionlist, + sections=3Dself.entry.sections, + purpose=3Dself.entry.declaration_purpose) + + def dump_declaration(self, ln, prototype): + """ + Stores a data declaration inside self.entries array. + """ + + if self.entry.decl_type =3D=3D "enum": + self.dump_enum(ln, prototype) + return + + if self.entry.decl_type =3D=3D "typedef": + self.dump_typedef(ln, prototype) + return + + if self.entry.decl_type in ["union", "struct"]: + self.dump_struct(ln, prototype) + return + + # TODO: handle other types + self.output_declaration(self.entry.decl_type, prototype, + entry=3Dself.entry) + + def dump_function(self, ln, prototype): + """ + Stores a function of function macro inside self.entries array. + """ + + func_macro =3D False + return_type =3D '' + decl_type =3D 'function' + + # Prefixes that would be removed + sub_prefixes =3D [ + (r"^static +", "", 0), + (r"^extern +", "", 0), + (r"^asmlinkage +", "", 0), + (r"^inline +", "", 0), + (r"^__inline__ +", "", 0), + (r"^__inline +", "", 0), + (r"^__always_inline +", "", 0), + (r"^noinline +", "", 0), + (r"^__FORTIFY_INLINE +", "", 0), + (r"__init +", "", 0), + (r"__init_or_module +", "", 0), + (r"__deprecated +", "", 0), + (r"__flatten +", "", 0), + (r"__meminit +", "", 0), + (r"__must_check +", "", 0), + (r"__weak +", "", 0), + (r"__sched +", "", 0), + (r"_noprof", "", 0), + (r"__printf\s*\(\s*\d*\s*,\s*\d*\s*\) +", "", 0), + (r"__(?:re)?alloc_size\s*\(\s*\d+\s*(?:,\s*\d+\s*)?\) +", "", = 0), + (r"__diagnose_as\s*\(\s*\S+\s*(?:,\s*\d+\s*)*\) +", "", 0), + (r"DECL_BUCKET_PARAMS\s*\(\s*(\S+)\s*,\s*(\S+)\s*\)", r"\1, \2= ", 0), + (r"__attribute_const__ +", "", 0), + + # It seems that Python support for re.X is broken: + # At least for me (Python 3.13), this didn't work +# (r""" +# __attribute__\s*\(\( +# (?: +# [\w\s]+ # attribute name +# (?:\([^)]*\))? # attribute arguments +# \s*,? # optional comma at the end +# )+ +# \)\)\s+ +# """, "", re.X), + + # So, remove whitespaces and comments from it + (r"__attribute__\s*\(\((?:[\w\s]+(?:\([^)]*\))?\s*,?)+\)\)\s+"= , "", 0), + ] + + for search, sub, flags in sub_prefixes: + prototype =3D Re(search, flags).sub(sub, prototype) + + # Macros are a special case, as they change the prototype format + new_proto =3D Re(r"^#\s*define\s+").sub("", prototype) + if new_proto !=3D prototype: + is_define_proto =3D True + prototype =3D new_proto + else: + is_define_proto =3D False + + # Yes, this truly is vile. We are looking for: + # 1. Return type (may be nothing if we're looking at a macro) + # 2. Function name + # 3. Function parameters. + # + # All the while we have to watch out for function pointer paramete= rs + # (which IIRC is what the two sections are for), C types (these + # regexps don't even start to express all the possibilities), and + # so on. + # + # If you mess with these regexps, it's a good idea to check that + # the following functions' documentation still comes out right: + # - parport_register_device (function pointer parameters) + # - atomic_set (macro) + # - pci_match_device, __copy_to_user (long return type) + + name =3D r'[a-zA-Z0-9_~:]+' + prototype_end1 =3D r'[^\(]*' + prototype_end2 =3D r'[^\{]*' + prototype_end =3D fr'\(({prototype_end1}|{prototype_end2})\)' + + # Besides compiling, Perl qr{[\w\s]+} works as a non-capturing gro= up. + # So, this needs to be mapped in Python with (?:...)? or (?:...)+ + + type1 =3D r'(?:[\w\s]+)?' + type2 =3D r'(?:[\w\s]+\*+)+' + + found =3D False + + if is_define_proto: + r =3D Re(r'^()(' + name + r')\s+') + + if r.search(prototype): + return_type =3D '' + declaration_name =3D r.group(2) + func_macro =3D True + + found =3D True + + if not found: + patterns =3D [ + rf'^()({name})\s*{prototype_end}', + rf'^({type1})\s+({name})\s*{prototype_end}', + rf'^({type2})\s*({name})\s*{prototype_end}', + ] + + for p in patterns: + r =3D Re(p) + + if r.match(prototype): + + return_type =3D r.group(1) + declaration_name =3D r.group(2) + args =3D r.group(3) + + self.create_parameter_list(ln, decl_type, args, ',', + declaration_name) + + found =3D True + break + if not found: + self.emit_warning(ln, + f"cannot understand function prototype: '{pr= ototype}'") + return + + if self.entry.identifier !=3D declaration_name: + self.emit_warning(ln, + f"expecting prototype for {self.entry.identi= fier}(). Prototype was for {declaration_name}() instead") + return + + prms =3D " ".join(self.entry.parameterlist) + self.check_sections(ln, declaration_name, "function", + self.entry.sectcheck, prms) + + self.check_return_section(ln, declaration_name, return_type) + + if 'typedef' in return_type: + self.output_declaration(decl_type, declaration_name, + function=3Ddeclaration_name, + typedef=3DTrue, + module=3Dself.config.modulename, + functiontype=3Dreturn_type, + parameterlist=3Dself.entry.parameterli= st, + parameterdescs=3Dself.entry.parameterd= escs, + parametertypes=3Dself.entry.parametert= ypes, + sectionlist=3Dself.entry.sectionlist, + sections=3Dself.entry.sections, + purpose=3Dself.entry.declaration_purpo= se, + func_macro=3Dfunc_macro) + else: + self.output_declaration(decl_type, declaration_name, + function=3Ddeclaration_name, + typedef=3DFalse, + module=3Dself.config.modulename, + functiontype=3Dreturn_type, + parameterlist=3Dself.entry.parameterli= st, + parameterdescs=3Dself.entry.parameterd= escs, + parametertypes=3Dself.entry.parametert= ypes, + sectionlist=3Dself.entry.sectionlist, + sections=3Dself.entry.sections, + purpose=3Dself.entry.declaration_purpo= se, + func_macro=3Dfunc_macro) + + def dump_typedef(self, ln, proto): + """ + Stores a typedef inside self.entries array. + """ + + typedef_type =3D r'((?:\s+[\w\*]+\b){1,8})\s*' + typedef_ident =3D r'\*?\s*(\w\S+)\s*' + typedef_args =3D r'\s*\((.*)\);' + + typedef1 =3D Re(r'typedef' + typedef_type + r'\(' + typedef_ident = + r'\)' + typedef_args) + typedef2 =3D Re(r'typedef' + typedef_type + typedef_ident + typede= f_args) + + # Strip comments + proto =3D Re(r'/\*.*?\*/', flags=3Dre.S).sub('', proto) + + # Parse function typedef prototypes + for r in [typedef1, typedef2]: + if not r.match(proto): + continue + + return_type =3D r.group(1).strip() + declaration_name =3D r.group(2) + args =3D r.group(3) + + if self.entry.identifier !=3D declaration_name: + self.emit_warning(ln, + f"expecting prototype for typedef {self.= entry.identifier}. Prototype was for typedef {declaration_name} instead\n") + return + + decl_type =3D 'function' + self.create_parameter_list(ln, decl_type, args, ',', declarati= on_name) + + self.output_declaration(decl_type, declaration_name, + function=3Ddeclaration_name, + typedef=3DTrue, + module=3Dself.entry.modulename, + functiontype=3Dreturn_type, + parameterlist=3Dself.entry.parameterli= st, + parameterdescs=3Dself.entry.parameterd= escs, + parametertypes=3Dself.entry.parametert= ypes, + sectionlist=3Dself.entry.sectionlist, + sections=3Dself.entry.sections, + purpose=3Dself.entry.declaration_purpo= se) + return + + # Handle nested parentheses or brackets + r =3D Re(r'(\(*.\)\s*|\[*.\]\s*);$') + while r.search(proto): + proto =3D r.sub('', proto) + + # Parse simple typedefs + r =3D Re(r'typedef.*\s+(\w+)\s*;') + if r.match(proto): + declaration_name =3D r.group(1) + + if self.entry.identifier !=3D declaration_name: + self.emit_warning(ln, f"expecting prototype for typedef {s= elf.entry.identifier}. Prototype was for typedef {declaration_name} instead= \n") + return + + self.output_declaration('typedef', declaration_name, + typedef=3Ddeclaration_name, + module=3Dself.entry.modulename, + sectionlist=3Dself.entry.sectionlist, + sections=3Dself.entry.sections, + purpose=3Dself.entry.declaration_purpo= se) + return + + self.emit_warning(ln, "error: Cannot parse typedef!") + self.config.errors +=3D 1 + + @staticmethod + def process_export(function_table, line): + """ + process EXPORT_SYMBOL* tags + + This method is called both internally and externally, so, it + doesn't use self. + """ + + if export_symbol.search(line): + symbol =3D export_symbol.group(2) + function_table.add(symbol) + + if export_symbol_ns.search(line): + symbol =3D export_symbol_ns.group(2) + function_table.add(symbol) + + def process_normal(self, ln, line): + """ + STATE_NORMAL: looking for the /** to begin everything. + """ + + if not doc_start.match(line): + return + + # start a new entry + self.reset_state(ln + 1) + self.entry.in_doc_sect =3D False + + # next line is always the function name + self.state =3D self.STATE_NAME + + def process_name(self, ln, line): + """ + STATE_NAME: Looking for the "name - description" line + """ + + if doc_block.search(line): + self.entry.new_start_line =3D ln + + if not doc_block.group(1): + self.entry.section =3D self.section_intro + else: + self.entry.section =3D doc_block.group(1) + + self.state =3D self.STATE_DOCBLOCK + return + + if doc_decl.search(line): + self.entry.identifier =3D doc_decl.group(1) + self.entry.is_kernel_comment =3D False + + decl_start =3D str(doc_com) # comment block asterisk + fn_type =3D r"(?:\w+\s*\*\s*)?" # type (for non-functions) + parenthesis =3D r"(?:\(\w*\))?" # optional parenthesis on fu= nction + decl_end =3D r"(?:[-:].*)" # end of the name part + + # test for pointer declaration type, foo * bar() - desc + r =3D Re(fr"^{decl_start}([\w\s]+?){parenthesis}?\s*{decl_end}= ?$") + if r.search(line): + self.entry.identifier =3D r.group(1) + + # Test for data declaration + r =3D Re(r"^\s*\*?\s*(struct|union|enum|typedef)\b\s*(\w*)") + if r.search(line): + self.entry.decl_type =3D r.group(1) + self.entry.identifier =3D r.group(2) + self.entry.is_kernel_comment =3D True + else: + # Look for foo() or static void foo() - description; + # or misspelt identifier + + r1 =3D Re(fr"^{decl_start}{fn_type}(\w+)\s*{parenthesis}\s= *{decl_end}?$") + r2 =3D Re(fr"^{decl_start}{fn_type}(\w+[^-:]*){parenthesis= }\s*{decl_end}$") + + for r in [r1, r2]: + if r.search(line): + self.entry.identifier =3D r.group(1) + self.entry.decl_type =3D "function" + + r =3D Re(r"define\s+") + self.entry.identifier =3D r.sub("", self.entry.ide= ntifier) + self.entry.is_kernel_comment =3D True + break + + self.entry.identifier =3D self.entry.identifier.strip(" ") + + self.state =3D self.STATE_BODY + + # if there's no @param blocks need to set up default section h= ere + self.entry.section =3D self.section_default + self.entry.new_start_line =3D ln + 1 + + r =3D Re("[-:](.*)") + if r.search(line): + # strip leading/trailing/multiple spaces + self.entry.descr =3D r.group(1).strip(" ") + + r =3D Re(r"\s+") + self.entry.descr =3D r.sub(" ", self.entry.descr) + self.entry.declaration_purpose =3D self.entry.descr + self.state =3D self.STATE_BODY_MAYBE + else: + self.entry.declaration_purpose =3D "" + + if not self.entry.is_kernel_comment: + self.emit_warning(ln, + f"This comment starts with '/**', but is= n't a kernel-doc comment. Refer Documentation/doc-guide/kernel-doc.rst\n{li= ne}") + self.state =3D self.STATE_NORMAL + + if not self.entry.declaration_purpose and self.config.wshort_d= esc: + self.emit_warning(ln, + f"missing initial short description on l= ine:\n{line}") + + if not self.entry.identifier and self.entry.decl_type !=3D "en= um": + self.emit_warning(ln, + f"wrong kernel-doc identifier on line:\n= {line}") + self.state =3D self.STATE_NORMAL + + if self.config.verbose: + self.emit_warning(ln, + f"Scanning doc for {self.entry.decl_type= } {self.entry.identifier}", + warning=3DFalse) + + return + + # Failed to find an identifier. Emit a warning + self.emit_warning(ln, f"Cannot find identifier on line:\n{line}") + + def process_body(self, ln, line): + """ + STATE_BODY and STATE_BODY_MAYBE: the bulk of a kerneldoc comment. + """ + + if self.state =3D=3D self.STATE_BODY_WITH_BLANK_LINE: + r =3D Re(r"\s*\*\s?\S") + if r.match(line): + self.dump_section() + self.entry.section =3D self.section_default + self.entry.new_start_line =3D line + self.entry.contents =3D "" + + if doc_sect.search(line): + self.entry.in_doc_sect =3D True + newsection =3D doc_sect.group(1) + + if newsection.lower() in ["description", "context"]: + newsection =3D newsection.title() + + # Special case: @return is a section, not a param description + if newsection.lower() in ["@return", "@returns", + "return", "returns"]: + newsection =3D "Return" + + # Perl kernel-doc has a check here for contents before section= s. + # the logic there is always false, as in_doc_sect variable is + # always true. So, just don't implement Wcontents_before_secti= ons + + # .title() + newcontents =3D doc_sect.group(2) + if not newcontents: + newcontents =3D "" + + if self.entry.contents.strip("\n"): + self.dump_section() + + self.entry.new_start_line =3D ln + self.entry.section =3D newsection + self.entry.leading_space =3D None + + self.entry.contents =3D newcontents.lstrip() + if self.entry.contents: + self.entry.contents +=3D "\n" + + self.state =3D self.STATE_BODY + return + + if doc_end.search(line): + self.dump_section() + + # Look for doc_com + + doc_end: + r =3D Re(r'\s*\*\s*[a-zA-Z_0-9:\.]+\*/') + if r.match(line): + self.emit_warning(ln, f"suspicious ending line: {line}") + + self.entry.prototype =3D "" + self.entry.new_start_line =3D ln + 1 + + self.state =3D self.STATE_PROTO + return + + if doc_content.search(line): + cont =3D doc_content.group(1) + + if cont =3D=3D "": + if self.entry.section =3D=3D self.section_context: + self.dump_section() + + self.entry.new_start_line =3D ln + self.state =3D self.STATE_BODY + else: + if self.entry.section !=3D self.section_default: + self.state =3D self.STATE_BODY_WITH_BLANK_LINE + else: + self.state =3D self.STATE_BODY + + self.entry.contents +=3D "\n" + + elif self.state =3D=3D self.STATE_BODY_MAYBE: + + # Continued declaration purpose + self.entry.declaration_purpose =3D self.entry.declaration_= purpose.rstrip() + self.entry.declaration_purpose +=3D " " + cont + + r =3D Re(r"\s+") + self.entry.declaration_purpose =3D r.sub(' ', + self.entry.declarat= ion_purpose) + + else: + if self.entry.section.startswith('@') or \ + self.entry.section =3D=3D self.section_context: + if self.entry.leading_space is None: + r =3D Re(r'^(\s+)') + if r.match(cont): + self.entry.leading_space =3D len(r.group(1)) + else: + self.entry.leading_space =3D 0 + + # Double-check if leading space are realy spaces + pos =3D 0 + for i in range(0, self.entry.leading_space): + if cont[i] !=3D " ": + break + pos +=3D 1 + + cont =3D cont[pos:] + + # NEW LOGIC: + # In case it is different, update it + if self.entry.leading_space !=3D pos: + self.entry.leading_space =3D pos + + self.entry.contents +=3D cont + "\n" + return + + # Unknown line, ignore + self.emit_warning(ln, f"bad line: {line}") + + def process_inline(self, ln, line): + """STATE_INLINE: docbook comments within a prototype.""" + + if self.inline_doc_state =3D=3D self.STATE_INLINE_NAME and \ + doc_inline_sect.search(line): + self.entry.section =3D doc_inline_sect.group(1) + self.entry.new_start_line =3D ln + + self.entry.contents =3D doc_inline_sect.group(2).lstrip() + if self.entry.contents !=3D "": + self.entry.contents +=3D "\n" + + self.inline_doc_state =3D self.STATE_INLINE_TEXT + # Documentation block end */ + return + + if doc_inline_end.search(line): + if self.entry.contents not in ["", "\n"]: + self.dump_section() + + self.state =3D self.STATE_PROTO + self.inline_doc_state =3D self.STATE_INLINE_NA + return + + if doc_content.search(line): + if self.inline_doc_state =3D=3D self.STATE_INLINE_TEXT: + self.entry.contents +=3D doc_content.group(1) + "\n" + if not self.entry.contents.strip(" ").rstrip("\n"): + self.entry.contents =3D "" + + elif self.inline_doc_state =3D=3D self.STATE_INLINE_NAME: + self.emit_warning(ln, + f"Incorrect use of kernel-doc format: {l= ine}") + + self.inline_doc_state =3D self.STATE_INLINE_ERROR + + def syscall_munge(self, ln, proto): # pylint: disable=3DW0613 + """ + Handle syscall definitions + """ + + is_void =3D False + + # Strip newlines/CR's + proto =3D re.sub(r'[\r\n]+', ' ', proto) + + # Check if it's a SYSCALL_DEFINE0 + if 'SYSCALL_DEFINE0' in proto: + is_void =3D True + + # Replace SYSCALL_DEFINE with correct return type & function name + proto =3D Re(r'SYSCALL_DEFINE.*\(').sub('long sys_', proto) + + r =3D Re(r'long\s+(sys_.*?),') + if r.search(proto): + proto =3D proto.replace(',', '(', count=3D1) + elif is_void: + proto =3D proto.replace(')', '(void)', count=3D1) + + # Now delete all of the odd-numbered commas in the proto + # so that argument types & names don't have a comma between them + count =3D 0 + length =3D len(proto) + + if is_void: + length =3D 0 # skip the loop if is_void + + for ix in range(length): + if proto[ix] =3D=3D ',': + count +=3D 1 + if count % 2 =3D=3D 1: + proto =3D proto[:ix] + ' ' + proto[ix + 1:] + + return proto + + def tracepoint_munge(self, ln, proto): + """ + Handle tracepoint definitions + """ + + tracepointname =3D None + tracepointargs =3D None + + # Match tracepoint name based on different patterns + r =3D Re(r'TRACE_EVENT\((.*?),') + if r.search(proto): + tracepointname =3D r.group(1) + + r =3D Re(r'DEFINE_SINGLE_EVENT\((.*?),') + if r.search(proto): + tracepointname =3D r.group(1) + + r =3D Re(r'DEFINE_EVENT\((.*?),(.*?),') + if r.search(proto): + tracepointname =3D r.group(2) + + if tracepointname: + tracepointname =3D tracepointname.lstrip() + + r =3D Re(r'TP_PROTO\((.*?)\)') + if r.search(proto): + tracepointargs =3D r.group(1) + + if not tracepointname or not tracepointargs: + self.emit_warning(ln, + f"Unrecognized tracepoint format:\n{proto}\n= ") + else: + proto =3D f"static inline void trace_{tracepointname}({tracepo= intargs})" + self.entry.identifier =3D f"trace_{self.entry.identifier}" + + return proto + + def process_proto_function(self, ln, line): + """Ancillary routine to process a function prototype""" + + # strip C99-style comments to end of line + r =3D Re(r"\/\/.*$", re.S) + line =3D r.sub('', line) + + if Re(r'\s*#\s*define').match(line): + self.entry.prototype =3D line + elif line.startswith('#'): + # Strip other macros like #ifdef/#ifndef/#endif/... + pass + else: + r =3D Re(r'([^\{]*)') + if r.match(line): + self.entry.prototype +=3D r.group(1) + " " + + if '{' in line or ';' in line or Re(r'\s*#\s*define').match(line): + # strip comments + r =3D Re(r'/\*.*?\*/') + self.entry.prototype =3D r.sub('', self.entry.prototype) + + # strip newlines/cr's + r =3D Re(r'[\r\n]+') + self.entry.prototype =3D r.sub(' ', self.entry.prototype) + + # strip leading spaces + r =3D Re(r'^\s+') + self.entry.prototype =3D r.sub('', self.entry.prototype) + + # Handle self.entry.prototypes for function pointers like: + # int (*pcs_config)(struct foo) + + r =3D Re(r'^(\S+\s+)\(\s*\*(\S+)\)') + self.entry.prototype =3D r.sub(r'\1\2', self.entry.prototype) + + if 'SYSCALL_DEFINE' in self.entry.prototype: + self.entry.prototype =3D self.syscall_munge(ln, + self.entry.proto= type) + + r =3D Re(r'TRACE_EVENT|DEFINE_EVENT|DEFINE_SINGLE_EVENT') + if r.search(self.entry.prototype): + self.entry.prototype =3D self.tracepoint_munge(ln, + self.entry.pr= ototype) + + self.dump_function(ln, self.entry.prototype) + self.reset_state(ln) + + def process_proto_type(self, ln, line): + """Ancillary routine to process a type""" + + # Strip newlines/cr's. + line =3D Re(r'[\r\n]+', re.S).sub(' ', line) + + # Strip leading spaces + line =3D Re(r'^\s+', re.S).sub('', line) + + # Strip trailing spaces + line =3D Re(r'\s+$', re.S).sub('', line) + + # Strip C99-style comments to the end of the line + line =3D Re(r"\/\/.*$", re.S).sub('', line) + + # To distinguish preprocessor directive from regular declaration l= ater. + if line.startswith('#'): + line +=3D ";" + + r =3D Re(r'([^\{\};]*)([\{\};])(.*)') + while True: + if r.search(line): + if self.entry.prototype: + self.entry.prototype +=3D " " + self.entry.prototype +=3D r.group(1) + r.group(2) + + self.entry.brcount +=3D r.group(2).count('{') + self.entry.brcount -=3D r.group(2).count('}') + + self.entry.brcount =3D max(self.entry.brcount, 0) + + if r.group(2) =3D=3D ';' and self.entry.brcount =3D=3D 0: + self.dump_declaration(ln, self.entry.prototype) + self.reset_state(ln) + break + + line =3D r.group(3) + else: + self.entry.prototype +=3D line + break + + def process_proto(self, ln, line): + """STATE_PROTO: reading a function/whatever prototype.""" + + if doc_inline_oneline.search(line): + self.entry.section =3D doc_inline_oneline.group(1) + self.entry.contents =3D doc_inline_oneline.group(2) + + if self.entry.contents !=3D "": + self.entry.contents +=3D "\n" + self.dump_section(start_new=3DFalse) + + elif doc_inline_start.search(line): + self.state =3D self.STATE_INLINE + self.inline_doc_state =3D self.STATE_INLINE_NAME + + elif self.entry.decl_type =3D=3D 'function': + self.process_proto_function(ln, line) + + else: + self.process_proto_type(ln, line) + + def process_docblock(self, ln, line): + """STATE_DOCBLOCK: within a DOC: block.""" + + if doc_end.search(line): + self.dump_section() + self.output_declaration("doc", None, + sectionlist=3Dself.entry.sectionlist, + sections=3Dself.entry.sections, module= =3Dself.config.modulename) + self.reset_state(ln) + + elif doc_content.search(line): + self.entry.contents +=3D doc_content.group(1) + "\n" + + def run(self): + """ + Open and process each line of a C source file. + he parsing is controlled via a state machine, and the line is pass= ed + to a different process function depending on the state. The process + function may update the state as needed. + """ + + cont =3D False + prev =3D "" + prev_ln =3D None + + try: + with open(self.fname, "r", encoding=3D"utf8", + errors=3D"backslashreplace") as fp: + for ln, line in enumerate(fp): + + line =3D line.expandtabs().strip("\n") + + # Group continuation lines on prototypes + if self.state =3D=3D self.STATE_PROTO: + if line.endswith("\\"): + prev +=3D line.removesuffix("\\") + cont =3D True + + if not prev_ln: + prev_ln =3D ln + + continue + + if cont: + ln =3D prev_ln + line =3D prev + line + prev =3D "" + cont =3D False + prev_ln =3D None + + self.config.log.debug("%d %s%s: %s", + ln, self.st_name[self.state], + self.st_inline_name[self.inline_= doc_state], + line) + + # TODO: not all states allow EXPORT_SYMBOL*, so this + # can be optimized later on to speedup parsing + self.process_export(self.config.function_table, line) + + # Hand this line to the appropriate state handler + if self.state =3D=3D self.STATE_NORMAL: + self.process_normal(ln, line) + elif self.state =3D=3D self.STATE_NAME: + self.process_name(ln, line) + elif self.state in [self.STATE_BODY, self.STATE_BODY_M= AYBE, + self.STATE_BODY_WITH_BLANK_LINE]: + self.process_body(ln, line) + elif self.state =3D=3D self.STATE_INLINE: # scanning = for inline parameters + self.process_inline(ln, line) + elif self.state =3D=3D self.STATE_PROTO: + self.process_proto(ln, line) + elif self.state =3D=3D self.STATE_DOCBLOCK: + self.process_docblock(ln, line) + except OSError: + self.config.log.error(f"Error: Cannot open file {self.fname}") + self.config.errors +=3D 1 --=20 2.48.1 From nobody Sat Feb 7 14:01:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ACC331DC04A; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; cv=none; b=G+/Lpz9eMqXNCd9f49o6nXBDcLm67Q8Rs2Kon2hPhVE73kk7hHOUNMkf4WvoT6Il0a9PfI69nleDpNNfQRAYZVbJRrbt0Q9bg05YsNv2cYHip/zydNl+R2f0Sx7vBMyS0jtVVDRerqTJAQ51JQAid98IoEKbTxmCtDc9K43a4jU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; c=relaxed/simple; bh=2+Yr/2kIpnCitwTGtyjeQMnDH8wv+vGuRcW8RKqXkZ0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=GLUlkreF4SexHslc9qLyWjvh6cNaJJajrHWV5jngFxmOZV3V7swgHOWCj9lwVgwRw8Xb41p8Nhzeyrk2CsRN2cxkxxEGSHCm7ZV8ZF3hpnWJaJcpsouz2pdf7G2RyKNNSV2cCQTX7SiMT4gMjfNWWZDwL7ptStmD9gZCu4MpnJM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=JAv9YyDT; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="JAv9YyDT" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2D8D1C4CEFD; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739953972; bh=2+Yr/2kIpnCitwTGtyjeQMnDH8wv+vGuRcW8RKqXkZ0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=JAv9YyDTuiprHRGNMZ1qjSEkGj5oa4aBfrz++fFKJtI/ZIdmEmWythQGKFXJXdmGk FgZn2VLwTTpIV4cc9f2zHjBQhctoYGHldwnhrM5AMO9u7qiXCTCTtGxBy7/bFIUYNv qyB//s3GynHuz7+lB4oTJnvAi8JRAZdthit9ARfSGranTS2hEPSaS2AtTOqBMoCaa8 hjkQn87O9EuTexMMa88OYIXlhP4dSicmx52SUag2UuNvitVu+fX7vVewcd3djyS9KX misqQe3v7ne663QT6uCDeSNtLdUjbjLMsrwIWzcysynzOyJwYlcPguV7xxO0Gvwhzd PMwhQw88NODKw== Received: from mchehab by mail.kernel.org with local (Exim 4.98) (envelope-from ) id 1tkfVi-0000000Gv5F-1PUT; Wed, 19 Feb 2025 09:32:50 +0100 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , "Mauro Carvalho Chehab" , linux-kernel@vger.kernel.org Subject: [PATCH 15/27] scripts/kernel-doc.py: move KernelFiles class to a separate file Date: Wed, 19 Feb 2025 09:32:31 +0100 Message-ID: X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" The KernelFiles class is the main dispatcher which parses each source file. In preparation for letting kerneldoc Sphinx extension to import Python libraries, move regex ancillary classes to a separate file. Signed-off-by: Mauro Carvalho Chehab --- scripts/kernel-doc.py | 220 +-------------------------- scripts/lib/kdoc/kdoc_files.py | 269 +++++++++++++++++++++++++++++++++ 2 files changed, 270 insertions(+), 219 deletions(-) create mode 100755 scripts/lib/kdoc/kdoc_files.py diff --git a/scripts/kernel-doc.py b/scripts/kernel-doc.py index 193a30fcfb7c..cd79b2c1b746 100755 --- a/scripts/kernel-doc.py +++ b/scripts/kernel-doc.py @@ -44,6 +44,7 @@ sys.path.insert(0, os.path.join(SRC_DIR, LIB_DIR)) =20 from kdoc_parser import KernelDoc, type_param from kdoc_re import Re +from kdoc_files import KernelFiles =20 function_pointer =3D Re(r"([^\(]*\(\*)\s*\)\s*\(([^\)]*)\)", cache=3DFalse) =20 @@ -68,225 +69,6 @@ type_member =3D Re(r"\&([_\w]+)(\.|->)([_\w]+)", cache= =3DFalse) type_fallback =3D Re(r"\&([_\w]+)", cache=3DFalse) type_member_func =3D type_member + Re(r"\(\)", cache=3DFalse) =20 -class GlobSourceFiles: - """ - Parse C source code file names and directories via an Interactor. - - """ - - def __init__(self, srctree=3DNone, valid_extensions=3DNone): - """ - Initialize valid extensions with a tuple. - - If not defined, assume default C extensions (.c and .h) - - It would be possible to use python's glob function, but it is - very slow, and it is not interactive. So, it would wait to read all - directories before actually do something. - - So, let's use our own implementation. - """ - - if not valid_extensions: - self.extensions =3D (".c", ".h") - else: - self.extensions =3D valid_extensions - - self.srctree =3D srctree - - def _parse_dir(self, dirname): - """Internal function to parse files recursively""" - - with os.scandir(dirname) as obj: - for entry in obj: - name =3D os.path.join(dirname, entry.name) - - if entry.is_dir(): - yield from self._parse_dir(name) - - if not entry.is_file(): - continue - - basename =3D os.path.basename(name) - - if not basename.endswith(self.extensions): - continue - - yield name - - def parse_files(self, file_list, file_not_found_cb): - for fname in file_list: - if self.srctree: - f =3D os.path.join(self.srctree, fname) - else: - f =3D fname - - if os.path.isdir(f): - yield from self._parse_dir(f) - elif os.path.isfile(f): - yield f - elif file_not_found_cb: - file_not_found_cb(fname) - - -class KernelFiles(): - - def parse_file(self, fname): - - doc =3D KernelDoc(self.config, fname) - doc.run() - - return doc - - def process_export_file(self, fname): - try: - with open(fname, "r", encoding=3D"utf8", - errors=3D"backslashreplace") as fp: - for line in fp: - KernelDoc.process_export(self.config.function_table, l= ine) - - except IOError: - print(f"Error: Cannot open fname {fname}", fname=3Dsys.stderr) - self.config.errors +=3D 1 - - def file_not_found_cb(self, fname): - self.config.log.error("Cannot find file %s", fname) - self.config.errors +=3D 1 - - def __init__(self, files=3DNone, verbose=3DFalse, out_style=3DNone, - werror=3DFalse, wreturn=3DFalse, wshort_desc=3DFalse, - wcontents_before_sections=3DFalse, - logger=3DNone, modulename=3DNone, export_file=3DNone): - """Initialize startup variables and parse all files""" - - - if not verbose: - verbose =3D bool(os.environ.get("KBUILD_VERBOSE", 0)) - - if not modulename: - modulename =3D "Kernel API" - - dt =3D datetime.now() - if os.environ.get("KBUILD_BUILD_TIMESTAMP", None): - # use UTC TZ - to_zone =3D tz.gettz('UTC') - dt =3D dt.astimezone(to_zone) - - if not werror: - kcflags =3D os.environ.get("KCFLAGS", None) - if kcflags: - match =3D re.search(r"(\s|^)-Werror(\s|$)/", kcflags) - if match: - werror =3D True - - # reading this variable is for backwards compat just in case - # someone was calling it with the variable from outside the - # kernel's build system - kdoc_werror =3D os.environ.get("KDOC_WERROR", None) - if kdoc_werror: - werror =3D kdoc_werror - - # Set global config data used on all files - self.config =3D argparse.Namespace - - self.config.verbose =3D verbose - self.config.werror =3D werror - self.config.wreturn =3D wreturn - self.config.wshort_desc =3D wshort_desc - self.config.wcontents_before_sections =3D wcontents_before_sections - self.config.modulename =3D modulename - - self.config.function_table =3D set() - self.config.source_map =3D {} - - if not logger: - self.config.log =3D logging.getLogger("kernel-doc") - else: - self.config.log =3D logger - - self.config.kernel_version =3D os.environ.get("KERNELVERSION", - "unknown kernel versio= n'") - self.config.src_tree =3D os.environ.get("SRCTREE", None) - - self.out_style =3D out_style - self.export_file =3D export_file - - # Initialize internal variables - - self.config.errors =3D 0 - self.results =3D [] - - self.file_list =3D files - self.files =3D set() - - def parse(self): - """ - Parse all files - """ - - glob =3D GlobSourceFiles(srctree=3Dself.config.src_tree) - - # Let's use a set here to avoid duplicating files - - for fname in glob.parse_files(self.file_list, self.file_not_found_= cb): - if fname in self.files: - continue - - self.files.add(fname) - - res =3D self.parse_file(fname) - self.results.append((res.fname, res.entries)) - - if not self.files: - sys.exit(1) - - # If a list of export files was provided, parse EXPORT_SYMBOL* - # from the ones not already parsed - - if self.export_file: - files =3D self.files - - glob =3D GlobSourceFiles(srctree=3Dself.config.src_tree) - - for fname in glob.parse_files(self.export_file, - self.file_not_found_cb): - if fname not in files: - files.add(fname) - - self.process_export_file(fname) - - def out_msg(self, fname, name, arg): - # TODO: filter out unwanted parts - - return self.out_style.msg(fname, name, arg) - - def msg(self, enable_lineno=3DFalse, export=3DFalse, internal=3DFalse, - symbol=3DNone, nosymbol=3DNone): - - function_table =3D self.config.function_table - - if symbol: - for s in symbol: - function_table.add(s) - - # Output none mode: only warnings will be shown - if not self.out_style: - return - - self.out_style.set_config(self.config) - - self.out_style.set_filter(export, internal, symbol, nosymbol, - function_table, enable_lineno) - - for fname, arg_tuple in self.results: - for name, arg in arg_tuple: - if self.out_msg(fname, name, arg): - ln =3D arg.get("ln", 0) - dtype =3D arg.get('type', "") - - self.config.log.warning("%s:%d Can't handle %s", - fname, ln, dtype) - =20 class OutputFormat: # output mode. diff --git a/scripts/lib/kdoc/kdoc_files.py b/scripts/lib/kdoc/kdoc_files.py new file mode 100755 index 000000000000..76dd53611c08 --- /dev/null +++ b/scripts/lib/kdoc/kdoc_files.py @@ -0,0 +1,269 @@ +#!/usr/bin/env python3 +# pylint: disable=3DR0903,R0913,R0914,R0917 +# Copyright(c) 2025: Mauro Carvalho Chehab . +# SPDX-License-Identifier: GPL-2.0 + +# TODO: implement warning filtering + +""" +Parse lernel-doc tags on multiple kernel source files. +""" + +import argparse +import logging +import os +import re +import sys +from datetime import datetime + +from dateutil import tz + +from kdoc_parser import KernelDoc + + +class GlobSourceFiles: + """ + Parse C source code file names and directories via an Interactor. + """ + + def __init__(self, srctree=3DNone, valid_extensions=3DNone): + """ + Initialize valid extensions with a tuple. + + If not defined, assume default C extensions (.c and .h) + + It would be possible to use python's glob function, but it is + very slow, and it is not interactive. So, it would wait to read all + directories before actually do something. + + So, let's use our own implementation. + """ + + if not valid_extensions: + self.extensions =3D (".c", ".h") + else: + self.extensions =3D valid_extensions + + self.srctree =3D srctree + + def _parse_dir(self, dirname): + """Internal function to parse files recursively""" + + with os.scandir(dirname) as obj: + for entry in obj: + name =3D os.path.join(dirname, entry.name) + + if entry.is_dir(): + yield from self._parse_dir(name) + + if not entry.is_file(): + continue + + basename =3D os.path.basename(name) + + if not basename.endswith(self.extensions): + continue + + yield name + + def parse_files(self, file_list, file_not_found_cb): + """ + Define an interator to parse all source files from file_list, + handling directories if any + """ + + for fname in file_list: + if self.srctree: + f =3D os.path.join(self.srctree, fname) + else: + f =3D fname + + if os.path.isdir(f): + yield from self._parse_dir(f) + elif os.path.isfile(f): + yield f + elif file_not_found_cb: + file_not_found_cb(fname) + + +class KernelFiles(): + """ + Parse lernel-doc tags on multiple kernel source files. + """ + + def parse_file(self, fname): + """ + Parse a single Kernel source. + """ + + doc =3D KernelDoc(self.config, fname) + doc.run() + + return doc + + def process_export_file(self, fname): + """ + Parses EXPORT_SYMBOL* macros from a single Kernel source file. + """ + try: + with open(fname, "r", encoding=3D"utf8", + errors=3D"backslashreplace") as fp: + for line in fp: + KernelDoc.process_export(self.config.function_table, l= ine) + + except IOError: + print(f"Error: Cannot open fname {fname}", fname=3Dsys.stderr) + self.config.errors +=3D 1 + + def file_not_found_cb(self, fname): + """ + Callback to warn if a file was not found. + """ + + self.config.log.error("Cannot find file %s", fname) + self.config.errors +=3D 1 + + def __init__(self, files=3DNone, verbose=3DFalse, out_style=3DNone, + werror=3DFalse, wreturn=3DFalse, wshort_desc=3DFalse, + wcontents_before_sections=3DFalse, + logger=3DNone, modulename=3DNone, export_file=3DNone): + """ + Initialize startup variables and parse all files + """ + + if not verbose: + verbose =3D bool(os.environ.get("KBUILD_VERBOSE", 0)) + + if not modulename: + modulename =3D "Kernel API" + + dt =3D datetime.now() + if os.environ.get("KBUILD_BUILD_TIMESTAMP", None): + # use UTC TZ + to_zone =3D tz.gettz('UTC') + dt =3D dt.astimezone(to_zone) + + if not werror: + kcflags =3D os.environ.get("KCFLAGS", None) + if kcflags: + match =3D re.search(r"(\s|^)-Werror(\s|$)/", kcflags) + if match: + werror =3D True + + # reading this variable is for backwards compat just in case + # someone was calling it with the variable from outside the + # kernel's build system + kdoc_werror =3D os.environ.get("KDOC_WERROR", None) + if kdoc_werror: + werror =3D kdoc_werror + + # Set global config data used on all files + self.config =3D argparse.Namespace + + self.config.verbose =3D verbose + self.config.werror =3D werror + self.config.wreturn =3D wreturn + self.config.wshort_desc =3D wshort_desc + self.config.wcontents_before_sections =3D wcontents_before_sections + self.config.modulename =3D modulename + + self.config.function_table =3D set() + self.config.source_map =3D {} + + if not logger: + self.config.log =3D logging.getLogger("kernel-doc") + else: + self.config.log =3D logger + + self.config.kernel_version =3D os.environ.get("KERNELVERSION", + "unknown kernel versio= n'") + self.config.src_tree =3D os.environ.get("SRCTREE", None) + + self.out_style =3D out_style + self.export_file =3D export_file + + # Initialize internal variables + + self.config.errors =3D 0 + self.results =3D [] + + self.file_list =3D files + self.files =3D set() + + def parse(self): + """ + Parse all files + """ + + glob =3D GlobSourceFiles(srctree=3Dself.config.src_tree) + + # Let's use a set here to avoid duplicating files + + for fname in glob.parse_files(self.file_list, self.file_not_found_= cb): + if fname in self.files: + continue + + self.files.add(fname) + + res =3D self.parse_file(fname) + self.results.append((res.fname, res.entries)) + + if not self.files: + sys.exit(1) + + # If a list of export files was provided, parse EXPORT_SYMBOL* + # from the ones not already parsed + + if self.export_file: + files =3D self.files + + glob =3D GlobSourceFiles(srctree=3Dself.config.src_tree) + + for fname in glob.parse_files(self.export_file, + self.file_not_found_cb): + if fname not in files: + files.add(fname) + + self.process_export_file(fname) + + def out_msg(self, fname, name, arg): + """ + Output messages from a file name using the output style filtering. + + If output type was not handled by the syler, return False. + """ + + # NOTE: we can add rules here to filter out unwanted parts, + # although OutputFormat.msg already does that. + + return self.out_style.msg(fname, name, arg) + + def msg(self, enable_lineno=3DFalse, export=3DFalse, internal=3DFalse, + symbol=3DNone, nosymbol=3DNone): + """ + Interacts over the kernel-doc results and output messages. + """ + + function_table =3D self.config.function_table + + if symbol: + for s in symbol: + function_table.add(s) + + # Output none mode: only warnings will be shown + if not self.out_style: + return + + self.out_style.set_config(self.config) + + self.out_style.set_filter(export, internal, symbol, nosymbol, + function_table, enable_lineno) + + for fname, arg_tuple in self.results: + for name, arg in arg_tuple: + if self.out_msg(fname, name, arg): + ln =3D arg.get("ln", 0) + dtype =3D arg.get('type', "") + + self.config.log.warning("%s:%d Can't handle %s", + fname, ln, dtype) --=20 2.48.1 From nobody Sat Feb 7 14:01:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F2F2F1DE2DE; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953973; cv=none; b=eoAAeKc6RPRaMnKzo07QoiGHPoYzXjDX+8t9OmbQHrsSQv/LkR2Wuj0imXNuo7+/jSKDH+/81UVCWrjyt68NKsGMKggbn7rL3nqwMfXmstRSk43i7W6iUhiWQa8DN0kvtJOijgWEketmvpNv9r1u44pz9gFUJv/v001AuW9zBMs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953973; c=relaxed/simple; bh=hPv6z9fZFY9mkuOaJfh9tzYOURGM+7iDxiBMcb2e6xg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=jYBFtSE2nJLBzcnFVpKO1GIEgbJRvRDw+sCerK95rDgx1oMSeibZNQ42zTFjka3pkyX7ERWUPEqtDbBXcYpjYRXiwNbY1Ghld4KfS1Fo+CALxRg2IJXEe2lbbndG/eiPSi7PVXyC8zVbULGtWXOwrGYYyX22JOsvryDlA1O+tNg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=YYpsAxUp; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="YYpsAxUp" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 39C4EC4CEFE; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739953972; bh=hPv6z9fZFY9mkuOaJfh9tzYOURGM+7iDxiBMcb2e6xg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YYpsAxUpm0rM5lweENK33/SIDOyvQRVtVjMsDoxk8j35T49ObulauvT92N80h0DgL vcvGkqJjaXHbdoCtu8bgvLqyE60ppiAlqGdRRy/dJwbu/8Nttt0X3+WBEtNUjxOXWb nyA8vOrZc4iYwEryJfO1/qTDKKIffWYE1+CLVsbw2RoxuNrzbV2sCDdHckfAIOOKnj G3VFPxIAuoyyU+mOk9DU8ZiJXZJE4M2xo6Twb7VZcxJeXsLiaGvI7bG5U7kKPlsivs cT5mnHtCiIS23iav9snjd9IMQNrT6L+JbqOwQG6VamXE9ChRpn/ttX7Q97wxZ/uATA XRgTTgz0MY/gg== Received: from mchehab by mail.kernel.org with local (Exim 4.98) (envelope-from ) id 1tkfVi-0000000Gv5J-1Wso; Wed, 19 Feb 2025 09:32:50 +0100 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , "Mauro Carvalho Chehab" , linux-kernel@vger.kernel.org Subject: [PATCH 16/27] scripts/kernel-doc.py: move output classes to a separate file Date: Wed, 19 Feb 2025 09:32:32 +0100 Message-ID: <64f56c12fef462cc9e22d7f11ae0ff818a4081ad.1739952783.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" In preparation for letting kerneldoc Sphinx extension to import Python libraries, move kernel-doc output logic to a separate file. Signed-off-by: Mauro Carvalho Chehab --- scripts/kernel-doc.py | 727 +------------------------------ scripts/lib/kdoc/kdoc_output.py | 735 ++++++++++++++++++++++++++++++++ 2 files changed, 738 insertions(+), 724 deletions(-) create mode 100755 scripts/lib/kdoc/kdoc_output.py diff --git a/scripts/kernel-doc.py b/scripts/kernel-doc.py index cd79b2c1b746..0596c711d448 100755 --- a/scripts/kernel-doc.py +++ b/scripts/kernel-doc.py @@ -1,7 +1,5 @@ #!/usr/bin/env python3 -# pylint: disable=3DR0902,R0903,R0904,R0911,R0912,R0913,R0914,R0915,R0917,= R1702 -# pylint: disable=3DC0302,C0103,C0301 -# pylint: disable=3DC0116,C0115,W0511,W0613 +# pylint: disable=3DC0103, # Copyright(c) 2025: Mauro Carvalho Chehab . # SPDX-License-Identifier: GPL-2.0 =20 @@ -27,14 +25,8 @@ documentation comment syntax. import argparse import logging import os -import re import sys =20 -from datetime import datetime -from pprint import pformat - -from dateutil import tz - # Import Python modules =20 LIB_DIR =3D "lib/kdoc" @@ -42,721 +34,8 @@ SRC_DIR =3D os.path.dirname(os.path.realpath(__file__)) =20 sys.path.insert(0, os.path.join(SRC_DIR, LIB_DIR)) =20 -from kdoc_parser import KernelDoc, type_param -from kdoc_re import Re -from kdoc_files import KernelFiles - -function_pointer =3D Re(r"([^\(]*\(\*)\s*\)\s*\(([^\)]*)\)", cache=3DFalse) - -# match expressions used to find embedded type information -type_constant =3D Re(r"\b``([^\`]+)``\b", cache=3DFalse) -type_constant2 =3D Re(r"\%([-_*\w]+)", cache=3DFalse) -type_func =3D Re(r"(\w+)\(\)", cache=3DFalse) -type_param_ref =3D Re(r"([\!~\*]?)\@(\w*((\.\w+)|(->\w+))*(\.\.\.)?)", cac= he=3DFalse) - -# Special RST handling for func ptr params -type_fp_param =3D Re(r"\@(\w+)\(\)", cache=3DFalse) - -# Special RST handling for structs with func ptr params -type_fp_param2 =3D Re(r"\@(\w+->\S+)\(\)", cache=3DFalse) - -type_env =3D Re(r"(\$\w+)", cache=3DFalse) -type_enum =3D Re(r"\&(enum\s*([_\w]+))", cache=3DFalse) -type_struct =3D Re(r"\&(struct\s*([_\w]+))", cache=3DFalse) -type_typedef =3D Re(r"\&(typedef\s*([_\w]+))", cache=3DFalse) -type_union =3D Re(r"\&(union\s*([_\w]+))", cache=3DFalse) -type_member =3D Re(r"\&([_\w]+)(\.|->)([_\w]+)", cache=3DFalse) -type_fallback =3D Re(r"\&([_\w]+)", cache=3DFalse) -type_member_func =3D type_member + Re(r"\(\)", cache=3DFalse) - - -class OutputFormat: - # output mode. - OUTPUT_ALL =3D 0 # output all symbols and doc sections - OUTPUT_INCLUDE =3D 1 # output only specified symbols - OUTPUT_EXPORTED =3D 2 # output exported symbols - OUTPUT_INTERNAL =3D 3 # output non-exported symbols - - # Virtual member to be overriden at the inherited classes - highlights =3D [] - - def __init__(self): - """Declare internal vars and set mode to OUTPUT_ALL""" - - self.out_mode =3D self.OUTPUT_ALL - self.enable_lineno =3D None - self.nosymbol =3D {} - self.symbol =3D None - self.function_table =3D set() - self.config =3D None - - def set_config(self, config): - self.config =3D config - - def set_filter(self, export, internal, symbol, nosymbol, function_tabl= e, - enable_lineno): - """ - Initialize filter variables according with the requested mode. - - Only one choice is valid between export, internal and symbol. - - The nosymbol filter can be used on all modes. - """ - - self.enable_lineno =3D enable_lineno - - if symbol: - self.out_mode =3D self.OUTPUT_INCLUDE - function_table =3D symbol - elif export: - self.out_mode =3D self.OUTPUT_EXPORTED - elif internal: - self.out_mode =3D self.OUTPUT_INTERNAL - else: - self.out_mode =3D self.OUTPUT_ALL - - if nosymbol: - self.nosymbol =3D set(nosymbol) - - if function_table: - self.function_table =3D function_table - - def highlight_block(self, block): - """ - Apply the RST highlights to a sub-block of text. - """ - - for r, sub in self.highlights: - block =3D r.sub(sub, block) - - return block - - def check_doc(self, name): - """Check if DOC should be output""" - - if self.out_mode =3D=3D self.OUTPUT_ALL: - return True - - if self.out_mode =3D=3D self.OUTPUT_INCLUDE: - if name in self.nosymbol: - return False - - if name in self.function_table: - return True - - return False - - def check_declaration(self, dtype, name): - if name in self.nosymbol: - return False - - if self.out_mode =3D=3D self.OUTPUT_ALL: - return True - - if self.out_mode in [ self.OUTPUT_INCLUDE, self.OUTPUT_EXPORTED ]: - if name in self.function_table: - return True - - if self.out_mode =3D=3D self.OUTPUT_INTERNAL: - if dtype !=3D "function": - return True - - if name not in self.function_table: - return True - - return False - - def check_function(self, fname, name, args): - return True - - def check_enum(self, fname, name, args): - return True - - def check_typedef(self, fname, name, args): - return True - - def msg(self, fname, name, args): - - dtype =3D args.get('type', "") - - if dtype =3D=3D "doc": - self.out_doc(fname, name, args) - return False - - if not self.check_declaration(dtype, name): - return False - - if dtype =3D=3D "function": - self.out_function(fname, name, args) - return False - - if dtype =3D=3D "enum": - self.out_enum(fname, name, args) - return False - - if dtype =3D=3D "typedef": - self.out_typedef(fname, name, args) - return False - - if dtype in ["struct", "union"]: - self.out_struct(fname, name, args) - return False - - # Warn if some type requires an output logic - self.config.log.warning("doesn't now how to output '%s' block", - dtype) - - return True - - # Virtual methods to be overridden by inherited classes - def out_doc(self, fname, name, args): - pass - - def out_function(self, fname, name, args): - pass - - def out_enum(self, fname, name, args): - pass - - def out_typedef(self, fname, name, args): - pass - - def out_struct(self, fname, name, args): - pass - - -class RestFormat(OutputFormat): - # """Consts and functions used by ReST output""" - - highlights =3D [ - (type_constant, r"``\1``"), - (type_constant2, r"``\1``"), - - # Note: need to escape () to avoid func matching later - (type_member_func, r":c:type:`\1\2\3\\(\\) <\1>`"), - (type_member, r":c:type:`\1\2\3 <\1>`"), - (type_fp_param, r"**\1\\(\\)**"), - (type_fp_param2, r"**\1\\(\\)**"), - (type_func, r"\1()"), - (type_enum, r":c:type:`\1 <\2>`"), - (type_struct, r":c:type:`\1 <\2>`"), - (type_typedef, r":c:type:`\1 <\2>`"), - (type_union, r":c:type:`\1 <\2>`"), - - # in rst this can refer to any type - (type_fallback, r":c:type:`\1`"), - (type_param_ref, r"**\1\2**") - ] - blankline =3D "\n" - - sphinx_literal =3D Re(r'^[^.].*::$', cache=3DFalse) - sphinx_cblock =3D Re(r'^\.\.\ +code-block::', cache=3DFalse) - - def __init__(self): - """ - Creates class variables. - - Not really mandatory, but it is a good coding style and makes - pylint happy. - """ - - super().__init__() - self.lineprefix =3D "" - - def print_lineno (self, ln): - """Outputs a line number""" - - if self.enable_lineno and ln: - print(f".. LINENO {ln}") - - def output_highlight(self, args): - input_text =3D args - output =3D "" - in_literal =3D False - litprefix =3D "" - block =3D "" - - for line in input_text.strip("\n").split("\n"): - - # If we're in a literal block, see if we should drop out of it. - # Otherwise, pass the line straight through unmunged. - if in_literal: - if line.strip(): # If the line is not blank - # If this is the first non-blank line in a literal blo= ck, - # figure out the proper indent. - if not litprefix: - r =3D Re(r'^(\s*)') - if r.match(line): - litprefix =3D '^' + r.group(1) - else: - litprefix =3D "" - - output +=3D line + "\n" - elif not Re(litprefix).match(line): - in_literal =3D False - else: - output +=3D line + "\n" - else: - output +=3D line + "\n" - - # Not in a literal block (or just dropped out) - if not in_literal: - block +=3D line + "\n" - if self.sphinx_literal.match(line) or self.sphinx_cblock.m= atch(line): - in_literal =3D True - litprefix =3D "" - output +=3D self.highlight_block(block) - block =3D "" - - # Handle any remaining block - if block: - output +=3D self.highlight_block(block) - - # Print the output with the line prefix - for line in output.strip("\n").split("\n"): - print(self.lineprefix + line) - - def out_section(self, args, out_reference=3DFalse): - """ - Outputs a block section. - - This could use some work; it's used to output the DOC: sections, a= nd - starts by putting out the name of the doc section itself, but that - tends to duplicate a header already in the template file. - """ - - sectionlist =3D args.get('sectionlist', []) - sections =3D args.get('sections', {}) - section_start_lines =3D args.get('section_start_lines', {}) - - for section in sectionlist: - # Skip sections that are in the nosymbol_table - if section in self.nosymbol: - continue - - if not self.out_mode =3D=3D self.OUTPUT_INCLUDE: - if out_reference: - print(f".. _{section}:\n") - - if not self.symbol: - print(f'{self.lineprefix}**{section}**\n') - - self.print_lineno(section_start_lines.get(section, 0)) - self.output_highlight(sections[section]) - print() - print() - - def out_doc(self, fname, name, args): - if not self.check_doc(name): - return - - self.out_section(args, out_reference=3DTrue) - - def out_function(self, fname, name, args): - - oldprefix =3D self.lineprefix - signature =3D "" - - func_macro =3D args.get('func_macro', False) - if func_macro: - signature =3D args['function'] - else: - if args.get('functiontype'): - signature =3D args['functiontype'] + " " - signature +=3D args['function'] + " (" - - parameterlist =3D args.get('parameterlist', []) - parameterdescs =3D args.get('parameterdescs', {}) - parameterdesc_start_lines =3D args.get('parameterdesc_start_lines'= , {}) - - ln =3D args.get('ln', 0) - - count =3D 0 - for parameter in parameterlist: - if count !=3D 0: - signature +=3D ", " - count +=3D 1 - dtype =3D args['parametertypes'].get(parameter, "") - - if function_pointer.search(dtype): - signature +=3D function_pointer.group(1) + parameter + fun= ction_pointer.group(3) - else: - signature +=3D dtype - - if not func_macro: - signature +=3D ")" - - if args.get('typedef') or not args.get('functiontype'): - print(f".. c:macro:: {args['function']}\n") - - if args.get('typedef'): - self.print_lineno(ln) - print(" **Typedef**: ", end=3D"") - self.lineprefix =3D "" - self.output_highlight(args.get('purpose', "")) - print("\n\n**Syntax**\n") - print(f" ``{signature}``\n") - else: - print(f"``{signature}``\n") - else: - print(f".. c:function:: {signature}\n") - - if not args.get('typedef'): - self.print_lineno(ln) - self.lineprefix =3D " " - self.output_highlight(args.get('purpose', "")) - print() - - # Put descriptive text into a container (HTML
) to help set - # function prototypes apart - self.lineprefix =3D " " - - if parameterlist: - print(".. container:: kernelindent\n") - print(f"{self.lineprefix}**Parameters**\n") - - for parameter in parameterlist: - parameter_name =3D Re(r'\[.*').sub('', parameter) - dtype =3D args['parametertypes'].get(parameter, "") - - if dtype: - print(f"{self.lineprefix}``{dtype}``") - else: - print(f"{self.lineprefix}``{parameter}``") - - self.print_lineno(parameterdesc_start_lines.get(parameter_name= , 0)) - - self.lineprefix =3D " " - if parameter_name in parameterdescs and \ - parameterdescs[parameter_name] !=3D KernelDoc.undescribed: - - self.output_highlight(parameterdescs[parameter_name]) - print() - else: - print(f"{self.lineprefix}*undescribed*\n") - self.lineprefix =3D " " - - self.out_section(args) - self.lineprefix =3D oldprefix - - def out_enum(self, fname, name, args): - - oldprefix =3D self.lineprefix - name =3D args.get('enum', '') - parameterlist =3D args.get('parameterlist', []) - parameterdescs =3D args.get('parameterdescs', {}) - ln =3D args.get('ln', 0) - - print(f"\n\n.. c:enum:: {name}\n") - - self.print_lineno(ln) - self.lineprefix =3D " " - self.output_highlight(args.get('purpose', '')) - print() - - print(".. container:: kernelindent\n") - outer =3D self.lineprefix + " " - self.lineprefix =3D outer + " " - print(f"{outer}**Constants**\n") - - for parameter in parameterlist: - print(f"{outer}``{parameter}``") - - if parameterdescs.get(parameter, '') !=3D KernelDoc.undescribe= d: - self.output_highlight(parameterdescs[parameter]) - else: - print(f"{self.lineprefix}*undescribed*\n") - print() - - self.lineprefix =3D oldprefix - self.out_section(args) - - def out_typedef(self, fname, name, args): - - oldprefix =3D self.lineprefix - name =3D args.get('typedef', '') - ln =3D args.get('ln', 0) - - print(f"\n\n.. c:type:: {name}\n") - - self.print_lineno(ln) - self.lineprefix =3D " " - - self.output_highlight(args.get('purpose', '')) - - print() - - self.lineprefix =3D oldprefix - self.out_section(args) - - def out_struct(self, fname, name, args): - - name =3D args.get('struct', "") - purpose =3D args.get('purpose', "") - declaration =3D args.get('definition', "") - dtype =3D args.get('type', "struct") - ln =3D args.get('ln', 0) - - parameterlist =3D args.get('parameterlist', []) - parameterdescs =3D args.get('parameterdescs', {}) - parameterdesc_start_lines =3D args.get('parameterdesc_start_lines'= , {}) - - print(f"\n\n.. c:{dtype}:: {name}\n") - - self.print_lineno(ln) - - oldprefix =3D self.lineprefix - self.lineprefix +=3D " " - - self.output_highlight(purpose) - print() - - print(".. container:: kernelindent\n") - print(f"{self.lineprefix}**Definition**::\n") - - self.lineprefix =3D self.lineprefix + " " - - declaration =3D declaration.replace("\t", self.lineprefix) - - print(f"{self.lineprefix}{dtype} {name}" + ' {') - print(f"{declaration}{self.lineprefix}" + "};\n") - - self.lineprefix =3D " " - print(f"{self.lineprefix}**Members**\n") - for parameter in parameterlist: - if not parameter or parameter.startswith("#"): - continue - - parameter_name =3D parameter.split("[", maxsplit=3D1)[0] - - if parameterdescs.get(parameter_name) =3D=3D KernelDoc.undescr= ibed: - continue - - self.print_lineno(parameterdesc_start_lines.get(parameter_name= , 0)) - - print(f"{self.lineprefix}``{parameter}``") - - self.lineprefix =3D " " - self.output_highlight(parameterdescs[parameter_name]) - self.lineprefix =3D " " - - print() - - print() - - self.lineprefix =3D oldprefix - self.out_section(args) - - -class ManFormat(OutputFormat): - """Consts and functions used by man pages output""" - - highlights =3D ( - (type_constant, r"\1"), - (type_constant2, r"\1"), - (type_func, r"\\fB\1\\fP"), - (type_enum, r"\\fI\1\\fP"), - (type_struct, r"\\fI\1\\fP"), - (type_typedef, r"\\fI\1\\fP"), - (type_union, r"\\fI\1\\fP"), - (type_param, r"\\fI\1\\fP"), - (type_param_ref, r"\\fI\1\2\\fP"), - (type_member, r"\\fI\1\2\3\\fP"), - (type_fallback, r"\\fI\1\\fP") - ) - blankline =3D "" - - def __init__(self): - """ - Creates class variables. - - Not really mandatory, but it is a good coding style and makes - pylint happy. - """ - - super().__init__() - - dt =3D datetime.now() - if os.environ.get("KBUILD_BUILD_TIMESTAMP", None): - # use UTC TZ - to_zone =3D tz.gettz('UTC') - dt =3D dt.astimezone(to_zone) - - self.man_date =3D dt.strftime("%B %Y") - - def output_highlight(self, block): - - contents =3D self.highlight_block(block) - - if isinstance(contents, list): - contents =3D "\n".join(contents) - - for line in contents.strip("\n").split("\n"): - line =3D Re(r"^\s*").sub("", line) - - if line and line[0] =3D=3D ".": - print("\\&" + line) - else: - print(line) - - def out_doc(self, fname, name, args): - module =3D args.get('module') - sectionlist =3D args.get('sectionlist', []) - sections =3D args.get('sections', {}) - - print(f'.TH "{module}" 9 "{module}" "{self.man_date}" "API Manual"= LINUX') - - for section in sectionlist: - print(f'.SH "{section}"') - self.output_highlight(sections.get(section)) - - def out_function(self, fname, name, args): - """output function in man""" - - parameterlist =3D args.get('parameterlist', []) - parameterdescs =3D args.get('parameterdescs', {}) - sectionlist =3D args.get('sectionlist', []) - sections =3D args.get('sections', {}) - - print(f'.TH "{args['function']}" 9 "{args['function']}" "{self.man= _date}" "Kernel Hacker\'s Manual" LINUX') - - print(".SH NAME") - print(f"{args['function']} \\- {args['purpose']}") - - print(".SH SYNOPSIS") - if args.get('functiontype', ''): - print(f'.B "{args['functiontype']}" {args['function']}') - else: - print(f'.B "{args['function']}') - - count =3D 0 - parenth =3D "(" - post =3D "," - - for parameter in parameterlist: - if count =3D=3D len(parameterlist) - 1: - post =3D ");" - - dtype =3D args['parametertypes'].get(parameter, "") - if function_pointer.match(dtype): - # Pointer-to-function - print(f'".BI "{parenth}{function_pointer.group(1)}" " ") (= {function_pointer.group(2)}){post}"') - else: - dtype =3D Re(r'([^\*])$').sub(r'\1 ', dtype) - - print(f'.BI "{parenth}{dtype}" "{post}"') - count +=3D 1 - parenth =3D "" - - if parameterlist: - print(".SH ARGUMENTS") - - for parameter in parameterlist: - parameter_name =3D re.sub(r'\[.*', '', parameter) - - print(f'.IP "{parameter}" 12') - self.output_highlight(parameterdescs.get(parameter_name, "")) - - for section in sectionlist: - print(f'.SH "{section.upper()}"') - self.output_highlight(sections[section]) - - def out_enum(self, fname, name, args): - - name =3D args.get('enum', '') - parameterlist =3D args.get('parameterlist', []) - sectionlist =3D args.get('sectionlist', []) - sections =3D args.get('sections', {}) - - print(f'.TH "{args['module']}" 9 "enum {args['enum']}" "{self.man_= date}" "API Manual" LINUX') - - print(".SH NAME") - print(f"enum {args['enum']} \\- {args['purpose']}") - - print(".SH SYNOPSIS") - print(f"enum {args['enum']}" + " {") - - count =3D 0 - for parameter in parameterlist: - print(f'.br\n.BI " {parameter}"') - if count =3D=3D len(parameterlist) - 1: - print("\n};") - else: - print(", \n.br") - - count +=3D 1 - - print(".SH Constants") - - for parameter in parameterlist: - parameter_name =3D Re(r'\[.*').sub('', parameter) - print(f'.IP "{parameter}" 12') - self.output_highlight(args['parameterdescs'].get(parameter_nam= e, "")) - - for section in sectionlist: - print(f'.SH "{section}"') - self.output_highlight(sections[section]) - - def out_typedef(self, fname, name, args): - module =3D args.get('module') - typedef =3D args.get('typedef') - purpose =3D args.get('purpose') - sectionlist =3D args.get('sectionlist', []) - sections =3D args.get('sections', {}) - - print(f'.TH "{module}" 9 "{typedef}" "{self.man_date}" "API Manual= " LINUX') - - print(".SH NAME") - print(f"typedef {typedef} \\- {purpose}") - - for section in sectionlist: - print(f'.SH "{section}"') - self.output_highlight(sections.get(section)) - - def out_struct(self, fname, name, args): - module =3D args.get('module') - struct_type =3D args.get('type') - struct_name =3D args.get('struct') - purpose =3D args.get('purpose') - definition =3D args.get('definition') - sectionlist =3D args.get('sectionlist', []) - parameterlist =3D args.get('parameterlist', []) - sections =3D args.get('sections', {}) - parameterdescs =3D args.get('parameterdescs', {}) - - print(f'.TH "{module}" 9 "{struct_type} {struct_name}" "{self.man_= date}" "API Manual" LINUX') - - print(".SH NAME") - print(f"{struct_type} {struct_name} \\- {purpose}") - - # Replace tabs with two spaces and handle newlines - declaration =3D definition.replace("\t", " ") - declaration =3D Re(r"\n").sub('"\n.br\n.BI "', declaration) - - print(".SH SYNOPSIS") - print(f"{struct_type} {struct_name} " + "{" +"\n.br") - print(f'.BI "{declaration}\n' + "};\n.br\n") - - print(".SH Members") - for parameter in parameterlist: - if parameter.startswith("#"): - continue - - parameter_name =3D re.sub(r"\[.*", "", parameter) - - if parameterdescs.get(parameter_name) =3D=3D KernelDoc.undescr= ibed: - continue - - print(f'.IP "{parameter}" 12') - self.output_highlight(parameterdescs.get(parameter_name)) - - for section in sectionlist: - print(f'.SH "{section}"') - self.output_highlight(sections.get(section)) - - -# Command line interface - +from kdoc_files import KernelFiles # pylint: disable= =3DC0413 +from kdoc_output import RestFormat, ManFormat # pylint: disable= =3DC0413 =20 DESC =3D """ Read C language source or header FILEs, extract embedded documentation com= ments, diff --git a/scripts/lib/kdoc/kdoc_output.py b/scripts/lib/kdoc/kdoc_output= .py new file mode 100755 index 000000000000..d080440caa1c --- /dev/null +++ b/scripts/lib/kdoc/kdoc_output.py @@ -0,0 +1,735 @@ +#!/usr/bin/env python3 +# pylint: disable=3DC0301,R0911,R0912,R0913,R0914,R0915,R0917 +# Copyright(c) 2025: Mauro Carvalho Chehab . +# SPDX-License-Identifier: GPL-2.0 + +# TODO: implement warning filtering + +""" +Implement output filters to print kernel-doc documentation. + +The implementation uses a virtual base class (OutputFormat) which +contains a dispatches to virtual methods, and some code to filter +out output messages. + +The actual implementation is done on one separate class per each type +of output. Currently, there are output classes for ReST and man/troff. +""" + +import os +import re +from datetime import datetime + +from dateutil import tz + +from kdoc_parser import KernelDoc, type_param +from kdoc_re import Re + + +function_pointer =3D Re(r"([^\(]*\(\*)\s*\)\s*\(([^\)]*)\)", cache=3DFalse) + +# match expressions used to find embedded type information +type_constant =3D Re(r"\b``([^\`]+)``\b", cache=3DFalse) +type_constant2 =3D Re(r"\%([-_*\w]+)", cache=3DFalse) +type_func =3D Re(r"(\w+)\(\)", cache=3DFalse) +type_param_ref =3D Re(r"([\!~\*]?)\@(\w*((\.\w+)|(->\w+))*(\.\.\.)?)", cac= he=3DFalse) + +# Special RST handling for func ptr params +type_fp_param =3D Re(r"\@(\w+)\(\)", cache=3DFalse) + +# Special RST handling for structs with func ptr params +type_fp_param2 =3D Re(r"\@(\w+->\S+)\(\)", cache=3DFalse) + +type_env =3D Re(r"(\$\w+)", cache=3DFalse) +type_enum =3D Re(r"\&(enum\s*([_\w]+))", cache=3DFalse) +type_struct =3D Re(r"\&(struct\s*([_\w]+))", cache=3DFalse) +type_typedef =3D Re(r"\&(typedef\s*([_\w]+))", cache=3DFalse) +type_union =3D Re(r"\&(union\s*([_\w]+))", cache=3DFalse) +type_member =3D Re(r"\&([_\w]+)(\.|->)([_\w]+)", cache=3DFalse) +type_fallback =3D Re(r"\&([_\w]+)", cache=3DFalse) +type_member_func =3D type_member + Re(r"\(\)", cache=3DFalse) + + +class OutputFormat: + # output mode. + OUTPUT_ALL =3D 0 # output all symbols and doc sections + OUTPUT_INCLUDE =3D 1 # output only specified symbols + OUTPUT_EXPORTED =3D 2 # output exported symbols + OUTPUT_INTERNAL =3D 3 # output non-exported symbols + + # Virtual member to be overriden at the inherited classes + highlights =3D [] + + def __init__(self): + """Declare internal vars and set mode to OUTPUT_ALL""" + + self.out_mode =3D self.OUTPUT_ALL + self.enable_lineno =3D None + self.nosymbol =3D {} + self.symbol =3D None + self.function_table =3D set() + self.config =3D None + + def set_config(self, config): + self.config =3D config + + def set_filter(self, export, internal, symbol, nosymbol, function_tabl= e, + enable_lineno): + """ + Initialize filter variables according with the requested mode. + + Only one choice is valid between export, internal and symbol. + + The nosymbol filter can be used on all modes. + """ + + self.enable_lineno =3D enable_lineno + + if symbol: + self.out_mode =3D self.OUTPUT_INCLUDE + function_table =3D symbol + elif export: + self.out_mode =3D self.OUTPUT_EXPORTED + elif internal: + self.out_mode =3D self.OUTPUT_INTERNAL + else: + self.out_mode =3D self.OUTPUT_ALL + + if nosymbol: + self.nosymbol =3D set(nosymbol) + + if function_table: + self.function_table =3D function_table + + def highlight_block(self, block): + """ + Apply the RST highlights to a sub-block of text. + """ + + for r, sub in self.highlights: + block =3D r.sub(sub, block) + + return block + + def check_doc(self, name): + """Check if DOC should be output""" + + if self.out_mode =3D=3D self.OUTPUT_ALL: + return True + + if self.out_mode =3D=3D self.OUTPUT_INCLUDE: + if name in self.nosymbol: + return False + + if name in self.function_table: + return True + + return False + + def check_declaration(self, dtype, name): + if name in self.nosymbol: + return False + + if self.out_mode =3D=3D self.OUTPUT_ALL: + return True + + if self.out_mode in [self.OUTPUT_INCLUDE, self.OUTPUT_EXPORTED]: + if name in self.function_table: + return True + + if self.out_mode =3D=3D self.OUTPUT_INTERNAL: + if dtype !=3D "function": + return True + + if name not in self.function_table: + return True + + return False + + def check_function(self, fname, name, args): + return True + + def check_enum(self, fname, name, args): + return True + + def check_typedef(self, fname, name, args): + return True + + def msg(self, fname, name, args): + + dtype =3D args.get('type', "") + + if dtype =3D=3D "doc": + self.out_doc(fname, name, args) + return False + + if not self.check_declaration(dtype, name): + return False + + if dtype =3D=3D "function": + self.out_function(fname, name, args) + return False + + if dtype =3D=3D "enum": + self.out_enum(fname, name, args) + return False + + if dtype =3D=3D "typedef": + self.out_typedef(fname, name, args) + return False + + if dtype in ["struct", "union"]: + self.out_struct(fname, name, args) + return False + + # Warn if some type requires an output logic + self.config.log.warning("doesn't now how to output '%s' block", + dtype) + + return True + + # Virtual methods to be overridden by inherited classes + def out_doc(self, fname, name, args): + pass + + def out_function(self, fname, name, args): + pass + + def out_enum(self, fname, name, args): + pass + + def out_typedef(self, fname, name, args): + pass + + def out_struct(self, fname, name, args): + pass + + +class RestFormat(OutputFormat): + # """Consts and functions used by ReST output""" + + highlights =3D [ + (type_constant, r"``\1``"), + (type_constant2, r"``\1``"), + + # Note: need to escape () to avoid func matching later + (type_member_func, r":c:type:`\1\2\3\\(\\) <\1>`"), + (type_member, r":c:type:`\1\2\3 <\1>`"), + (type_fp_param, r"**\1\\(\\)**"), + (type_fp_param2, r"**\1\\(\\)**"), + (type_func, r"\1()"), + (type_enum, r":c:type:`\1 <\2>`"), + (type_struct, r":c:type:`\1 <\2>`"), + (type_typedef, r":c:type:`\1 <\2>`"), + (type_union, r":c:type:`\1 <\2>`"), + + # in rst this can refer to any type + (type_fallback, r":c:type:`\1`"), + (type_param_ref, r"**\1\2**") + ] + blankline =3D "\n" + + sphinx_literal =3D Re(r'^[^.].*::$', cache=3DFalse) + sphinx_cblock =3D Re(r'^\.\.\ +code-block::', cache=3DFalse) + + def __init__(self): + """ + Creates class variables. + + Not really mandatory, but it is a good coding style and makes + pylint happy. + """ + + super().__init__() + self.lineprefix =3D "" + + def print_lineno(self, ln): + """Outputs a line number""" + + if self.enable_lineno and ln: + print(f".. LINENO {ln}") + + def output_highlight(self, args): + input_text =3D args + output =3D "" + in_literal =3D False + litprefix =3D "" + block =3D "" + + for line in input_text.strip("\n").split("\n"): + + # If we're in a literal block, see if we should drop out of it. + # Otherwise, pass the line straight through unmunged. + if in_literal: + if line.strip(): # If the line is not blank + # If this is the first non-blank line in a literal blo= ck, + # figure out the proper indent. + if not litprefix: + r =3D Re(r'^(\s*)') + if r.match(line): + litprefix =3D '^' + r.group(1) + else: + litprefix =3D "" + + output +=3D line + "\n" + elif not Re(litprefix).match(line): + in_literal =3D False + else: + output +=3D line + "\n" + else: + output +=3D line + "\n" + + # Not in a literal block (or just dropped out) + if not in_literal: + block +=3D line + "\n" + if self.sphinx_literal.match(line) or self.sphinx_cblock.m= atch(line): + in_literal =3D True + litprefix =3D "" + output +=3D self.highlight_block(block) + block =3D "" + + # Handle any remaining block + if block: + output +=3D self.highlight_block(block) + + # Print the output with the line prefix + for line in output.strip("\n").split("\n"): + print(self.lineprefix + line) + + def out_section(self, args, out_reference=3DFalse): + """ + Outputs a block section. + + This could use some work; it's used to output the DOC: sections, a= nd + starts by putting out the name of the doc section itself, but that + tends to duplicate a header already in the template file. + """ + + sectionlist =3D args.get('sectionlist', []) + sections =3D args.get('sections', {}) + section_start_lines =3D args.get('section_start_lines', {}) + + for section in sectionlist: + # Skip sections that are in the nosymbol_table + if section in self.nosymbol: + continue + + if not self.out_mode =3D=3D self.OUTPUT_INCLUDE: + if out_reference: + print(f".. _{section}:\n") + + if not self.symbol: + print(f'{self.lineprefix}**{section}**\n') + + self.print_lineno(section_start_lines.get(section, 0)) + self.output_highlight(sections[section]) + print() + print() + + def out_doc(self, fname, name, args): + if not self.check_doc(name): + return + + self.out_section(args, out_reference=3DTrue) + + def out_function(self, fname, name, args): + + oldprefix =3D self.lineprefix + signature =3D "" + + func_macro =3D args.get('func_macro', False) + if func_macro: + signature =3D args['function'] + else: + if args.get('functiontype'): + signature =3D args['functiontype'] + " " + signature +=3D args['function'] + " (" + + parameterlist =3D args.get('parameterlist', []) + parameterdescs =3D args.get('parameterdescs', {}) + parameterdesc_start_lines =3D args.get('parameterdesc_start_lines'= , {}) + + ln =3D args.get('ln', 0) + + count =3D 0 + for parameter in parameterlist: + if count !=3D 0: + signature +=3D ", " + count +=3D 1 + dtype =3D args['parametertypes'].get(parameter, "") + + if function_pointer.search(dtype): + signature +=3D function_pointer.group(1) + parameter + fun= ction_pointer.group(3) + else: + signature +=3D dtype + + if not func_macro: + signature +=3D ")" + + if args.get('typedef') or not args.get('functiontype'): + print(f".. c:macro:: {args['function']}\n") + + if args.get('typedef'): + self.print_lineno(ln) + print(" **Typedef**: ", end=3D"") + self.lineprefix =3D "" + self.output_highlight(args.get('purpose', "")) + print("\n\n**Syntax**\n") + print(f" ``{signature}``\n") + else: + print(f"``{signature}``\n") + else: + print(f".. c:function:: {signature}\n") + + if not args.get('typedef'): + self.print_lineno(ln) + self.lineprefix =3D " " + self.output_highlight(args.get('purpose', "")) + print() + + # Put descriptive text into a container (HTML
) to help set + # function prototypes apart + self.lineprefix =3D " " + + if parameterlist: + print(".. container:: kernelindent\n") + print(f"{self.lineprefix}**Parameters**\n") + + for parameter in parameterlist: + parameter_name =3D Re(r'\[.*').sub('', parameter) + dtype =3D args['parametertypes'].get(parameter, "") + + if dtype: + print(f"{self.lineprefix}``{dtype}``") + else: + print(f"{self.lineprefix}``{parameter}``") + + self.print_lineno(parameterdesc_start_lines.get(parameter_name= , 0)) + + self.lineprefix =3D " " + if parameter_name in parameterdescs and \ + parameterdescs[parameter_name] !=3D KernelDoc.undescribed: + + self.output_highlight(parameterdescs[parameter_name]) + print() + else: + print(f"{self.lineprefix}*undescribed*\n") + self.lineprefix =3D " " + + self.out_section(args) + self.lineprefix =3D oldprefix + + def out_enum(self, fname, name, args): + + oldprefix =3D self.lineprefix + name =3D args.get('enum', '') + parameterlist =3D args.get('parameterlist', []) + parameterdescs =3D args.get('parameterdescs', {}) + ln =3D args.get('ln', 0) + + print(f"\n\n.. c:enum:: {name}\n") + + self.print_lineno(ln) + self.lineprefix =3D " " + self.output_highlight(args.get('purpose', '')) + print() + + print(".. container:: kernelindent\n") + outer =3D self.lineprefix + " " + self.lineprefix =3D outer + " " + print(f"{outer}**Constants**\n") + + for parameter in parameterlist: + print(f"{outer}``{parameter}``") + + if parameterdescs.get(parameter, '') !=3D KernelDoc.undescribe= d: + self.output_highlight(parameterdescs[parameter]) + else: + print(f"{self.lineprefix}*undescribed*\n") + print() + + self.lineprefix =3D oldprefix + self.out_section(args) + + def out_typedef(self, fname, name, args): + + oldprefix =3D self.lineprefix + name =3D args.get('typedef', '') + ln =3D args.get('ln', 0) + + print(f"\n\n.. c:type:: {name}\n") + + self.print_lineno(ln) + self.lineprefix =3D " " + + self.output_highlight(args.get('purpose', '')) + + print() + + self.lineprefix =3D oldprefix + self.out_section(args) + + def out_struct(self, fname, name, args): + + name =3D args.get('struct', "") + purpose =3D args.get('purpose', "") + declaration =3D args.get('definition', "") + dtype =3D args.get('type', "struct") + ln =3D args.get('ln', 0) + + parameterlist =3D args.get('parameterlist', []) + parameterdescs =3D args.get('parameterdescs', {}) + parameterdesc_start_lines =3D args.get('parameterdesc_start_lines'= , {}) + + print(f"\n\n.. c:{dtype}:: {name}\n") + + self.print_lineno(ln) + + oldprefix =3D self.lineprefix + self.lineprefix +=3D " " + + self.output_highlight(purpose) + print() + + print(".. container:: kernelindent\n") + print(f"{self.lineprefix}**Definition**::\n") + + self.lineprefix =3D self.lineprefix + " " + + declaration =3D declaration.replace("\t", self.lineprefix) + + print(f"{self.lineprefix}{dtype} {name}" + ' {') + print(f"{declaration}{self.lineprefix}" + "};\n") + + self.lineprefix =3D " " + print(f"{self.lineprefix}**Members**\n") + for parameter in parameterlist: + if not parameter or parameter.startswith("#"): + continue + + parameter_name =3D parameter.split("[", maxsplit=3D1)[0] + + if parameterdescs.get(parameter_name) =3D=3D KernelDoc.undescr= ibed: + continue + + self.print_lineno(parameterdesc_start_lines.get(parameter_name= , 0)) + + print(f"{self.lineprefix}``{parameter}``") + + self.lineprefix =3D " " + self.output_highlight(parameterdescs[parameter_name]) + self.lineprefix =3D " " + + print() + + print() + + self.lineprefix =3D oldprefix + self.out_section(args) + + +class ManFormat(OutputFormat): + """Consts and functions used by man pages output""" + + highlights =3D ( + (type_constant, r"\1"), + (type_constant2, r"\1"), + (type_func, r"\\fB\1\\fP"), + (type_enum, r"\\fI\1\\fP"), + (type_struct, r"\\fI\1\\fP"), + (type_typedef, r"\\fI\1\\fP"), + (type_union, r"\\fI\1\\fP"), + (type_param, r"\\fI\1\\fP"), + (type_param_ref, r"\\fI\1\2\\fP"), + (type_member, r"\\fI\1\2\3\\fP"), + (type_fallback, r"\\fI\1\\fP") + ) + blankline =3D "" + + def __init__(self): + """ + Creates class variables. + + Not really mandatory, but it is a good coding style and makes + pylint happy. + """ + + super().__init__() + + dt =3D datetime.now() + if os.environ.get("KBUILD_BUILD_TIMESTAMP", None): + # use UTC TZ + to_zone =3D tz.gettz('UTC') + dt =3D dt.astimezone(to_zone) + + self.man_date =3D dt.strftime("%B %Y") + + def output_highlight(self, block): + + contents =3D self.highlight_block(block) + + if isinstance(contents, list): + contents =3D "\n".join(contents) + + for line in contents.strip("\n").split("\n"): + line =3D Re(r"^\s*").sub("", line) + + if line and line[0] =3D=3D ".": + print("\\&" + line) + else: + print(line) + + def out_doc(self, fname, name, args): + module =3D args.get('module') + sectionlist =3D args.get('sectionlist', []) + sections =3D args.get('sections', {}) + + print(f'.TH "{module}" 9 "{module}" "{self.man_date}" "API Manual"= LINUX') + + for section in sectionlist: + print(f'.SH "{section}"') + self.output_highlight(sections.get(section)) + + def out_function(self, fname, name, args): + """output function in man""" + + parameterlist =3D args.get('parameterlist', []) + parameterdescs =3D args.get('parameterdescs', {}) + sectionlist =3D args.get('sectionlist', []) + sections =3D args.get('sections', {}) + + print(f'.TH "{args['function']}" 9 "{args['function']}" "{self.man= _date}" "Kernel Hacker\'s Manual" LINUX') + + print(".SH NAME") + print(f"{args['function']} \\- {args['purpose']}") + + print(".SH SYNOPSIS") + if args.get('functiontype', ''): + print(f'.B "{args['functiontype']}" {args['function']}') + else: + print(f'.B "{args['function']}') + + count =3D 0 + parenth =3D "(" + post =3D "," + + for parameter in parameterlist: + if count =3D=3D len(parameterlist) - 1: + post =3D ");" + + dtype =3D args['parametertypes'].get(parameter, "") + if function_pointer.match(dtype): + # Pointer-to-function + print(f'".BI "{parenth}{function_pointer.group(1)}" " ") (= {function_pointer.group(2)}){post}"') + else: + dtype =3D Re(r'([^\*])$').sub(r'\1 ', dtype) + + print(f'.BI "{parenth}{dtype}" "{post}"') + count +=3D 1 + parenth =3D "" + + if parameterlist: + print(".SH ARGUMENTS") + + for parameter in parameterlist: + parameter_name =3D re.sub(r'\[.*', '', parameter) + + print(f'.IP "{parameter}" 12') + self.output_highlight(parameterdescs.get(parameter_name, "")) + + for section in sectionlist: + print(f'.SH "{section.upper()}"') + self.output_highlight(sections[section]) + + def out_enum(self, fname, name, args): + + name =3D args.get('enum', '') + parameterlist =3D args.get('parameterlist', []) + sectionlist =3D args.get('sectionlist', []) + sections =3D args.get('sections', {}) + + print(f'.TH "{args['module']}" 9 "enum {args['enum']}" "{self.man_= date}" "API Manual" LINUX') + + print(".SH NAME") + print(f"enum {args['enum']} \\- {args['purpose']}") + + print(".SH SYNOPSIS") + print(f"enum {args['enum']}" + " {") + + count =3D 0 + for parameter in parameterlist: + print(f'.br\n.BI " {parameter}"') + if count =3D=3D len(parameterlist) - 1: + print("\n};") + else: + print(", \n.br") + + count +=3D 1 + + print(".SH Constants") + + for parameter in parameterlist: + parameter_name =3D Re(r'\[.*').sub('', parameter) + print(f'.IP "{parameter}" 12') + self.output_highlight(args['parameterdescs'].get(parameter_nam= e, "")) + + for section in sectionlist: + print(f'.SH "{section}"') + self.output_highlight(sections[section]) + + def out_typedef(self, fname, name, args): + module =3D args.get('module') + typedef =3D args.get('typedef') + purpose =3D args.get('purpose') + sectionlist =3D args.get('sectionlist', []) + sections =3D args.get('sections', {}) + + print(f'.TH "{module}" 9 "{typedef}" "{self.man_date}" "API Manual= " LINUX') + + print(".SH NAME") + print(f"typedef {typedef} \\- {purpose}") + + for section in sectionlist: + print(f'.SH "{section}"') + self.output_highlight(sections.get(section)) + + def out_struct(self, fname, name, args): + module =3D args.get('module') + struct_type =3D args.get('type') + struct_name =3D args.get('struct') + purpose =3D args.get('purpose') + definition =3D args.get('definition') + sectionlist =3D args.get('sectionlist', []) + parameterlist =3D args.get('parameterlist', []) + sections =3D args.get('sections', {}) + parameterdescs =3D args.get('parameterdescs', {}) + + print(f'.TH "{module}" 9 "{struct_type} {struct_name}" "{self.man_= date}" "API Manual" LINUX') + + print(".SH NAME") + print(f"{struct_type} {struct_name} \\- {purpose}") + + # Replace tabs with two spaces and handle newlines + declaration =3D definition.replace("\t", " ") + declaration =3D Re(r"\n").sub('"\n.br\n.BI "', declaration) + + print(".SH SYNOPSIS") + print(f"{struct_type} {struct_name} " + "{" + "\n.br") + print(f'.BI "{declaration}\n' + "};\n.br\n") + + print(".SH Members") + for parameter in parameterlist: + if parameter.startswith("#"): + continue + + parameter_name =3D re.sub(r"\[.*", "", parameter) + + if parameterdescs.get(parameter_name) =3D=3D KernelDoc.undescr= ibed: + continue + + print(f'.IP "{parameter}" 12') + self.output_highlight(parameterdescs.get(parameter_name)) + + for section in sectionlist: + print(f'.SH "{section}"') + self.output_highlight(sections.get(section)) --=20 2.48.1 From nobody Sat Feb 7 14:01:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F2FD31DE3A3; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953973; cv=none; b=poBiO5ewwe8vi5fLNdpcc5XphQMM9IdOo5NzNdNQF+do7uEA/hjbqIg8grs/fiweWWMT5AXs+Hs21dEHdrEZBhUys+X9VacZ3OyLr4chMAN1Es3VKEHIj8MQzOGQqcKQvSKUkJVfeX3L9mNcsP4dHoJkiB8d/pZy0WPLbLXZ5UU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953973; c=relaxed/simple; bh=+233tNh0yGNXfYZxQx+uxFiZ7ugw73yF3W1jWprPI1A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BhprkR5d2wi1gxdLMmjv3u2yY7QYDPYtjprybGtJl7Zq976BAwx9p8Ftq4RqZeguBu0y+UsjK8SQ9ea4GfCYK4K4/SyoL6xfbyvHvYiqMMKhx8221TfZL4r0LuPsgFBJMJOgPWlF1UOAnyDw1LBR8zf2mBCRfemXCW1riC7e27U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=oioGjcgP; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="oioGjcgP" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3ED16C113CF; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739953972; bh=+233tNh0yGNXfYZxQx+uxFiZ7ugw73yF3W1jWprPI1A=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=oioGjcgP/rgaQ16e0sJrzpota3HtAKniRKckPqbb6DHNKRZ9tLs4dBRxQm0hCDpsx MPB70sZXo4FujDpA0OWNnC4wicBRDW3EwKsBDRdr0FI5PLKXmRUyg30qvDS8CKD3Mn iymV0Lzi92El5a4h+mWlqiwahsN3+Z0bMglg0Tm2xzH1xxlhoCUkcW/iQRg0STHG96 NpKMMvNP9q0Kfyz8oRkKtVYJ4ONk2jscThcj2JOytHxG3ruzgJq+eAeZZLIxU+djcP IVqp5lPVJcRUuJ/w/cu2WH2ceepHwVR7mBpnaWGDMowdaff33w4gIpAAKUcZDD2Sol KYcdzC5upsgfw== Received: from mchehab by mail.kernel.org with local (Exim 4.98) (envelope-from ) id 1tkfVi-0000000Gv5N-1e0G; Wed, 19 Feb 2025 09:32:50 +0100 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , "Mauro Carvalho Chehab" , linux-kernel@vger.kernel.org Subject: [PATCH 17/27] scripts/kernel-doc.py: convert message output to an interactor Date: Wed, 19 Feb 2025 09:32:33 +0100 Message-ID: X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" Instead of directly printing output messages, change kdoc classes to return an interactor with the output message, letting the actual display to happen at the command-line command. Signed-off-by: Mauro Carvalho Chehab --- scripts/kernel-doc.py | 9 +- scripts/lib/kdoc/kdoc_files.py | 15 ++- scripts/lib/kdoc/kdoc_output.py | 171 ++++++++++++++++---------------- 3 files changed, 104 insertions(+), 91 deletions(-) diff --git a/scripts/kernel-doc.py b/scripts/kernel-doc.py index 0596c711d448..6c3179a2da65 100755 --- a/scripts/kernel-doc.py +++ b/scripts/kernel-doc.py @@ -208,9 +208,12 @@ def main(): =20 kfiles.parse() =20 - kfiles.msg(enable_lineno=3Dargs.enable_lineno, export=3Dargs.export, - internal=3Dargs.internal, symbol=3Dargs.symbol, - nosymbol=3Dargs.nosymbol) + for t in kfiles.msg(enable_lineno=3Dargs.enable_lineno, export=3Dargs.= export, + internal=3Dargs.internal, symbol=3Dargs.symbol, + nosymbol=3Dargs.nosymbol): + msg =3D t[1] + if msg: + print(msg) =20 =20 # Call main method diff --git a/scripts/lib/kdoc/kdoc_files.py b/scripts/lib/kdoc/kdoc_files.py index 76dd53611c08..434fc66a9dad 100755 --- a/scripts/lib/kdoc/kdoc_files.py +++ b/scripts/lib/kdoc/kdoc_files.py @@ -228,9 +228,10 @@ class KernelFiles(): =20 def out_msg(self, fname, name, arg): """ - Output messages from a file name using the output style filtering. + Return output messages from a file name using the output style + filtering. =20 - If output type was not handled by the syler, return False. + If output type was not handled by the syler, return None. """ =20 # NOTE: we can add rules here to filter out unwanted parts, @@ -241,7 +242,8 @@ class KernelFiles(): def msg(self, enable_lineno=3DFalse, export=3DFalse, internal=3DFalse, symbol=3DNone, nosymbol=3DNone): """ - Interacts over the kernel-doc results and output messages. + Interacts over the kernel-doc results and output messages, + returning kernel-doc markups on each interaction """ =20 function_table =3D self.config.function_table @@ -260,10 +262,15 @@ class KernelFiles(): function_table, enable_lineno) =20 for fname, arg_tuple in self.results: + msg =3D "" for name, arg in arg_tuple: - if self.out_msg(fname, name, arg): + msg +=3D self.out_msg(fname, name, arg) + + if msg is None: ln =3D arg.get("ln", 0) dtype =3D arg.get('type', "") =20 self.config.log.warning("%s:%d Can't handle %s", fname, ln, dtype) + if msg: + yield fname, msg diff --git a/scripts/lib/kdoc/kdoc_output.py b/scripts/lib/kdoc/kdoc_output= .py index d080440caa1c..91f6e356d03d 100755 --- a/scripts/lib/kdoc/kdoc_output.py +++ b/scripts/lib/kdoc/kdoc_output.py @@ -70,6 +70,8 @@ class OutputFormat: self.function_table =3D set() self.config =3D None =20 + self.data =3D "" + def set_config(self, config): self.config =3D config =20 @@ -156,37 +158,38 @@ class OutputFormat: return True =20 def msg(self, fname, name, args): + self.data =3D "" =20 dtype =3D args.get('type', "") =20 if dtype =3D=3D "doc": self.out_doc(fname, name, args) - return False + return self.data =20 if not self.check_declaration(dtype, name): - return False + return self.data =20 if dtype =3D=3D "function": self.out_function(fname, name, args) - return False + return self.data =20 if dtype =3D=3D "enum": self.out_enum(fname, name, args) - return False + return self.data =20 if dtype =3D=3D "typedef": self.out_typedef(fname, name, args) - return False + return self.data =20 if dtype in ["struct", "union"]: self.out_struct(fname, name, args) - return False + return self.data =20 # Warn if some type requires an output logic self.config.log.warning("doesn't now how to output '%s' block", dtype) =20 - return True + return None =20 # Virtual methods to be overridden by inherited classes def out_doc(self, fname, name, args): @@ -247,7 +250,7 @@ class RestFormat(OutputFormat): """Outputs a line number""" =20 if self.enable_lineno and ln: - print(f".. LINENO {ln}") + self.data +=3D f".. LINENO {ln}\n" =20 def output_highlight(self, args): input_text =3D args @@ -294,7 +297,7 @@ class RestFormat(OutputFormat): =20 # Print the output with the line prefix for line in output.strip("\n").split("\n"): - print(self.lineprefix + line) + self.data +=3D self.lineprefix + line + "\n" =20 def out_section(self, args, out_reference=3DFalse): """ @@ -316,15 +319,15 @@ class RestFormat(OutputFormat): =20 if not self.out_mode =3D=3D self.OUTPUT_INCLUDE: if out_reference: - print(f".. _{section}:\n") + self.data +=3D f".. _{section}:\n\n" =20 if not self.symbol: - print(f'{self.lineprefix}**{section}**\n') + self.data +=3D f'{self.lineprefix}**{section}**\n\n' =20 self.print_lineno(section_start_lines.get(section, 0)) self.output_highlight(sections[section]) - print() - print() + self.data +=3D "\n" + self.data +=3D "\n" =20 def out_doc(self, fname, name, args): if not self.check_doc(name): @@ -367,42 +370,42 @@ class RestFormat(OutputFormat): signature +=3D ")" =20 if args.get('typedef') or not args.get('functiontype'): - print(f".. c:macro:: {args['function']}\n") + self.data +=3D f".. c:macro:: {args['function']}\n\n" =20 if args.get('typedef'): self.print_lineno(ln) - print(" **Typedef**: ", end=3D"") + self.data +=3D " **Typedef**: " self.lineprefix =3D "" self.output_highlight(args.get('purpose', "")) - print("\n\n**Syntax**\n") - print(f" ``{signature}``\n") + self.data +=3D "\n\n**Syntax**\n\n" + self.data +=3D f" ``{signature}``\n\n" else: - print(f"``{signature}``\n") + self.data +=3D f"``{signature}``\n\n" else: - print(f".. c:function:: {signature}\n") + self.data +=3D f".. c:function:: {signature}\n\n" =20 if not args.get('typedef'): self.print_lineno(ln) self.lineprefix =3D " " self.output_highlight(args.get('purpose', "")) - print() + self.data +=3D "\n" =20 # Put descriptive text into a container (HTML
) to help set # function prototypes apart self.lineprefix =3D " " =20 if parameterlist: - print(".. container:: kernelindent\n") - print(f"{self.lineprefix}**Parameters**\n") + self.data +=3D ".. container:: kernelindent\n\n" + self.data +=3D f"{self.lineprefix}**Parameters**\n\n" =20 for parameter in parameterlist: parameter_name =3D Re(r'\[.*').sub('', parameter) dtype =3D args['parametertypes'].get(parameter, "") =20 if dtype: - print(f"{self.lineprefix}``{dtype}``") + self.data +=3D f"{self.lineprefix}``{dtype}``\n" else: - print(f"{self.lineprefix}``{parameter}``") + self.data +=3D f"{self.lineprefix}``{parameter}``\n" =20 self.print_lineno(parameterdesc_start_lines.get(parameter_name= , 0)) =20 @@ -411,9 +414,9 @@ class RestFormat(OutputFormat): parameterdescs[parameter_name] !=3D KernelDoc.undescribed: =20 self.output_highlight(parameterdescs[parameter_name]) - print() + self.data +=3D "\n" else: - print(f"{self.lineprefix}*undescribed*\n") + self.data +=3D f"{self.lineprefix}*undescribed*\n\n" self.lineprefix =3D " " =20 self.out_section(args) @@ -427,26 +430,26 @@ class RestFormat(OutputFormat): parameterdescs =3D args.get('parameterdescs', {}) ln =3D args.get('ln', 0) =20 - print(f"\n\n.. c:enum:: {name}\n") + self.data +=3D f"\n\n.. c:enum:: {name}\n\n" =20 self.print_lineno(ln) self.lineprefix =3D " " self.output_highlight(args.get('purpose', '')) - print() + self.data +=3D "\n" =20 - print(".. container:: kernelindent\n") + self.data +=3D ".. container:: kernelindent\n\n" outer =3D self.lineprefix + " " self.lineprefix =3D outer + " " - print(f"{outer}**Constants**\n") + self.data +=3D f"{outer}**Constants**\n\n" =20 for parameter in parameterlist: - print(f"{outer}``{parameter}``") + self.data +=3D f"{outer}``{parameter}``\n" =20 if parameterdescs.get(parameter, '') !=3D KernelDoc.undescribe= d: self.output_highlight(parameterdescs[parameter]) else: - print(f"{self.lineprefix}*undescribed*\n") - print() + self.data +=3D f"{self.lineprefix}*undescribed*\n\n" + self.data +=3D "\n" =20 self.lineprefix =3D oldprefix self.out_section(args) @@ -457,14 +460,14 @@ class RestFormat(OutputFormat): name =3D args.get('typedef', '') ln =3D args.get('ln', 0) =20 - print(f"\n\n.. c:type:: {name}\n") + self.data +=3D f"\n\n.. c:type:: {name}\n\n" =20 self.print_lineno(ln) self.lineprefix =3D " " =20 self.output_highlight(args.get('purpose', '')) =20 - print() + self.data +=3D "\n" =20 self.lineprefix =3D oldprefix self.out_section(args) @@ -481,7 +484,7 @@ class RestFormat(OutputFormat): parameterdescs =3D args.get('parameterdescs', {}) parameterdesc_start_lines =3D args.get('parameterdesc_start_lines'= , {}) =20 - print(f"\n\n.. c:{dtype}:: {name}\n") + self.data +=3D f"\n\n.. c:{dtype}:: {name}\n\n" =20 self.print_lineno(ln) =20 @@ -489,20 +492,20 @@ class RestFormat(OutputFormat): self.lineprefix +=3D " " =20 self.output_highlight(purpose) - print() + self.data +=3D "\n" =20 - print(".. container:: kernelindent\n") - print(f"{self.lineprefix}**Definition**::\n") + self.data +=3D ".. container:: kernelindent\n\n" + self.data +=3D f"{self.lineprefix}**Definition**::\n\n" =20 self.lineprefix =3D self.lineprefix + " " =20 declaration =3D declaration.replace("\t", self.lineprefix) =20 - print(f"{self.lineprefix}{dtype} {name}" + ' {') - print(f"{declaration}{self.lineprefix}" + "};\n") + self.data +=3D f"{self.lineprefix}{dtype} {name}" + ' {' + "\n" + self.data +=3D f"{declaration}{self.lineprefix}" + "};\n\n" =20 self.lineprefix =3D " " - print(f"{self.lineprefix}**Members**\n") + self.data +=3D f"{self.lineprefix}**Members**\n\n" for parameter in parameterlist: if not parameter or parameter.startswith("#"): continue @@ -514,15 +517,15 @@ class RestFormat(OutputFormat): =20 self.print_lineno(parameterdesc_start_lines.get(parameter_name= , 0)) =20 - print(f"{self.lineprefix}``{parameter}``") + self.data +=3D f"{self.lineprefix}``{parameter}``\n" =20 self.lineprefix =3D " " self.output_highlight(parameterdescs[parameter_name]) self.lineprefix =3D " " =20 - print() + self.data +=3D "\n" =20 - print() + self.data +=3D "\n" =20 self.lineprefix =3D oldprefix self.out_section(args) @@ -575,19 +578,19 @@ class ManFormat(OutputFormat): line =3D Re(r"^\s*").sub("", line) =20 if line and line[0] =3D=3D ".": - print("\\&" + line) + self.data +=3D "\\&" + line + "\n" else: - print(line) + self.data +=3D line + "\n" =20 def out_doc(self, fname, name, args): module =3D args.get('module') sectionlist =3D args.get('sectionlist', []) sections =3D args.get('sections', {}) =20 - print(f'.TH "{module}" 9 "{module}" "{self.man_date}" "API Manual"= LINUX') + self.data +=3D f'.TH "{module}" 9 "{module}" "{self.man_date}" "AP= I Manual" LINUX' + "\n" =20 for section in sectionlist: - print(f'.SH "{section}"') + self.data +=3D f'.SH "{section}"' + "\n" self.output_highlight(sections.get(section)) =20 def out_function(self, fname, name, args): @@ -598,16 +601,16 @@ class ManFormat(OutputFormat): sectionlist =3D args.get('sectionlist', []) sections =3D args.get('sections', {}) =20 - print(f'.TH "{args['function']}" 9 "{args['function']}" "{self.man= _date}" "Kernel Hacker\'s Manual" LINUX') + self.data +=3D f'.TH "{args['function']}" 9 "{args['function']}" "= {self.man_date}" "Kernel Hacker\'s Manual" LINUX' + "\n" =20 - print(".SH NAME") - print(f"{args['function']} \\- {args['purpose']}") + self.data +=3D ".SH NAME\n" + self.data +=3D f"{args['function']} \\- {args['purpose']}\n" =20 - print(".SH SYNOPSIS") + self.data +=3D ".SH SYNOPSIS\n" if args.get('functiontype', ''): - print(f'.B "{args['functiontype']}" {args['function']}') + self.data +=3D f'.B "{args['functiontype']}" {args['function']= }' + "\n" else: - print(f'.B "{args['function']}') + self.data +=3D f'.B "{args['function']}' + "\n" =20 count =3D 0 parenth =3D "(" @@ -620,25 +623,25 @@ class ManFormat(OutputFormat): dtype =3D args['parametertypes'].get(parameter, "") if function_pointer.match(dtype): # Pointer-to-function - print(f'".BI "{parenth}{function_pointer.group(1)}" " ") (= {function_pointer.group(2)}){post}"') + self.data +=3D f'".BI "{parenth}{function_pointer.group(1)= }" " ") ({function_pointer.group(2)}){post}"' + "\n" else: dtype =3D Re(r'([^\*])$').sub(r'\1 ', dtype) =20 - print(f'.BI "{parenth}{dtype}" "{post}"') + self.data +=3D f'.BI "{parenth}{dtype}" "{post}"' + "\n" count +=3D 1 parenth =3D "" =20 if parameterlist: - print(".SH ARGUMENTS") + self.data +=3D ".SH ARGUMENTS\n" =20 for parameter in parameterlist: parameter_name =3D re.sub(r'\[.*', '', parameter) =20 - print(f'.IP "{parameter}" 12') + self.data +=3D f'.IP "{parameter}" 12' + "\n" self.output_highlight(parameterdescs.get(parameter_name, "")) =20 for section in sectionlist: - print(f'.SH "{section.upper()}"') + self.data +=3D f'.SH "{section.upper()}"' + "\n" self.output_highlight(sections[section]) =20 def out_enum(self, fname, name, args): @@ -648,33 +651,33 @@ class ManFormat(OutputFormat): sectionlist =3D args.get('sectionlist', []) sections =3D args.get('sections', {}) =20 - print(f'.TH "{args['module']}" 9 "enum {args['enum']}" "{self.man_= date}" "API Manual" LINUX') + self.data +=3D f'.TH "{args['module']}" 9 "enum {args['enum']}" "{= self.man_date}" "API Manual" LINUX' + "\n" =20 - print(".SH NAME") - print(f"enum {args['enum']} \\- {args['purpose']}") + self.data +=3D ".SH NAME\n" + self.data +=3D f"enum {args['enum']} \\- {args['purpose']}\n" =20 - print(".SH SYNOPSIS") - print(f"enum {args['enum']}" + " {") + self.data +=3D ".SH SYNOPSIS\n" + self.data +=3D f"enum {args['enum']}" + " {\n" =20 count =3D 0 for parameter in parameterlist: - print(f'.br\n.BI " {parameter}"') + self.data +=3D f'.br\n.BI " {parameter}"' + "\n" if count =3D=3D len(parameterlist) - 1: - print("\n};") + self.data +=3D "\n};\n" else: - print(", \n.br") + self.data +=3D ", \n.br\n" =20 count +=3D 1 =20 - print(".SH Constants") + self.data +=3D ".SH Constants\n" =20 for parameter in parameterlist: parameter_name =3D Re(r'\[.*').sub('', parameter) - print(f'.IP "{parameter}" 12') + self.data +=3D f'.IP "{parameter}" 12' + "\n" self.output_highlight(args['parameterdescs'].get(parameter_nam= e, "")) =20 for section in sectionlist: - print(f'.SH "{section}"') + self.data +=3D f'.SH "{section}"' + "\n" self.output_highlight(sections[section]) =20 def out_typedef(self, fname, name, args): @@ -684,13 +687,13 @@ class ManFormat(OutputFormat): sectionlist =3D args.get('sectionlist', []) sections =3D args.get('sections', {}) =20 - print(f'.TH "{module}" 9 "{typedef}" "{self.man_date}" "API Manual= " LINUX') + self.data +=3D f'.TH "{module}" 9 "{typedef}" "{self.man_date}" "A= PI Manual" LINUX' + "\n" =20 - print(".SH NAME") - print(f"typedef {typedef} \\- {purpose}") + self.data +=3D ".SH NAME\n" + self.data +=3D f"typedef {typedef} \\- {purpose}\n" =20 for section in sectionlist: - print(f'.SH "{section}"') + self.data +=3D f'.SH "{section}"' + "\n" self.output_highlight(sections.get(section)) =20 def out_struct(self, fname, name, args): @@ -704,20 +707,20 @@ class ManFormat(OutputFormat): sections =3D args.get('sections', {}) parameterdescs =3D args.get('parameterdescs', {}) =20 - print(f'.TH "{module}" 9 "{struct_type} {struct_name}" "{self.man_= date}" "API Manual" LINUX') + self.data +=3D f'.TH "{module}" 9 "{struct_type} {struct_name}" "{= self.man_date}" "API Manual" LINUX' + "\n" =20 - print(".SH NAME") - print(f"{struct_type} {struct_name} \\- {purpose}") + self.data +=3D ".SH NAME\n" + self.data +=3D f"{struct_type} {struct_name} \\- {purpose}\n" =20 # Replace tabs with two spaces and handle newlines declaration =3D definition.replace("\t", " ") declaration =3D Re(r"\n").sub('"\n.br\n.BI "', declaration) =20 - print(".SH SYNOPSIS") - print(f"{struct_type} {struct_name} " + "{" + "\n.br") - print(f'.BI "{declaration}\n' + "};\n.br\n") + self.data +=3D ".SH SYNOPSIS\n" + self.data +=3D f"{struct_type} {struct_name} " + "{" + "\n.br\n" + self.data +=3D f'.BI "{declaration}\n' + "};\n.br\n\n" =20 - print(".SH Members") + self.data +=3D ".SH Members\n" for parameter in parameterlist: if parameter.startswith("#"): continue @@ -727,9 +730,9 @@ class ManFormat(OutputFormat): if parameterdescs.get(parameter_name) =3D=3D KernelDoc.undescr= ibed: continue =20 - print(f'.IP "{parameter}" 12') + self.data +=3D f'.IP "{parameter}" 12' + "\n" self.output_highlight(parameterdescs.get(parameter_name)) =20 for section in sectionlist: - print(f'.SH "{section}"') + self.data +=3D f'.SH "{section}"' + "\n" self.output_highlight(sections.get(section)) --=20 2.48.1 From nobody Sat Feb 7 14:01:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8322F1D6DAD; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; cv=none; b=uIv3pEnNlntQMezXsQj6PnWnS5EHgU09USJgamXEuI6eohVGEhFpbfgdRysTfoyN2SXbv52/jAZ2d0ZelQF1nq7PwMlm0QakN/wc8LKoX9WPRMBtzhyfiEW+434uOrMKQ+rqxcWItoXlmD/fJqudqoaHFzBv5ksKicldh/R6Vhk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; c=relaxed/simple; bh=Q2bW4rHBvXUepqtRYw3jrHwqyqtoRshTMOAzsCvXTFo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qMkPncGjLaXSsB7SuFntci9pGqTZsGedKZWr3/ScbO5q7MNzfjQlFCvyWad3z/7NzEitP38cR9sK1JAGg4y7oKXF/7CWGgsTbCzCRV7TSp6fFez1UF5lMgROOAnCxWir7l4oPRnmu6EOsXp8ZvdkGEPC8QsHovyrEpq5BIrPVek= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=kIpWHncw; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="kIpWHncw" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4517CC116B1; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739953972; bh=Q2bW4rHBvXUepqtRYw3jrHwqyqtoRshTMOAzsCvXTFo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kIpWHncwKDu947B17a0ctNBsPEJInGdEIUGgSitnv8ydhWqi50jYS2Tu4qiUncVsO M2XGvh2eVltRu2wOCPMowVIc/A9QndTNnauM39CWbgtUFsoT+/9Plpl4yMCs5Ygk0O n765C60hy0wFihmyqlFbxZj1M2qVUfVQBiQoPIU0MYnsEwHNFzyDb8zF3fTtE7S057 KW7QNp95NClzKWt2Vry81DEU0W8XxDLs1NuGYvATMB1haKtgB0cyRY6NA+Xw6sOg9g nnQLvzzxS2DQvEYJN9TwyhsITuA/MuPwVj6lbDAVFYbc9p+4a+CxKsAt6GyU9xYlNt sIsfP0h/w0njg== Received: from mchehab by mail.kernel.org with local (Exim 4.98) (envelope-from ) id 1tkfVi-0000000Gv5R-1ktQ; Wed, 19 Feb 2025 09:32:50 +0100 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , "Mauro Carvalho Chehab" , linux-kernel@vger.kernel.org Subject: [PATCH 18/27] scripts/kernel-doc.py: move file lists to the parser function Date: Wed, 19 Feb 2025 09:32:34 +0100 Message-ID: X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" Instead of setting file lists at __init__ time, move it to the actual parsing function. This allows adding more files to be parsed in real time, by calling parse function multiple times. With the new way, the export_files logic was rewritten to avoid parsing twice EXPORT_SYMBOL for partial matches. Please notice that, with this logic, it can still read the same file twice when export_file is used. Signed-off-by: Mauro Carvalho Chehab --- scripts/kernel-doc.py | 7 +++---- scripts/lib/kdoc/kdoc_files.py | 37 ++++++++++++++++------------------ 2 files changed, 20 insertions(+), 24 deletions(-) diff --git a/scripts/kernel-doc.py b/scripts/kernel-doc.py index 6c3179a2da65..a687a7abb3b9 100755 --- a/scripts/kernel-doc.py +++ b/scripts/kernel-doc.py @@ -199,14 +199,13 @@ def main(): else: out_style =3D RestFormat() =20 - kfiles =3D KernelFiles(files=3Dargs.files, verbose=3Dargs.verbose, + kfiles =3D KernelFiles(verbose=3Dargs.verbose, out_style=3Dout_style, werror=3Dargs.werror, wreturn=3Dargs.wreturn, wshort_desc=3Dargs.wshort= _desc, wcontents_before_sections=3Dargs.wcontents_before= _sections, - modulename=3Dargs.modulename, - export_file=3Dargs.export_file) + modulename=3Dargs.modulename) =20 - kfiles.parse() + kfiles.parse(args.files, export_file=3Dargs.export_file) =20 for t in kfiles.msg(enable_lineno=3Dargs.enable_lineno, export=3Dargs.= export, internal=3Dargs.internal, symbol=3Dargs.symbol, diff --git a/scripts/lib/kdoc/kdoc_files.py b/scripts/lib/kdoc/kdoc_files.py index 434fc66a9dad..4a6e75dbdbdd 100755 --- a/scripts/lib/kdoc/kdoc_files.py +++ b/scripts/lib/kdoc/kdoc_files.py @@ -123,7 +123,7 @@ class KernelFiles(): self.config.log.error("Cannot find file %s", fname) self.config.errors +=3D 1 =20 - def __init__(self, files=3DNone, verbose=3DFalse, out_style=3DNone, + def __init__(self, verbose=3DFalse, out_style=3DNone, werror=3DFalse, wreturn=3DFalse, wshort_desc=3DFalse, wcontents_before_sections=3DFalse, logger=3DNone, modulename=3DNone, export_file=3DNone): @@ -180,51 +180,48 @@ class KernelFiles(): self.config.src_tree =3D os.environ.get("SRCTREE", None) =20 self.out_style =3D out_style - self.export_file =3D export_file =20 # Initialize internal variables =20 self.config.errors =3D 0 self.results =3D [] =20 - self.file_list =3D files self.files =3D set() + self.export_files =3D set() =20 - def parse(self): + def parse(self, file_list, export_file=3DNone): """ Parse all files """ =20 glob =3D GlobSourceFiles(srctree=3Dself.config.src_tree) =20 - # Let's use a set here to avoid duplicating files + # Prevent parsing the same file twice to speedup parsing and + # avoid reporting errors multiple times =20 - for fname in glob.parse_files(self.file_list, self.file_not_found_= cb): + for fname in glob.parse_files(file_list, self.file_not_found_cb): if fname in self.files: continue =20 - self.files.add(fname) - res =3D self.parse_file(fname) + self.results.append((res.fname, res.entries)) - - if not self.files: - sys.exit(1) + self.files.add(fname) =20 # If a list of export files was provided, parse EXPORT_SYMBOL* - # from the ones not already parsed + # from files that weren't fully parsed =20 - if self.export_file: - files =3D self.files + if not export_file: + return =20 - glob =3D GlobSourceFiles(srctree=3Dself.config.src_tree) + self.export_files |=3D self.files =20 - for fname in glob.parse_files(self.export_file, - self.file_not_found_cb): - if fname not in files: - files.add(fname) + glob =3D GlobSourceFiles(srctree=3Dself.config.src_tree) =20 - self.process_export_file(fname) + for fname in glob.parse_files(export_file, self.file_not_found_cb): + if fname not in self.export_files: + self.process_export_file(fname) + self.export_files.add(fname) =20 def out_msg(self, fname, name, arg): """ --=20 2.48.1 From nobody Sat Feb 7 14:01:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 87E2D1CCEF0; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; cv=none; b=FCcbfvMLXsJe8zJMUDhrvsrYLrOmtSs9mOcpCgXubmD6geOEUtO2St/B2crMjqYc6QYCqJRRTi+Pvqqdx1BjcT5tNNkNI1EwAHvxeoGG1fRkOos3g20nfr6hqpAdoRbUkOV/7snulOQq/oj6t+9oTIp/2zzqcumGP6fK4SPb+Pc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; c=relaxed/simple; bh=IpThprTfJ/pbtHwAeyiILQ0MdhJxWk9SeTU5TEeUoNg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bJ0alw7RUPibgGQcmRQHC/chZPeKL8cNQ4e3UVCwhM132A/ulKrsFYUZQg8kL5g+rdhwfktemkJ7m3gxSfYVKgW1IVl+hbyqVSLcbN+hrdCn7RIOdzWUe9v4WGD0KRJ5tEtds4L3A1oDBVzxBXgIxYQFWJwETmvx6m8ciZg/+wE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=f9faGlxq; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="f9faGlxq" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 46D95C113D0; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739953972; bh=IpThprTfJ/pbtHwAeyiILQ0MdhJxWk9SeTU5TEeUoNg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=f9faGlxqxHrqr7dzrGSvo+gxHsWQ64ynVCsdJ+lymjr0H1W5PFhLCvyy8ixgCJXkW /0FC2tqOdjZye73uIinvtCHMkEZeZAHJciwjY8f9gdOdfspQw++fGLciDgWFQ2U4pv CJxDB1RbFbsQ6KQZogtDZaxUUzSAnHSVTQYtN6fxZsaHuuV0ZWyOBD8gOGFsj9684a C02fjz/M+Csk5INFjaOEeen2CLDoO8d/y174JX27Glnu+5Sh84fGhmONSFOJQLdls5 adY7qAiMkOy0zlv+v1SfVC8H0LtGlp+U8KGf+2gB9JpZyFy3qoJCLsdFQ0qIhMuOg+ qfAoxEAw++6ww== Received: from mchehab by mail.kernel.org with local (Exim 4.98) (envelope-from ) id 1tkfVi-0000000Gv5V-1rhZ; Wed, 19 Feb 2025 09:32:50 +0100 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , "Mauro Carvalho Chehab" , linux-kernel@vger.kernel.org Subject: [PATCH 19/27] scripts/kernel-doc.py: implement support for -no-doc-sections Date: Wed, 19 Feb 2025 09:32:35 +0100 Message-ID: <21d60f48197a03781154755418a38e870a24de5d.1739952783.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" The venerable kernel-doc Perl script has a number of options that aren't properly documented. Among them, there is -no-doc-sections, which is used by the Sphinx extension. Implement support for it. Signed-off-by: Mauro Carvalho Chehab --- scripts/kernel-doc.py | 8 ++++++-- scripts/lib/kdoc/kdoc_files.py | 5 +++-- scripts/lib/kdoc/kdoc_output.py | 7 ++++++- 3 files changed, 15 insertions(+), 5 deletions(-) diff --git a/scripts/kernel-doc.py b/scripts/kernel-doc.py index a687a7abb3b9..d700451e9541 100755 --- a/scripts/kernel-doc.py +++ b/scripts/kernel-doc.py @@ -164,10 +164,13 @@ def main(): sel_mut.add_argument("-s", "-function", "--symbol", action=3D'append', help=3DFUNCTION_DESC) =20 - # This one is valid for all 3 types of filter + # Those are valid for all 3 types of filter parser.add_argument("-n", "-nosymbol", "--nosymbol", action=3D'append', help=3DNOSYMBOL_DESC) =20 + parser.add_argument("-D", "-no-doc-sections", "--no-doc-sections", + action=3D'store_true', help=3D"Don't outputt DOC s= ections") + parser.add_argument("files", metavar=3D"FILE", nargs=3D"+", help=3DFILES_DESC) =20 @@ -209,7 +212,8 @@ def main(): =20 for t in kfiles.msg(enable_lineno=3Dargs.enable_lineno, export=3Dargs.= export, internal=3Dargs.internal, symbol=3Dargs.symbol, - nosymbol=3Dargs.nosymbol): + nosymbol=3Dargs.nosymbol, + no_doc_sections=3Dargs.no_doc_sections): msg =3D t[1] if msg: print(msg) diff --git a/scripts/lib/kdoc/kdoc_files.py b/scripts/lib/kdoc/kdoc_files.py index 4a6e75dbdbdd..c215ae3047b8 100755 --- a/scripts/lib/kdoc/kdoc_files.py +++ b/scripts/lib/kdoc/kdoc_files.py @@ -237,7 +237,7 @@ class KernelFiles(): return self.out_style.msg(fname, name, arg) =20 def msg(self, enable_lineno=3DFalse, export=3DFalse, internal=3DFalse, - symbol=3DNone, nosymbol=3DNone): + symbol=3DNone, nosymbol=3DNone, no_doc_sections=3DFalse): """ Interacts over the kernel-doc results and output messages, returning kernel-doc markups on each interaction @@ -256,7 +256,8 @@ class KernelFiles(): self.out_style.set_config(self.config) =20 self.out_style.set_filter(export, internal, symbol, nosymbol, - function_table, enable_lineno) + function_table, enable_lineno, + no_doc_sections) =20 for fname, arg_tuple in self.results: msg =3D "" diff --git a/scripts/lib/kdoc/kdoc_output.py b/scripts/lib/kdoc/kdoc_output= .py index 91f6e356d03d..8729dc58e13c 100755 --- a/scripts/lib/kdoc/kdoc_output.py +++ b/scripts/lib/kdoc/kdoc_output.py @@ -69,6 +69,7 @@ class OutputFormat: self.symbol =3D None self.function_table =3D set() self.config =3D None + self.no_doc_sections =3D False =20 self.data =3D "" =20 @@ -76,7 +77,7 @@ class OutputFormat: self.config =3D config =20 def set_filter(self, export, internal, symbol, nosymbol, function_tabl= e, - enable_lineno): + enable_lineno, no_doc_sections): """ Initialize filter variables according with the requested mode. =20 @@ -86,6 +87,7 @@ class OutputFormat: """ =20 self.enable_lineno =3D enable_lineno + self.no_doc_sections =3D no_doc_sections =20 if symbol: self.out_mode =3D self.OUTPUT_INCLUDE @@ -116,6 +118,9 @@ class OutputFormat: def check_doc(self, name): """Check if DOC should be output""" =20 + if self.no_doc_sections: + return False + if self.out_mode =3D=3D self.OUTPUT_ALL: return True =20 --=20 2.48.1 From nobody Sat Feb 7 14:01:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ACB9C1DBB3A; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; cv=none; b=eRBtuhJFFxiravDheqt4fOQWAknJi8gPP0YmnaQqjwn6c9n02bagVhFKEj06gcvMFr606lNWjwPQjK4fO5s3TvU1PEyt/bSEtj3ZimywJ5AF6DOnQBLweUoxDq/Dn/FcmeNfa0MmIfvxz8JJoDIlXi8//vvxWsDu9Oq1K802+VU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; c=relaxed/simple; bh=06ZoIn6KWo8lW7Vli577BGyd08sXznBzolIIOLad8Zg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=SFgLu3zmqMowMKsyhKYqNvKJ9msQvKoz94jvQgv+969Wd80Qm+OM7cClmkllAI6vDEJPEIwE4Nh6z/NAVC7TUZMmUJyYkNMDaZ1W7kzw/Ead9TgWHLZ/ptBUrqo+F3KktgJk7Bw7kXwmMhF5R/sMOYsr6PuK7DqgZhrxYJ9p/xg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ZfzoMqXz; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ZfzoMqXz" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 53572C116C6; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739953972; bh=06ZoIn6KWo8lW7Vli577BGyd08sXznBzolIIOLad8Zg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZfzoMqXzWl5RMPzeRGbajkpt71oeiDO5OEu1VnuHsUzSvmL5vhYio77WO4tQY7Stj 5ib1HFdR4eCTHtiqOXgMxeSzu+uFD7x821JJFerpqTbp0NVM3XWr7+Dd2of30ZWcrW Ns0rx2DgKajr/P1IrIpi+Psr0+iN7wNoTpRqJb3fTlSn+olKW1rmwM3J+FqHGFGeKn LbmSI97Tb5BPwNxXUua4k87Qrb9GvNU1PpENIefiFoH/XDk/Bu9c9WFugGZ2Wemuy6 rEatxO6tij93gD1JLFGJLMZO+3T1j8U/+Z4KN94fsB2k9cu0zofdYoPM+Y8kfRGxK1 u0OWdj9Dc/JJg== Received: from mchehab by mail.kernel.org with local (Exim 4.98) (envelope-from ) id 1tkfVi-0000000Gv5Z-1yYk; Wed, 19 Feb 2025 09:32:50 +0100 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , "Mauro Carvalho Chehab" , linux-kernel@vger.kernel.org Subject: [PATCH 20/27] scripts/kernel-doc.py: fix line number output Date: Wed, 19 Feb 2025 09:32:36 +0100 Message-ID: X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" With the Pyhton version, the actual output happens after parsing, from records stored at self.entries. Ensure that line numbers will be properly stored there and that they'll produce the desired results at the ReST output. Signed-off-by: Mauro Carvalho Chehab --- scripts/lib/kdoc/kdoc_output.py | 13 +++++++------ scripts/lib/kdoc/kdoc_parser.py | 21 +++++++++++++++++---- 2 files changed, 24 insertions(+), 10 deletions(-) diff --git a/scripts/lib/kdoc/kdoc_output.py b/scripts/lib/kdoc/kdoc_output= .py index 8729dc58e13c..b9b39bc29463 100755 --- a/scripts/lib/kdoc/kdoc_output.py +++ b/scripts/lib/kdoc/kdoc_output.py @@ -254,7 +254,8 @@ class RestFormat(OutputFormat): def print_lineno(self, ln): """Outputs a line number""" =20 - if self.enable_lineno and ln: + if self.enable_lineno and ln is not None: + ln +=3D 1 self.data +=3D f".. LINENO {ln}\n" =20 def output_highlight(self, args): @@ -357,7 +358,7 @@ class RestFormat(OutputFormat): parameterdescs =3D args.get('parameterdescs', {}) parameterdesc_start_lines =3D args.get('parameterdesc_start_lines'= , {}) =20 - ln =3D args.get('ln', 0) + ln =3D args.get('declaration_start_line', 0) =20 count =3D 0 for parameter in parameterlist: @@ -374,11 +375,11 @@ class RestFormat(OutputFormat): if not func_macro: signature +=3D ")" =20 + self.print_lineno(ln) if args.get('typedef') or not args.get('functiontype'): self.data +=3D f".. c:macro:: {args['function']}\n\n" =20 if args.get('typedef'): - self.print_lineno(ln) self.data +=3D " **Typedef**: " self.lineprefix =3D "" self.output_highlight(args.get('purpose', "")) @@ -433,7 +434,7 @@ class RestFormat(OutputFormat): name =3D args.get('enum', '') parameterlist =3D args.get('parameterlist', []) parameterdescs =3D args.get('parameterdescs', {}) - ln =3D args.get('ln', 0) + ln =3D args.get('declaration_start_line', 0) =20 self.data +=3D f"\n\n.. c:enum:: {name}\n\n" =20 @@ -463,7 +464,7 @@ class RestFormat(OutputFormat): =20 oldprefix =3D self.lineprefix name =3D args.get('typedef', '') - ln =3D args.get('ln', 0) + ln =3D args.get('declaration_start_line', 0) =20 self.data +=3D f"\n\n.. c:type:: {name}\n\n" =20 @@ -483,7 +484,7 @@ class RestFormat(OutputFormat): purpose =3D args.get('purpose', "") declaration =3D args.get('definition', "") dtype =3D args.get('type', "struct") - ln =3D args.get('ln', 0) + ln =3D args.get('declaration_start_line', 0) =20 parameterlist =3D args.get('parameterlist', []) parameterdescs =3D args.get('parameterdescs', {}) diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser= .py index 6d6395e32093..633c95164b0c 100755 --- a/scripts/lib/kdoc/kdoc_parser.py +++ b/scripts/lib/kdoc/kdoc_parser.py @@ -275,7 +275,7 @@ class KernelDoc: self.entry.brcount =3D 0 =20 self.entry.in_doc_sect =3D False - self.entry.declaration_start_line =3D ln + self.entry.declaration_start_line =3D ln + 1 =20 def push_parameter(self, ln, decl_type, param, dtype, org_arg, declaration_name): @@ -805,8 +805,10 @@ class KernelDoc: parameterlist=3Dself.entry.parameterlist, parameterdescs=3Dself.entry.parameterdescs, parametertypes=3Dself.entry.parametertypes, + parameterdesc_start_lines=3Dself.entry.par= ameterdesc_start_lines, sectionlist=3Dself.entry.sectionlist, sections=3Dself.entry.sections, + section_start_lines=3Dself.entry.section_s= tart_lines, purpose=3Dself.entry.declaration_purpose) =20 def dump_enum(self, ln, proto): @@ -881,8 +883,10 @@ class KernelDoc: module=3Dself.config.modulename, parameterlist=3Dself.entry.parameterlist, parameterdescs=3Dself.entry.parameterdescs, + parameterdesc_start_lines=3Dself.entry.par= ameterdesc_start_lines, sectionlist=3Dself.entry.sectionlist, sections=3Dself.entry.sections, + section_start_lines=3Dself.entry.section_s= tart_lines, purpose=3Dself.entry.declaration_purpose) =20 def dump_declaration(self, ln, prototype): @@ -1053,8 +1057,10 @@ class KernelDoc: parameterlist=3Dself.entry.parameterli= st, parameterdescs=3Dself.entry.parameterd= escs, parametertypes=3Dself.entry.parametert= ypes, + parameterdesc_start_lines=3Dself.entry= .parameterdesc_start_lines, sectionlist=3Dself.entry.sectionlist, sections=3Dself.entry.sections, + section_start_lines=3Dself.entry.secti= on_start_lines, purpose=3Dself.entry.declaration_purpo= se, func_macro=3Dfunc_macro) else: @@ -1066,8 +1072,10 @@ class KernelDoc: parameterlist=3Dself.entry.parameterli= st, parameterdescs=3Dself.entry.parameterd= escs, parametertypes=3Dself.entry.parametert= ypes, + parameterdesc_start_lines=3Dself.entry= .parameterdesc_start_lines, sectionlist=3Dself.entry.sectionlist, sections=3Dself.entry.sections, + section_start_lines=3Dself.entry.secti= on_start_lines, purpose=3Dself.entry.declaration_purpo= se, func_macro=3Dfunc_macro) =20 @@ -1111,8 +1119,10 @@ class KernelDoc: parameterlist=3Dself.entry.parameterli= st, parameterdescs=3Dself.entry.parameterd= escs, parametertypes=3Dself.entry.parametert= ypes, + parameterdesc_start_lines=3Dself.entry= .parameterdesc_start_lines, sectionlist=3Dself.entry.sectionlist, sections=3Dself.entry.sections, + section_start_lines=3Dself.entry.secti= on_start_lines, purpose=3Dself.entry.declaration_purpo= se) return =20 @@ -1135,6 +1145,7 @@ class KernelDoc: module=3Dself.entry.modulename, sectionlist=3Dself.entry.sectionlist, sections=3Dself.entry.sections, + section_start_lines=3Dself.entry.secti= on_start_lines, purpose=3Dself.entry.declaration_purpo= se) return =20 @@ -1167,7 +1178,7 @@ class KernelDoc: return =20 # start a new entry - self.reset_state(ln + 1) + self.reset_state(ln) self.entry.in_doc_sect =3D False =20 # next line is always the function name @@ -1280,7 +1291,7 @@ class KernelDoc: if r.match(line): self.dump_section() self.entry.section =3D self.section_default - self.entry.new_start_line =3D line + self.entry.new_start_line =3D ln self.entry.contents =3D "" =20 if doc_sect.search(line): @@ -1618,7 +1629,9 @@ class KernelDoc: self.dump_section() self.output_declaration("doc", None, sectionlist=3Dself.entry.sectionlist, - sections=3Dself.entry.sections, module= =3Dself.config.modulename) + sections=3Dself.entry.sections, + section_start_lines=3Dself.entry.secti= on_start_lines, + module=3Dself.config.modulename) self.reset_state(ln) =20 elif doc_content.search(line): --=20 2.48.1 From nobody Sat Feb 7 14:01:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C840B1DE2A1; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; cv=none; b=lXbPFtRe9K8MCPJyLoJpcd9lXs97QIUy2TvOGUq9VBu9eC5zv5PaFMZTht5e6UV838QPA8lUUx6zCYx6dSnIWZNS4HGWErdH023ayTrWNpEeLQS7q8lKA/eiiO5jYCZuRS6POIVzW4CFsmGC1tcZWGWWv7M56LgHoXGkE3rglN4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; c=relaxed/simple; bh=AdyCuuoiDzM5JN3vT/qTkSEQUe1tw1eDWso5VHl1JSo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KprVbCJDbDk9SWlGzJPBcyyFrs/vqv2EL/4+Q6o5gawp54uv1opSkQDbd8TyDeKUodGZOnfG7m4efDDgh7LtqJQv502rgF9A/yxSKeU+UtlVp9c+H/CA3uKrnwMDIb3g6sh2OvCrwdq7fbdvbHmy6skLCSWl7btTReX8NSkGVcs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=KPdwBkfc; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="KPdwBkfc" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5D478C4CEEE; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739953972; bh=AdyCuuoiDzM5JN3vT/qTkSEQUe1tw1eDWso5VHl1JSo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=KPdwBkfcMGTjVLwOcNohEu549uigbIRfrM+HN6JWK/tSGFmrRxuw5WZBBTXaQUhFm 4kyqzjnI6fTGVaP7eZ/ZGjwnuxCWU+0NNQNWVQoM8/JthR9607FS5mYl/o1gJCJL25 YAHtjfHrPyXCX/pWWPII7NtWYRdlIAwXcy1rTxseRoYq91vKDHBKlSNBm98B0ccSQU o3p1xejfG34SjGbEB1w0m4+V1ruGhY1/E5+hQFbmJ8ReWaKm8Fs8ttbFI1HXgYzH20 TnzhtCpg9DaXKLTVR92u8OGRSAs1eSTaou8VQjrr9D+EB5XaHvZk26DJR4iIB8tuip /IpoJ+DT07HEQ== Received: from mchehab by mail.kernel.org with local (Exim 4.98) (envelope-from ) id 1tkfVi-0000000Gv5d-25L5; Wed, 19 Feb 2025 09:32:50 +0100 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , "Mauro Carvalho Chehab" , linux-kernel@vger.kernel.org Subject: [PATCH 21/27] scripts/kernel-doc.py: fix handling of doc output check Date: Wed, 19 Feb 2025 09:32:37 +0100 Message-ID: X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" The filtering logic was seeking for the DOC name to check for symbols, but such data is stored only inside a section. Add it to the output_declaration, as it is quicker/easier to check the declaration name than to check inside each section. While here, make sure that the output for both ReST and man after filtering will be similar to what kernel-doc Perl version does. Signed-off-by: Mauro Carvalho Chehab --- scripts/lib/kdoc/kdoc_output.py | 29 ++++++++++++----------------- scripts/lib/kdoc/kdoc_parser.py | 3 ++- 2 files changed, 14 insertions(+), 18 deletions(-) diff --git a/scripts/lib/kdoc/kdoc_output.py b/scripts/lib/kdoc/kdoc_output= .py index b9b39bc29463..6a392dad2e9d 100755 --- a/scripts/lib/kdoc/kdoc_output.py +++ b/scripts/lib/kdoc/kdoc_output.py @@ -121,13 +121,13 @@ class OutputFormat: if self.no_doc_sections: return False =20 + if name in self.nosymbol: + return False + if self.out_mode =3D=3D self.OUTPUT_ALL: return True =20 if self.out_mode =3D=3D self.OUTPUT_INCLUDE: - if name in self.nosymbol: - return False - if name in self.function_table: return True =20 @@ -153,15 +153,6 @@ class OutputFormat: =20 return False =20 - def check_function(self, fname, name, args): - return True - - def check_enum(self, fname, name, args): - return True - - def check_typedef(self, fname, name, args): - return True - def msg(self, fname, name, args): self.data =3D "" =20 @@ -305,7 +296,7 @@ class RestFormat(OutputFormat): for line in output.strip("\n").split("\n"): self.data +=3D self.lineprefix + line + "\n" =20 - def out_section(self, args, out_reference=3DFalse): + def out_section(self, args, out_docblock=3DFalse): """ Outputs a block section. =20 @@ -324,7 +315,7 @@ class RestFormat(OutputFormat): continue =20 if not self.out_mode =3D=3D self.OUTPUT_INCLUDE: - if out_reference: + if out_docblock: self.data +=3D f".. _{section}:\n\n" =20 if not self.symbol: @@ -338,8 +329,7 @@ class RestFormat(OutputFormat): def out_doc(self, fname, name, args): if not self.check_doc(name): return - - self.out_section(args, out_reference=3DTrue) + self.out_section(args, out_docblock=3DTrue) =20 def out_function(self, fname, name, args): =20 @@ -582,8 +572,10 @@ class ManFormat(OutputFormat): =20 for line in contents.strip("\n").split("\n"): line =3D Re(r"^\s*").sub("", line) + if not line: + continue =20 - if line and line[0] =3D=3D ".": + if line[0] =3D=3D ".": self.data +=3D "\\&" + line + "\n" else: self.data +=3D line + "\n" @@ -593,6 +585,9 @@ class ManFormat(OutputFormat): sectionlist =3D args.get('sectionlist', []) sections =3D args.get('sections', {}) =20 + if not self.check_doc(name): + return + self.data +=3D f'.TH "{module}" 9 "{module}" "{self.man_date}" "AP= I Manual" LINUX' + "\n" =20 for section in sectionlist: diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser= .py index 633c95164b0c..116289622f2c 100755 --- a/scripts/lib/kdoc/kdoc_parser.py +++ b/scripts/lib/kdoc/kdoc_parser.py @@ -1197,6 +1197,7 @@ class KernelDoc: else: self.entry.section =3D doc_block.group(1) =20 + self.entry.identifier =3D self.entry.section self.state =3D self.STATE_DOCBLOCK return =20 @@ -1627,7 +1628,7 @@ class KernelDoc: =20 if doc_end.search(line): self.dump_section() - self.output_declaration("doc", None, + self.output_declaration("doc", self.entry.identifier, sectionlist=3Dself.entry.sectionlist, sections=3Dself.entry.sections, section_start_lines=3Dself.entry.secti= on_start_lines, --=20 2.48.1 From nobody Sat Feb 7 14:01:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8A8FF1D79BE; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; cv=none; b=gXO8dYMHourRZXlzA2Tp5qC660OX8+tqbwmrW3gVYBcxS2wB1U9q3KQUFNrO8VNMefcCUC2lV042sv3D+AobDznGOvePFM4hhBwTeuZAzhtRCglvEoOUtRYjc6L1pWOWA+O4rYrWNBmgt3+h4bSmeM5H0TwrOnjtzHHrV6Tb1QI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; c=relaxed/simple; bh=NkNgWuYdW6yz0X6IJafkeOcfMUGHd/fARN30vGbl73o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=io1fjB2jd29vezDKSl35JeBdKgERgshl8O7ObG3xINSjp0HEs4KPnAPjxnL2WF3tTvXZYuae/+Rp0iQ6Cf7ZarTgKzGmIPC3FvdG01/e09wpCw+kPhJOzM6LYhcEqnTpHW7wTeDiNfb2wsa9NH4CAZt3GYxrEze4xMjrrnJjTiQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=RJ8Cz7++; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="RJ8Cz7++" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 67349C16AAE; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739953972; bh=NkNgWuYdW6yz0X6IJafkeOcfMUGHd/fARN30vGbl73o=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RJ8Cz7++Yjt3Ig9koGZyODHpnN6t2J1URjuM/m9BR8NAqW1mkgbYKZCg/QztwQvDG N72YASEoYfL7BQz2D1BAacJ/J/NNzVZ26DiA0XyIgGDCSUCBg2cekvs1D4AK6c+XNW DNxSAmorG5Sjev0UXRbTjbbpEgiq9/oRhDcbVH4nvHehPnUxJfJ1lVpb93XFdOfB/f n9SsIn5qJ7pa/an2qc6GX0s/Hf6hqQ4JM+sYA00KN+Z659VHtU8T2ST+GtDoj+WEKQ v/SniB3WtMNz6K8vz2OrjOlgYs/rW7tRWlpdoMH6jN8im9bkcU1hCC7beYVCOR9w9H 6tjpQYQigpgSA== Received: from mchehab by mail.kernel.org with local (Exim 4.98) (envelope-from ) id 1tkfVi-0000000Gv5h-2CAT; Wed, 19 Feb 2025 09:32:50 +0100 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , "Mauro Carvalho Chehab" , linux-kernel@vger.kernel.org Subject: [PATCH 22/27] scripts/kernel-doc.py: properly handle out_section for ReST Date: Wed, 19 Feb 2025 09:32:38 +0100 Message-ID: <9186a117f52b6875f61ec323159f2acf9a53464b.1739952783.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" There is a difference at the way DOC sections are output with the include mode. Handle such difference properly. Signed-off-by: Mauro Carvalho Chehab --- scripts/lib/kdoc/kdoc_output.py | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/scripts/lib/kdoc/kdoc_output.py b/scripts/lib/kdoc/kdoc_output= .py index 6a392dad2e9d..ca21cd856be4 100755 --- a/scripts/lib/kdoc/kdoc_output.py +++ b/scripts/lib/kdoc/kdoc_output.py @@ -314,12 +314,12 @@ class RestFormat(OutputFormat): if section in self.nosymbol: continue =20 - if not self.out_mode =3D=3D self.OUTPUT_INCLUDE: - if out_docblock: + if out_docblock: + if not self.out_mode =3D=3D self.OUTPUT_INCLUDE: self.data +=3D f".. _{section}:\n\n" - - if not self.symbol: self.data +=3D f'{self.lineprefix}**{section}**\n\n' + else: + self.data +=3D f'{self.lineprefix}**{section}**\n\n' =20 self.print_lineno(section_start_lines.get(section, 0)) self.output_highlight(sections[section]) --=20 2.48.1 From nobody Sat Feb 7 14:01:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8A8841D6DC5; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; cv=none; b=o6q1yJog+st9N47wFFeBUOLEDeYtvuNNxydhvU+vnCGJN2l3bKzNqPJvkSovivR4HILDyXHfG/foa3qUzkwQs7oVgHqm34stIouRWdHNK4KZYHvSyAaUY5gFM7BokvaG5kyzU1o2O0Kuqkm2xrKVM0FWkx2aufAGuhrpeAg8hL8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; c=relaxed/simple; bh=f3hhbbpUGqP6w/O5kQrl+GO4MbXaX4mT/0fXNELBwgs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dS7jtoSyiXpetlfc+L/U437Ts3dxHce8nRUZysziLn+IT2U550l24AdTXTNjWlyXAWebsXB1866Kuc+RIgLbme08Fj5KXFXYfI/PAuBR2CcQH/w4wp5raQHpU3Pf4G4VhBDGJF96+w6hT09m5cHeYr6igkR0TpwVqZ3iFmhNqFk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=U5iyjG86; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="U5iyjG86" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 65FA7C4CEE8; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739953972; bh=f3hhbbpUGqP6w/O5kQrl+GO4MbXaX4mT/0fXNELBwgs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=U5iyjG86twxw56LAXzUw97eNK6TNr2EOJnwhaROTOENZIwQF9ihxtZ8W8p/VXXnk/ NQvdZrHe2l8/P2cXsJLdFIrVB+cqwxIQYKAWji1gvbTpkJTf5FMQizy4KSGM4GvyOE XHeOKQXHwsuKVBdPYaTmRI1mPCPa/Hb2fWgqSY+/0QdRw2HNOHwAeLlwFCit0nglXa AhK56Znc3tWGDj0d5v8RCSSIuV3yaIIaoT6/Fiv1SfnoXnQp0QDRIQlxTwUo5Q72R1 l1qGzNJUcJtGH0qqD+fepFyA/S+q869IRYo+2fkdlCRoojFPpn2tlAMrxTYiTZDBQa 53mdjlKkwwTFw== Received: from mchehab by mail.kernel.org with local (Exim 4.98) (envelope-from ) id 1tkfVi-0000000Gv5l-2IxC; Wed, 19 Feb 2025 09:32:50 +0100 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , "Mauro Carvalho Chehab" , linux-kernel@vger.kernel.org Subject: [PATCH 23/27] scripts/kernel-doc.py: postpone warnings to the output plugin Date: Wed, 19 Feb 2025 09:32:39 +0100 Message-ID: <8aecc23f0c00f69b36ade8be7ceab880b2f26b20.1739952783.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" We don't want to have warnings displayed for symbols that weren't output. So, postpone warnings print to the output plugin, where symbol output is validated. Signed-off-by: Mauro Carvalho Chehab --- scripts/lib/kdoc/kdoc_output.py | 24 +++++++++++++++---- scripts/lib/kdoc/kdoc_parser.py | 41 ++++++++++++++++----------------- 2 files changed, 39 insertions(+), 26 deletions(-) diff --git a/scripts/lib/kdoc/kdoc_output.py b/scripts/lib/kdoc/kdoc_output= .py index ca21cd856be4..7aeaec884545 100755 --- a/scripts/lib/kdoc/kdoc_output.py +++ b/scripts/lib/kdoc/kdoc_output.py @@ -115,7 +115,16 @@ class OutputFormat: =20 return block =20 - def check_doc(self, name): + def out_warnings(self, args): + warnings =3D args.get('warnings', []) + + for warning, log_msg in warnings: + if warning: + self.config.log.warning(log_msg) + else: + self.config.log.info(log_msg) + + def check_doc(self, name, args): """Check if DOC should be output""" =20 if self.no_doc_sections: @@ -125,19 +134,22 @@ class OutputFormat: return False =20 if self.out_mode =3D=3D self.OUTPUT_ALL: + self.out_warnings(args) return True =20 if self.out_mode =3D=3D self.OUTPUT_INCLUDE: if name in self.function_table: + self.out_warnings(args) return True =20 return False =20 - def check_declaration(self, dtype, name): + def check_declaration(self, dtype, name, args): if name in self.nosymbol: return False =20 if self.out_mode =3D=3D self.OUTPUT_ALL: + self.out_warnings(args) return True =20 if self.out_mode in [self.OUTPUT_INCLUDE, self.OUTPUT_EXPORTED]: @@ -146,9 +158,11 @@ class OutputFormat: =20 if self.out_mode =3D=3D self.OUTPUT_INTERNAL: if dtype !=3D "function": + self.out_warnings(args) return True =20 if name not in self.function_table: + self.out_warnings(args) return True =20 return False @@ -162,7 +176,7 @@ class OutputFormat: self.out_doc(fname, name, args) return self.data =20 - if not self.check_declaration(dtype, name): + if not self.check_declaration(dtype, name, args): return self.data =20 if dtype =3D=3D "function": @@ -327,7 +341,7 @@ class RestFormat(OutputFormat): self.data +=3D "\n" =20 def out_doc(self, fname, name, args): - if not self.check_doc(name): + if not self.check_doc(name, args): return self.out_section(args, out_docblock=3DTrue) =20 @@ -585,7 +599,7 @@ class ManFormat(OutputFormat): sectionlist =3D args.get('sectionlist', []) sections =3D args.get('sections', {}) =20 - if not self.check_doc(name): + if not self.check_doc(name, args): return =20 self.data +=3D f'.TH "{module}" 9 "{module}" "{self.man_date}" "AP= I Manual" LINUX' + "\n" diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser= .py index 116289622f2c..a71145d531f2 100755 --- a/scripts/lib/kdoc/kdoc_parser.py +++ b/scripts/lib/kdoc/kdoc_parser.py @@ -130,23 +130,23 @@ class KernelDoc: # Place all potential outputs into an array self.entries =3D [] =20 - def show_warnings(self, dtype, declaration_name): # pylint: disable= =3DW0613 - """ - Allow filtering out warnings - """ - - # TODO: implement it - - return True - # TODO: rename to emit_message def emit_warning(self, ln, msg, warning=3DTrue): """Emit a message""" =20 + log_msg =3D f"{self.fname}:{ln} {msg}" + + if self.entry: + # Delegate warning output to output logic, as this way it + # will report warnings/info only for symbols that are output + + self.entry.warnings.append((warning, log_msg)) + return + if warning: - self.config.log.warning("%s:%d %s", self.fname, ln, msg) + self.config.log.warning(log_msg) else: - self.config.log.info("%s:%d %s", self.fname, ln, msg) + self.config.log.info(log_msg) =20 def dump_section(self, start_new=3DTrue): """ @@ -220,10 +220,9 @@ class KernelDoc: # For now, we're keeping the same name of the function just to make # easier to compare the source code of both scripts =20 - if "declaration_start_line" not in args: - args["declaration_start_line"] =3D self.entry.declaration_star= t_line - + args["declaration_start_line"] =3D self.entry.declaration_start_li= ne args["type"] =3D dtype + args["warnings"] =3D self.entry.warnings =20 # TODO: use colletions.OrderedDict =20 @@ -256,6 +255,8 @@ class KernelDoc: self.entry.struct_actual =3D "" self.entry.prototype =3D "" =20 + self.entry.warnings =3D [] + self.entry.parameterlist =3D [] self.entry.parameterdescs =3D {} self.entry.parametertypes =3D {} @@ -327,7 +328,7 @@ class KernelDoc: if param not in self.entry.parameterdescs and not param.startswith= ("#"): self.entry.parameterdescs[param] =3D self.undescribed =20 - if self.show_warnings(dtype, declaration_name) and "." not in = param: + if "." not in param: if decl_type =3D=3D 'function': dname =3D f"{decl_type} parameter" else: @@ -867,16 +868,14 @@ class KernelDoc: self.entry.parameterlist.append(arg) if arg not in self.entry.parameterdescs: self.entry.parameterdescs[arg] =3D self.undescribed - if self.show_warnings("enum", declaration_name): - self.emit_warning(ln, - f"Enum value '{arg}' not described i= n enum '{declaration_name}'") + self.emit_warning(ln, + f"Enum value '{arg}' not described in en= um '{declaration_name}'") member_set.add(arg) =20 for k in self.entry.parameterdescs: if k not in member_set: - if self.show_warnings("enum", declaration_name): - self.emit_warning(ln, - f"Excess enum value '%{k}' descripti= on in '{declaration_name}'") + self.emit_warning(ln, + f"Excess enum value '%{k}' description i= n '{declaration_name}'") =20 self.output_declaration('enum', declaration_name, enum=3Ddeclaration_name, --=20 2.48.1 From nobody Sat Feb 7 14:01:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F0C721DE2D8; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953973; cv=none; b=ERemHr3p483JPhN3/IP7b2olNel0ytHWoyAuDAZ4R363PYyU/iwfPO8DnetZj1gfH8+/+edPeylPODhk5K7/xBYyqyhw4FCuxhLHvBhgp4OA0i7rb4ApruvxgBhxRg0f4gNOMQvj/3rEDjyPN3xpgUYQkzXG/9dvvcGAFK0op8s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953973; c=relaxed/simple; bh=pOTBzlS7I7jZeyW/hygW6cvMdWH+uYsFy5DDt8PknnA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=toEHYmhrAve7m9ky5MK+ZtqNuKO2C2bOrAZYGuWfP0T1qmNDmcjWT4Lk7nltTxBOfGY876/kS5YBzono6BB06B5m240zqMpuyk8d9pI67duZDmLY4ASDSbsN1EZoDZdCqMP8EHkbBjVPX1z+EuRZ3qG6jKh8keV3ngMVJwWtV/8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=F1mpJ6FJ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="F1mpJ6FJ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7C457C4CEF0; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739953972; bh=pOTBzlS7I7jZeyW/hygW6cvMdWH+uYsFy5DDt8PknnA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=F1mpJ6FJHxbUhASHnwQfWob9BvW5TuOYs/6znrgkEnCD6oBJXTeAY05PQnRqvhBF2 taaZ9/NFjIMWS/ahOwbsfK508IXyiViwOm5LSrDla9bdc5sNQZn6jPyedhZSuuOWZN 7gbAmU6ILUbM97LJx/gKOhVKm7/CCnGWqkKNaiCeun/RVjHGpzjbJqMTrNwDHGJa1U VSlgCqJhENpB2h2H1a7V3SFZay1yB0Zfq912g+uXS+baaQTOrf2f+WutuyeAe4RqlO c9pdn/cJrgoaXS6ViW/Qf6D586qdmPPPAQVnnTCa6cABf1gkAPZqyMGgP9PpBnXEqi 1a3gaz2a8cBAw== Received: from mchehab by mail.kernel.org with local (Exim 4.98) (envelope-from ) id 1tkfVi-0000000Gv5p-2PeJ; Wed, 19 Feb 2025 09:32:50 +0100 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , "Mauro Carvalho Chehab" , linux-kernel@vger.kernel.org Subject: [PATCH 24/27] docs: add a .pylintrc file with sys path for docs scripts Date: Wed, 19 Feb 2025 09:32:40 +0100 Message-ID: <7cfc76bb53197582b1367976c08ee1364c996a42.1739952783.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" The docs scripts that are used by Documentation/sphinx are using scripts/lib/* directories to place classes that will be used by both extensions and scripts. When pylint is used, it needs to identify the path where such scripts are, otherwise it will bail out. Add a simple RC file placing the location of such files. Signed-off-by: Mauro Carvalho Chehab --- .pylintrc | 2 ++ 1 file changed, 2 insertions(+) create mode 100644 .pylintrc diff --git a/.pylintrc b/.pylintrc new file mode 100644 index 000000000000..30b8ae1659f8 --- /dev/null +++ b/.pylintrc @@ -0,0 +1,2 @@ +[MASTER] +init-hook=3D'import sys; sys.path +=3D ["scripts/lib/kdoc", "scripts/lib/a= bi"]' --=20 2.48.1 From nobody Sat Feb 7 14:01:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F33AA1DE3A7; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953973; cv=none; b=ZlP40/fKUd2RSQ2g+NqXK0tMeIXnoX8vvHBYB9KuBfmFP+M+4uIB1HiNC1+CDfxSNNaqgISDzR+jBbdCpmcLJRp87ETiw7x/z66XsaGZHdm1FMapdshRcrH36sU8V33bimCvIEL8wu18s7v1HV5zL+NpgdyyUFqtO8aiBanvh0E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953973; c=relaxed/simple; bh=visFeBdmwWEXGD09mdOdO30j7sy6oK5qUh63NaeMIjk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Hho270Ftm3EkP4IUI7Nbobp5e+6IDlRR95/dCqi51NWaHTygyEsKbsLawpkEiavCl+gusaHWaKnIOM6sxDfEPk5cuhPE2ljpkIlvDE1rmceMDrWcJAweQxhgUaqFySw+L8EIYN4jrwIvb8Kp0nS5ckajSclY6sWK3djm9WnV844= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=MbHE4clH; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="MbHE4clH" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A053DC2BCB4; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739953972; bh=visFeBdmwWEXGD09mdOdO30j7sy6oK5qUh63NaeMIjk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MbHE4clHSD3016HjRq0ep1cKrf+kilXVk26nNJ5USFIAnEgE/bhd6umy5uhWpunZu KEKQS9Ef3sBED1doregIU8M3LmoK7SMtC2DUcKwu5kN9HJR7CJ76/Y16xuu+9+RYg3 DCOIosWmFIAX367boDRkj1QmFuQQIDd2jKOwdK9yaEYBWrnwbi2PHydTWSOx5dLQL/ lEVoWF38Fmsj6LrEklz9Y4nJShwMOO4PhgluUMWMBQA3mHVAIMg3njOQ+5TnMXO2Js +PNVQCgXNlVEqv9Jw3GzvmLIHjJGR95wyjcJQFxJe7gjty6c2Z+/i5OQOfNFSQfNnX p8Nmo6RJho+RQ== Received: from mchehab by mail.kernel.org with local (Exim 4.98) (envelope-from ) id 1tkfVi-0000000Gv5t-2WX3; Wed, 19 Feb 2025 09:32:50 +0100 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , "Mauro Carvalho Chehab" , Kees Cook , Randy Dunlap , Vegard Nossum , linux-kernel@vger.kernel.org Subject: [PATCH 25/27] docs: sphinx: kerneldoc: verbose kernel-doc command if V=1 Date: Wed, 19 Feb 2025 09:32:41 +0100 Message-ID: <6d7af20038b540c30d53e2b214f1985b57a4e26f.1739952783.git.mchehab+huawei@kernel.org> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" It is useful to know what kernel-doc command was used during document build time, as it allows one to check the output the same way as Sphinx extension does. Signed-off-by: Mauro Carvalho Chehab --- Documentation/sphinx/kerneldoc.py | 34 +++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/Documentation/sphinx/kerneldoc.py b/Documentation/sphinx/kerne= ldoc.py index 39ddae6ae7dd..d206eb2be10a 100644 --- a/Documentation/sphinx/kerneldoc.py +++ b/Documentation/sphinx/kerneldoc.py @@ -43,6 +43,29 @@ from sphinx.util import logging =20 __version__ =3D '1.0' =20 +def cmd_str(cmd): + """ + Helper function to output a command line that can be used to produce + the same records via command line. Helpful to debug troubles at the + script. + """ + + cmd_line =3D "" + + for w in cmd: + if w =3D=3D "" or " " in w: + esc_cmd =3D "'" + w + "'" + else: + esc_cmd =3D w + + if cmd_line: + cmd_line +=3D " " + esc_cmd + continue + else: + cmd_line =3D esc_cmd + + return cmd_line + class KernelDocDirective(Directive): """Extract kernel-doc comments from the specified file""" required_argument =3D 1 @@ -57,6 +80,7 @@ class KernelDocDirective(Directive): } has_content =3D False logger =3D logging.getLogger('kerneldoc') + verbose =3D 0 =20 def run(self): env =3D self.state.document.settings.env @@ -65,6 +89,13 @@ class KernelDocDirective(Directive): filename =3D env.config.kerneldoc_srctree + '/' + self.arguments[0] export_file_patterns =3D [] =20 + verbose =3D os.environ.get("V") + if verbose: + try: + self.verbose =3D int(verbose) + except ValueError: + pass + # Tell sphinx of the dependency env.note_dependency(os.path.abspath(filename)) =20 @@ -104,6 +135,9 @@ class KernelDocDirective(Directive): =20 cmd +=3D [filename] =20 + if self.verbose >=3D 1: + print(cmd_str(cmd)) + try: self.logger.verbose("calling kernel-doc '%s'" % (" ".join(cmd)= )) =20 --=20 2.48.1 From nobody Sat Feb 7 14:01:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C8BFC1DE2A4; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; cv=none; b=AtkKB/qkclfa1NaJ+n4w9gVh9xMx38AGUx3ejeAYiKmv021bU4NKPaKEqqURwNe8kdRc8kOqA9jabI6iQsABWGpRgqoSyg05+USH/WCYgG/mZTZv1/aloaJ0NQfpEOKyH8UV/sNcvPFd2QRYKJSGx0i3AffqykalrKCRs3R0mcI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; c=relaxed/simple; bh=eHvxtpg5BcZjO8Wx7MxCSiuYJ025kjhp7csGrstaq+8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BvIvOQnlp/Orn/XDb+0zauVlEEJLOgaopv5ItqiRu0/kx4dZ9DASCTdqNhFTF1NGfat8NHl4iKiptQv6tu22xa6O/RFuO8x3MoHtVdKySXQsvlUNNJec+rSk6zGzdk76uQH/sPTcJ5TVeeiO8aEDO49CrXPkiqk097DFnpuLcK8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=bvuZwJBK; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="bvuZwJBK" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 92717C4AF0C; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739953972; bh=eHvxtpg5BcZjO8Wx7MxCSiuYJ025kjhp7csGrstaq+8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bvuZwJBKY+BCzjgyroayxTnMc6oSdlqSxsWIrQN+aYCvuZZpZ+xpiKstXge3ahodk i9dn32GHBiuRwuYalcjI9phMhk7hLgnNEpgeLB8D2V1lvnCd+q+TxVGXKyiQBaOnKi oCnVvUohqCM49/1JgIo+77cDITxMfJH78UquydhU7z3KsKMSifyQeh3P0zyg1Z0Abe 63USo3EgCKKBHKJrfMxFtFKsXvBh7RsTl9Msda397lsRQB6KgsgYYxmQSBVzl6E+AJ gGabKFZ1wectFQAJRrX966r39LPP6aZY8uBCg2A6OtaKTcJYL+7zvG6e3EFhhUlEkB vvOru2BUJwXCw== Received: from mchehab by mail.kernel.org with local (Exim 4.98) (envelope-from ) id 1tkfVi-0000000Gv5x-2dXe; Wed, 19 Feb 2025 09:32:50 +0100 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , "Mauro Carvalho Chehab" , Kees Cook , Randy Dunlap , Vegard Nossum , linux-kernel@vger.kernel.org Subject: [PATCH 26/27] docs: sphinx: kerneldoc: ignore "\" characters from options Date: Wed, 19 Feb 2025 09:32:42 +0100 Message-ID: X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" Documentation/driver-api/infiniband.rst has a kernel-doc tag with "\" characters at the end: .. kernel-doc:: drivers/infiniband/ulp/iser/iscsi_iser.c :functions: iscsi_iser_pdu_alloc iser_initialize_task_headers \ iscsi_iser_task_init iscsi_iser_mtask_xmit iscsi_iser_task_xmit \ iscsi_iser_cleanup_task iscsi_iser_check_protection \ iscsi_iser_conn_create iscsi_iser_conn_bind \ iscsi_iser_conn_start iscsi_iser_conn_stop \ iscsi_iser_session_destroy iscsi_iser_session_create \ iscsi_iser_set_param iscsi_iser_ep_connect iscsi_iser_ep_poll \ iscsi_iser_ep_disconnect This is not handled well, as the "\" strings will be just stored inside Sphinx options. While the actual problem deserves being fixed, better to relax the keneldoc.py extension to silently strip "\" from the end of strings, as otherwise this may cause troubles when preparing arguments to be executed by kernel-doc. Signed-off-by: Mauro Carvalho Chehab --- Documentation/sphinx/kerneldoc.py | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/Documentation/sphinx/kerneldoc.py b/Documentation/sphinx/kerne= ldoc.py index d206eb2be10a..344789ed9ea2 100644 --- a/Documentation/sphinx/kerneldoc.py +++ b/Documentation/sphinx/kerneldoc.py @@ -118,6 +118,10 @@ class KernelDocDirective(Directive): identifiers =3D self.options.get('identifiers').split() if identifiers: for i in identifiers: + i =3D i.rstrip("\\").strip() + if not i: + continue + cmd +=3D ['-function', i] else: cmd +=3D ['-no-doc-sections'] @@ -126,9 +130,17 @@ class KernelDocDirective(Directive): no_identifiers =3D self.options.get('no-identifiers').split() if no_identifiers: for i in no_identifiers: + i =3D i.rstrip("\\").strip() + if not i: + continue + cmd +=3D ['-nosymbol', i] =20 for pattern in export_file_patterns: + pattern =3D pattern.rstrip("\\").strip() + if not pattern: + continue + for f in glob.glob(env.config.kerneldoc_srctree + '/' + patter= n): env.note_dependency(os.path.abspath(f)) cmd +=3D ['-export-file', f] --=20 2.48.1 From nobody Sat Feb 7 14:01:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CA9EF1DE2A6; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; cv=none; b=evCJ8c3vwIY8bKyKTcLvccDc633x4HJSdk5dzIlKPf4dETl30Mn4uNzoCxPg9XU5n17K7d9aCthBRsJqeNwGqn7phADSsOYsvXWERAtS2SJj2uxrcsa9ZWNFPnhyd2/wtiQ8UXZFjOKHkE+isR52ciPvzBn9dAGoq1IpiBuRla0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739953972; c=relaxed/simple; bh=nv2FWo3LGyCll6dWzmoWsroR9XM0kzMdOA2DYqUHwCw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hzkTrmeHyWpzZBEVvqim+Wc9G4P5qDJPJAgq1qLDVwA5fstgMC3g1WrrUY20LfY9wAdU+/NSXsZAbnHMWYy6GWTrlOKTCs4Mlpqwp6tGCbG7D9MNM8F4B/QAMayu7ejCbX6yLT5fLY3mUREvmNEWhTJ8R65FWRW536exPKZ4naY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ny4sZzWe; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ny4sZzWe" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8F50DC4AF0B; Wed, 19 Feb 2025 08:32:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739953972; bh=nv2FWo3LGyCll6dWzmoWsroR9XM0kzMdOA2DYqUHwCw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ny4sZzWeUa7F64amtQrSA95MXHPF6R9tuqquIp1pwYt/84v+gouX65xMFltlGgbe8 FwApyZpv+/7W/ApoAumLjoy97R1VRf6byNrf+H1Ngp1lBO+FYCyTlZbWj3M2iGbjb0 bVf3j+dqN6Gzyr3b8BlmHJUwps2zYywmXqlXTpLTai7B3g2TYMkhu86zOtOWgRC9qI lUKoaHMEzZfMjwa4mllcggFGvtSSJN0HxUIPS6WIRUBCDwaPoWGnYqHQwG3zESPgyM h+g1RjdlahRj4Xb5TeKnKQM0PxYhaxlIxqZk7/BchEvgnCXUnXYcvTj01GqQuZNFVl aYGf7YoLEMTXg== Received: from mchehab by mail.kernel.org with local (Exim 4.98) (envelope-from ) id 1tkfVi-0000000Gv61-2kKE; Wed, 19 Feb 2025 09:32:50 +0100 From: Mauro Carvalho Chehab To: Linux Doc Mailing List , Jonathan Corbet Cc: Mauro Carvalho Chehab , "Mauro Carvalho Chehab" , linux-kernel@vger.kernel.org Subject: [PATCH 27/27] docs: sphinx: kerneldoc: use kernel-doc.py script Date: Wed, 19 Feb 2025 09:32:43 +0100 Message-ID: X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab Content-Type: text/plain; charset="utf-8" Switch to the new version when producing documentation. Signed-off-by: Mauro Carvalho Chehab --- Documentation/Makefile | 2 +- Documentation/conf.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Documentation/Makefile b/Documentation/Makefile index 63094646df28..c022b97c487e 100644 --- a/Documentation/Makefile +++ b/Documentation/Makefile @@ -60,7 +60,7 @@ endif #HAVE_LATEXMK # Internal variables. PAPEROPT_a4 =3D -D latex_paper_size=3Da4 PAPEROPT_letter =3D -D latex_paper_size=3Dletter -KERNELDOC =3D $(srctree)/scripts/kernel-doc +KERNELDOC =3D $(srctree)/scripts/kernel-doc.py KERNELDOC_CONF =3D -D kerneldoc_srctree=3D$(srctree) -D kerneldoc_bin=3D$= (KERNELDOC) ALLSPHINXOPTS =3D $(KERNELDOC_CONF) $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) ifneq ($(wildcard $(srctree)/.config),) diff --git a/Documentation/conf.py b/Documentation/conf.py index 3dad1f90b098..b126f6760b5f 100644 --- a/Documentation/conf.py +++ b/Documentation/conf.py @@ -540,7 +540,7 @@ pdf_documents =3D [ # kernel-doc extension configuration for running Sphinx directly (e.g. by = Read # the Docs). In a normal build, these are supplied from the Makefile via c= ommand # line arguments. -kerneldoc_bin =3D '../scripts/kernel-doc' +kerneldoc_bin =3D '../scripts/kernel-doc.py' kerneldoc_srctree =3D '..' =20 # ------------------------------------------------------------------------= ------ --=20 2.48.1