From nobody Sat May 18 16:18:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1588174745; cv=none; d=zohomail.com; s=zohoarc; b=jqnj+NDjNnHDKyAUqN9I9CtETemTbl69oVpbJT8lj7UVu/0t2dNAmdVXybndZHhIXPGouguVwzHd8Z9lK8vte2+OB/QyFnX1nHN+h5xHANBH3yeJlmbirTA2fQFYhLXMXpBd3EdGMg9fIIoM/IZAlxsl3fyuKnRn1uWQCYNUW0M= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1588174745; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:To; bh=tnUJG1X0ayXfwR1nvayzRVxX8tprQnFECdsWg5e0f4Q=; b=Yx7NRNOchcpZOxXnxRU+HFRKcfvPG9jEeeuxJSpyknIY5jsalJIVUcpga8LU3YnHVaBe5qvHjhpH8H5TIf+BS9GhuW5YNgAJ76/c/bF0HIEp/byrJu+d1xOcQXZcGT6auh+IZYUhJV8GMOpjGz8lA1dSuE8uQDhhUztbuxUN6Bs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 158817474522016.585324159566653; Wed, 29 Apr 2020 08:39:05 -0700 (PDT) Received: from localhost ([::1]:51710 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jTonf-0005tS-Sm for importer@patchew.org; Wed, 29 Apr 2020 11:39:03 -0400 Received: from eggs.gnu.org ([209.51.188.92]:32976) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jTolk-0002dZ-PW for qemu-devel@nongnu.org; Wed, 29 Apr 2020 11:37:08 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.90_1) (envelope-from ) id 1jTolJ-0005yK-DT for qemu-devel@nongnu.org; Wed, 29 Apr 2020 11:37:04 -0400 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:27835 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jTolI-0005y2-Uo for qemu-devel@nongnu.org; Wed, 29 Apr 2020 11:36:37 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-512-CrPub6U4P6KmKMBopEn_pg-1; Wed, 29 Apr 2020 11:36:31 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 55ABABFC2; Wed, 29 Apr 2020 15:36:30 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.36.110.2]) by smtp.corp.redhat.com (Postfix) with ESMTP id ADE626A942; Wed, 29 Apr 2020 15:36:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1588174596; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=tnUJG1X0ayXfwR1nvayzRVxX8tprQnFECdsWg5e0f4Q=; b=ZgM+XIt3usb+PaRg1F3hWqj74y0stayHH5Gj8VrRDHG2i+Lze7qWrjEdzfzaw/+YjwawBO Wmb1gLiqAigaoWQ9USC417mnVT24Nh+dLW7XiXM2VT3Tm7CLNaeXBjSqR/hh5/POvHpc9W Uq+YFMUhHCnTUq+oCguCWMNeVyJw4Zc= X-MC-Unique: CrPub6U4P6KmKMBopEn_pg-1 From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Subject: [PATCH] tests: add a "check-flake8" test for validating python code style Date: Wed, 29 Apr 2020 16:36:21 +0100 Message-Id: <20200429153621.1694266-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.61; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/04/29 00:53:13 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] X-Received-From: 205.139.110.61 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" The flake8 program is a standard tool used by Python projects for validating many commonly recommended style rules. It would be desirable for QEMU to come into alignment with normal Python coding style best practices. QEMU currently violates a huge number of the style rules, so we can't blindly turn it on. Instead this patch introduces use of flake8 with a huge ignore list to turn off everything that is currently violated. The following descriptions are mostly taken from: https://www.flake8rules.com/ Indentation: E111 Indentation is not a multiple of four E114 Indentation is not a multiple of four (comment) E115 Expected an indented block (comment) E116 Unexpected indentation (comment) E117 Over-indented E121 Continuation line under-indented for hanging indent E122 Continuation line missing indentation or outdented E123 Closing bracket does not match indentation of opening bracket's line E124 Closing bracket does not match visual indentation E125 Continuation line with same indent as next logical line E126 Continuation line over-indented for hanging indent E127 Continuation line over-indented for visual indent E128 Continuation line under-indented for visual indent E129 Visually indented line with same indent as next logical line E131 Continuation line unaligned for hanging indent Whitespace: E201 Whitespace after '(' E202 Whitespace before ')' E203 Whitespace before ':' E211 Whitespace before '(' E221 Multiple spaces before operator E222 Multiple spaces after operator E225 Missing whitespace around operator E226 Missing whitespace around arithmetic operator E228 Missing whitespace around modulo operator E231 Missing whitespace after ',', ';', or ':' E241 Multiple spaces after ',' E251 Unexpected spaces around keyword / parameter equals E261 At least two spaces before inline comment E265 Block comment should start with '# ' E266 Too many leading '#' for block comment Blank lines: E301 Expected 1 blank line, found 0 E302 Expected 2 blank lines, found 0 E303 Too many blank lines (3) E305 Expected 2 blank lines after end of function or class E306 Expected 1 blank line before a nested definition Imports: E401 Multiple imports on one line E402 Module level import not at top of file Line length: E501 Line too long (82 > 79 characters) E502 The backslash is redundant between brackets Statements: E701 Multiple statements on one line (colon) E702 Multiple statements on one line (semicolon) E703 Statement ends with a semicolon E711 Comparison to none should be 'if cond is none:' E712 Comparison to true should be 'if cond is true:' or 'if cond:' E713 Test for membership should be 'not in' E714 Test for object identity should be 'is not' E722 Do not use bare 'except' E731 Do not assign a lambda expression, use a def E741 Do not use variables named 'I', 'O', or 'l' Errors: F401 Module imported but unused F403 'from module import *' used; unable to detect undefined names F405 Name may be undefined, or defined from star imports: module F811 Redefinition of unused name from line n F821 Undefined name name F841 Local variable name is assigned to but never used Warnings: W391 Blank line at end of file W503 Line break occurred before a binary operator W504 Line break occurred after a binary operator W605 Invalid escape sequence 'x' Over time code should be fixed and rules removed from the ignore list. A handful of style rules may not warrant fixing as the cure is arguably worse and very subjective. e.g. E501: Force breaking lines at < 80 characters results in some really unnatural code formatting which harms readability. W504: Knuth code style requires the operators "or" and "and" etc to be at the start of line in a multi-line conditional. Signed-off-by: Daniel P. Berrang=C3=A9 --- On its own this patch doesn't really do much of use except try to stop the situation getting worse. To be valuable some motivated contributor(s) would need to go through fixing the code, and re-enabling each excluded warning category one at a time. I'm mostly proposing this patch as a starting point for discussion, to see if anyone is indeed motivated to take on the code cleanup challenge, and feed the fixes in through the various maintainers trees. tests/Makefile.include | 38 ++++++++++++++++++++++++++++++++++++-- 1 file changed, 36 insertions(+), 2 deletions(-) diff --git a/tests/Makefile.include b/tests/Makefile.include index 51de676298..f08e99b09c 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -4,7 +4,7 @@ check-help: @echo "Regression testing targets:" @echo - @echo " $(MAKE) check Run unit, qapi-schema, qtest and dec= odetree" + @echo " $(MAKE) check Run unit, qapi-schema, qtest and dec= odetree flake8" @echo @echo " $(MAKE) check-qtest-TARGET Run qtest tests for given target" @echo " $(MAKE) check-qtest Run qtest tests" @@ -19,6 +19,7 @@ check-help: @echo " $(MAKE) check-report.tap Generates an aggregated TAP test rep= ort" @echo " $(MAKE) check-venv Creates a Python venv for tests" @echo " $(MAKE) check-clean Clean the tests and related data" + @echo " $(MAKE) check-flake8 Clean the tests and related data" @echo @echo " $(MAKE) get-vm-images Downloads all images used by accepta= nce tests, according to configured targets (~350 MB each, 1.5 GB max)" @echo @@ -923,7 +924,40 @@ check-qtest: $(patsubst %,check-qtest-%, $(QTEST_TARGE= TS)) ifeq ($(CONFIG_TOOLS),y) check-block: $(patsubst %,check-%, $(check-block-y)) endif -check: check-block check-qapi-schema check-unit check-softfloat check-qtes= t check-decodetree + +is_git :=3D $(shell test -d $(SRC_PATH)/.git && echo 1 || echo 0) + +ifeq (1, $(is_git)) +PYTHON_FILES =3D $(shell git ls-tree -r HEAD: --name-only | grep '.py$$') +PYTHON_FILES +=3D $(shell git ls-tree -r HEAD: --name-only tests/qemu-iote= sts/??? | xargs grep -l '/usr/bin/env python') + +# Validate many python style rules +FLAKE8_INDENTATION =3D E111,E114,E115,E116,E117,E121,E122,E123,E124,E125,E= 126,E127,E128,E129,E131 +FLAKE8_WHITESPACE =3D E201,E202,E203,E211,E221,E222,E225,E226,E228,E231,E2= 41,E251,E261,E265,E266 +FLAKE8_BLANK_LINES =3D E301,E302,E303,E305,E306 +FLAKE8_IMPORTS =3D E401,E402 +FLAKE8_LINE_LENGTH =3D E501,E502 +FLAKE8_STATEMENTS =3D E701,E702,E703,E711,E712,E713,E714,E722,E731,E741 +FLAKE8_ERRORS =3D F401,F403,F405,F811,F821,F841 +FLAKE8_WARNINGS =3D W391,W503,W504,W605 + +FLAKE8_IGNORE =3D $(FLAKE8_INDENTATION),$\ + $(FLAKE8_WHITESPACE),$\ + $(FLAKE8_BLANK_LINES),$\ + $(FLAKE8_IMPORTS),$\ + $(FLAKE8_LINE_LENGTH),$\ + $(FLAKE8_STATEMENTS),$\ + $(FLAKE8_ERRORS),$\ + $(FLAKE8_WARNINGS) \ + $(NULL) + +check-flake8: + $(call quiet-command,flake8 --ignore=3D$(FLAKE8_IGNORE) $(PYTHON_FILES)) +else +check-flake8: +endif + +check: check-block check-qapi-schema check-unit check-softfloat check-qtes= t check-decodetree check-flake8 check-clean: rm -rf $(check-unit-y) tests/*.o tests/*/*.o $(QEMU_IOTESTS_HELPERS-y) rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(tar= get)-y:%=3Dtests/qtest/%$(EXESUF))) $(check-qtest-generic-y:%=3Dtests/qtest= /%$(EXESUF))) --=20 2.25.4